2024-09-13T17:47:39.934 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T17:47:39.954 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T17:47:40.047 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903942 branch: wip-rishabh-testing-20240910.100730-reef description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/cap-flush} email: null first_in_suite: false job_id: '7903942' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240910.100730-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - Replacing daemon mds.a sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240910.100730-reef sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6439 sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 sleep_before_teardown: 0 subset: 964/1024 suite: fs suite_branch: wip-rishabh-testing-20240910.100730-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi111.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP0JsIhiNS3NXAbO+jMiWn0QT19TOhQQ+r/RePFMw4DAJN0UUT8lX21bUFN1pJtgItbRoSfOJTGKpszyUtFlFg0= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKYIxogs95ToiLZ0PfQzz59A/nPna3cQsJ0F8Md5EE6RRoh1gCwY7RSudkr8Ze6RBagvtNLwfeAhQyIacRAWxvM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_cap_flush teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_14:42:16 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T17:47:40.048 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa; will attempt to use it 2024-09-13T17:47:40.048 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa/tasks 2024-09-13T17:47:40.048 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T17:47:40.050 INFO:teuthology.task.internal:Checking packages... 2024-09-13T17:47:40.073 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3' 2024-09-13T17:47:40.074 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T17:47:40.074 INFO:teuthology.packaging:ref: None 2024-09-13T17:47:40.074 INFO:teuthology.packaging:tag: None 2024-09-13T17:47:40.074 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240910.100730-reef 2024-09-13T17:47:40.074 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:47:40.074 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240910.100730-reef 2024-09-13T17:47:40.228 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-860.g6c6b083f 2024-09-13T17:47:40.230 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T17:47:40.236 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T17:47:40.236 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T17:47:40.281 INFO:teuthology.task.internal:Saving configuration 2024-09-13T17:47:40.296 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T17:47:40.313 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T17:47:40.345 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903942', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 17:42:08.370306', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP0JsIhiNS3NXAbO+jMiWn0QT19TOhQQ+r/RePFMw4DAJN0UUT8lX21bUFN1pJtgItbRoSfOJTGKpszyUtFlFg0='} 2024-09-13T17:47:40.371 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903942', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 17:42:08.368593', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKYIxogs95ToiLZ0PfQzz59A/nPna3cQsJ0F8Md5EE6RRoh1gCwY7RSudkr8Ze6RBagvtNLwfeAhQyIacRAWxvM='} 2024-09-13T17:47:40.371 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T17:47:40.377 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-13T17:47:40.377 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-13T17:47:40.377 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T17:47:40.519 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3167e83be0>, signals=[15]) 2024-09-13T17:47:40.519 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T17:47:40.525 INFO:teuthology.task.internal:Opening connections... 2024-09-13T17:47:40.525 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T17:47:40.526 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:47:40.609 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2024-09-13T17:47:40.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:47:40.692 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T17:47:40.698 DEBUG:teuthology.orchestra.run.smithi111:> uname -m 2024-09-13T17:47:40.716 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2024-09-13T17:47:40.716 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2024-09-13T17:47:40.773 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream" 2024-09-13T17:47:40.774 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="9" 2024-09-13T17:47:40.774 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos" 2024-09-13T17:47:40.774 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora" 2024-09-13T17:47:40.774 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="9" 2024-09-13T17:47:40.774 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el9" 2024-09-13T17:47:40.774 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T17:47:40.774 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2024-09-13T17:47:40.774 INFO:teuthology.orchestra.run.smithi111.stdout:LOGO="fedora-logo-icon" 2024-09-13T17:47:40.774 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T17:47:40.774 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/" 2024-09-13T17:47:40.774 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T17:47:40.774 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T17:47:40.774 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T17:47:40.775 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2024-09-13T17:47:40.805 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2024-09-13T17:47:40.823 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2024-09-13T17:47:40.823 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-09-13T17:47:40.881 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2024-09-13T17:47:40.881 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2024-09-13T17:47:40.882 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2024-09-13T17:47:40.882 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2024-09-13T17:47:40.882 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2024-09-13T17:47:40.882 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2024-09-13T17:47:40.882 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T17:47:40.882 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2024-09-13T17:47:40.882 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2024-09-13T17:47:40.882 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T17:47:40.882 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2024-09-13T17:47:40.882 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T17:47:40.882 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T17:47:40.882 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T17:47:40.883 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-09-13T17:47:40.913 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T17:47:40.921 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T17:47:40.947 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T17:47:40.948 DEBUG:teuthology.orchestra.run.smithi111:> test '!' -e /home/ubuntu/cephtest 2024-09-13T17:47:40.950 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2024-09-13T17:47:40.967 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T17:47:40.997 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T17:47:40.998 DEBUG:teuthology.orchestra.run.smithi111:> test -z $(ls -A /var/lib/ceph) 2024-09-13T17:47:41.007 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2024-09-13T17:47:41.114 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T17:47:41.183 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T17:47:41.218 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T17:47:41.219 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T17:47:41.219 DEBUG:teuthology.orchestra.run.smithi111:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T17:47:41.219 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T17:47:41.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:47:41.237 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2024-09-13T17:47:41.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:47:41.238 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-09-13T17:47:41.296 INFO:teuthology.orchestra.run.smithi111.stdout:5.14.0-503.el9.x86_64 2024-09-13T17:47:41.296 INFO:teuthology.task.kernel:Running kernel on smithi111: 5.14.0-503.el9.x86_64 2024-09-13T17:47:41.296 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum install -y kernel 2024-09-13T17:47:41.298 INFO:teuthology.orchestra.run.smithi188.stdout:5.14.0-503.el9.x86_64 2024-09-13T17:47:41.298 INFO:teuthology.task.kernel:Running kernel on smithi188: 5.14.0-503.el9.x86_64 2024-09-13T17:47:41.298 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum install -y kernel 2024-09-13T17:47:43.590 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-13T17:47:43.600 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-13T17:47:45.318 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-09-13T17:47:45.358 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-13T17:47:49.575 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-13T17:47:49.593 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-13T17:47:51.209 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 19 kB 00:00 2024-09-13T17:47:51.819 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-13T17:47:53.030 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 13 MB/s | 23 MB 00:01 2024-09-13T17:47:53.172 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-13T17:47:58.419 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-13T17:47:58.661 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-13T17:48:00.384 INFO:teuthology.orchestra.run.smithi111.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T17:48:00.384 INFO:teuthology.orchestra.run.smithi111.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T17:48:00.469 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-13T17:48:00.479 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-13T17:48:00.479 INFO:teuthology.orchestra.run.smithi111.stdout: Package Architecture Version Repository Size 2024-09-13T17:48:00.479 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-13T17:48:00.480 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-13T17:48:00.480 INFO:teuthology.orchestra.run.smithi111.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T17:48:00.480 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-09-13T17:48:00.480 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T17:48:00.480 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T17:48:00.480 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T17:48:00.480 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:48:00.480 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-13T17:48:00.480 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-13T17:48:00.480 INFO:teuthology.orchestra.run.smithi111.stdout:Install 4 Packages 2024-09-13T17:48:00.480 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:48:00.481 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 79 M 2024-09-13T17:48:00.481 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 126 M 2024-09-13T17:48:00.481 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-13T17:48:00.659 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T17:48:00.660 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T17:48:00.729 INFO:teuthology.orchestra.run.smithi111.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 186 kB/s | 23 kB 00:00 2024-09-13T17:48:00.741 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-09-13T17:48:00.745 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-09-13T17:48:00.745 INFO:teuthology.orchestra.run.smithi188.stdout: Package Architecture Version Repository Size 2024-09-13T17:48:00.745 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-09-13T17:48:00.745 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-09-13T17:48:00.745 INFO:teuthology.orchestra.run.smithi188.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T17:48:00.745 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-09-13T17:48:00.745 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T17:48:00.746 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T17:48:00.746 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T17:48:00.746 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:48:00.746 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-09-13T17:48:00.746 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-09-13T17:48:00.746 INFO:teuthology.orchestra.run.smithi188.stdout:Install 4 Packages 2024-09-13T17:48:00.746 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:48:00.747 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 79 M 2024-09-13T17:48:00.747 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 126 M 2024-09-13T17:48:00.747 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-09-13T17:48:01.229 INFO:teuthology.orchestra.run.smithi111.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-13T17:48:01.396 INFO:teuthology.orchestra.run.smithi188.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 43 kB/s | 23 kB 00:00 2024-09-13T17:48:01.789 INFO:teuthology.orchestra.run.smithi111.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 29 MB/s | 35 MB 00:01 2024-09-13T17:48:02.255 INFO:teuthology.orchestra.run.smithi111.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 19 MB/s | 29 MB 00:01 2024-09-13T17:48:02.309 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-13T17:48:02.310 INFO:teuthology.orchestra.run.smithi111.stdout:Total 44 MB/s | 79 MB 00:01 2024-09-13T17:48:02.354 INFO:teuthology.orchestra.run.smithi188.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-13T17:48:02.929 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-13T17:48:02.963 INFO:teuthology.orchestra.run.smithi188.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 16 MB/s | 35 MB 00:02 2024-09-13T17:48:03.289 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-13T17:48:03.289 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-13T17:48:03.680 INFO:teuthology.orchestra.run.smithi188.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-13T17:48:03.680 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-09-13T17:48:03.680 INFO:teuthology.orchestra.run.smithi188.stdout:Total 27 MB/s | 79 MB 00:02 2024-09-13T17:48:04.210 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-13T17:48:04.211 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-13T17:48:04.368 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-09-13T17:48:04.740 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-09-13T17:48:04.740 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-09-13T17:48:05.702 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-09-13T17:48:05.703 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-09-13T17:48:06.648 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-13T17:48:07.373 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T17:48:07.455 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:48:08.155 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-09-13T17:48:08.899 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T17:48:08.982 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:48:10.229 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:48:10.349 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:48:11.766 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:48:11.950 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:48:15.153 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:48:15.275 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:48:16.805 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:48:16.991 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:48:20.109 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:48:21.820 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:48:58.135 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:48:58.174 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:48:58.579 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:48:58.579 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T17:48:58.579 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:48:58.579 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:48:59.476 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:48:59.476 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:48:59.476 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-13T17:48:59.476 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T17:48:59.476 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T17:48:59.477 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T17:48:59.477 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T17:48:59.477 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:48:59.477 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-13T17:48:59.828 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T17:48:59.917 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T17:48:59.917 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T17:48:59.918 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T17:48:59.918 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T17:48:59.918 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T17:48:59.918 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T17:48:59.918 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi111.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T17:48:59.918 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T17:48:59.978 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T17:48:59.978 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T17:48:59.979 DEBUG:teuthology.orchestra.run.smithi111:> sudo rpm -qi grub2-tools 2024-09-13T17:49:00.011 INFO:teuthology.orchestra.run.smithi111.stdout:Name : grub2-tools 2024-09-13T17:49:00.011 INFO:teuthology.orchestra.run.smithi111.stdout:Epoch : 1 2024-09-13T17:49:00.011 INFO:teuthology.orchestra.run.smithi111.stdout:Version : 2.06 2024-09-13T17:49:00.011 INFO:teuthology.orchestra.run.smithi111.stdout:Release : 82.el9 2024-09-13T17:49:00.011 INFO:teuthology.orchestra.run.smithi111.stdout:Architecture: x86_64 2024-09-13T17:49:00.011 INFO:teuthology.orchestra.run.smithi111.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T17:49:00.011 INFO:teuthology.orchestra.run.smithi111.stdout:Group : Unspecified 2024-09-13T17:49:00.011 INFO:teuthology.orchestra.run.smithi111.stdout:Size : 8274670 2024-09-13T17:49:00.011 INFO:teuthology.orchestra.run.smithi111.stdout:License : GPLv3+ 2024-09-13T17:49:00.011 INFO:teuthology.orchestra.run.smithi111.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T17:49:00.012 INFO:teuthology.orchestra.run.smithi111.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T17:49:00.012 INFO:teuthology.orchestra.run.smithi111.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T17:49:00.012 INFO:teuthology.orchestra.run.smithi111.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T17:49:00.012 INFO:teuthology.orchestra.run.smithi111.stdout:Packager : builder@centos.org 2024-09-13T17:49:00.012 INFO:teuthology.orchestra.run.smithi111.stdout:Vendor : CentOS 2024-09-13T17:49:00.012 INFO:teuthology.orchestra.run.smithi111.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T17:49:00.012 INFO:teuthology.orchestra.run.smithi111.stdout:Summary : Support tools for GRUB. 2024-09-13T17:49:00.012 INFO:teuthology.orchestra.run.smithi111.stdout:Description : 2024-09-13T17:49:00.012 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:49:00.012 INFO:teuthology.orchestra.run.smithi111.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T17:49:00.012 INFO:teuthology.orchestra.run.smithi111.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T17:49:00.012 INFO:teuthology.orchestra.run.smithi111.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T17:49:00.012 INFO:teuthology.orchestra.run.smithi111.stdout:hardware devices. 2024-09-13T17:49:00.012 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:49:00.013 INFO:teuthology.orchestra.run.smithi111.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T17:49:00.013 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T17:49:00.013 INFO:teuthology.task.kernel:Updating grub on smithi111 to boot 5.14.0-505.el9.x86_64 2024-09-13T17:49:00.014 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T17:49:00.134 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:49:00.174 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:49:00.536 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:49:00.536 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T17:49:00.536 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:49:00.537 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:49:00.636 INFO:teuthology.orchestra.run.smithi111.stderr:Generating grub configuration file ... 2024-09-13T17:49:01.360 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:49:01.360 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:49:01.360 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-09-13T17:49:01.361 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T17:49:01.361 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T17:49:01.361 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T17:49:01.361 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T17:49:01.361 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:49:01.361 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-09-13T17:49:01.721 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T17:49:01.813 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T17:49:01.814 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T17:49:01.814 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T17:49:01.814 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T17:49:01.814 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T17:49:01.814 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T17:49:01.814 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi188.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T17:49:01.814 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T17:49:01.872 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T17:49:01.873 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T17:49:01.873 DEBUG:teuthology.orchestra.run.smithi188:> sudo rpm -qi grub2-tools 2024-09-13T17:49:01.904 INFO:teuthology.orchestra.run.smithi188.stdout:Name : grub2-tools 2024-09-13T17:49:01.904 INFO:teuthology.orchestra.run.smithi188.stdout:Epoch : 1 2024-09-13T17:49:01.904 INFO:teuthology.orchestra.run.smithi188.stdout:Version : 2.06 2024-09-13T17:49:01.904 INFO:teuthology.orchestra.run.smithi188.stdout:Release : 82.el9 2024-09-13T17:49:01.905 INFO:teuthology.orchestra.run.smithi188.stdout:Architecture: x86_64 2024-09-13T17:49:01.905 INFO:teuthology.orchestra.run.smithi188.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T17:49:01.905 INFO:teuthology.orchestra.run.smithi188.stdout:Group : Unspecified 2024-09-13T17:49:01.905 INFO:teuthology.orchestra.run.smithi188.stdout:Size : 8274670 2024-09-13T17:49:01.905 INFO:teuthology.orchestra.run.smithi188.stdout:License : GPLv3+ 2024-09-13T17:49:01.905 INFO:teuthology.orchestra.run.smithi188.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T17:49:01.905 INFO:teuthology.orchestra.run.smithi188.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T17:49:01.905 INFO:teuthology.orchestra.run.smithi188.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T17:49:01.905 INFO:teuthology.orchestra.run.smithi188.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T17:49:01.905 INFO:teuthology.orchestra.run.smithi188.stdout:Packager : builder@centos.org 2024-09-13T17:49:01.905 INFO:teuthology.orchestra.run.smithi188.stdout:Vendor : CentOS 2024-09-13T17:49:01.906 INFO:teuthology.orchestra.run.smithi188.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T17:49:01.906 INFO:teuthology.orchestra.run.smithi188.stdout:Summary : Support tools for GRUB. 2024-09-13T17:49:01.906 INFO:teuthology.orchestra.run.smithi188.stdout:Description : 2024-09-13T17:49:01.906 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:49:01.906 INFO:teuthology.orchestra.run.smithi188.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T17:49:01.906 INFO:teuthology.orchestra.run.smithi188.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T17:49:01.906 INFO:teuthology.orchestra.run.smithi188.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T17:49:01.906 INFO:teuthology.orchestra.run.smithi188.stdout:hardware devices. 2024-09-13T17:49:01.906 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:49:01.906 INFO:teuthology.orchestra.run.smithi188.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T17:49:01.907 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T17:49:01.907 INFO:teuthology.task.kernel:Updating grub on smithi188 to boot 5.14.0-505.el9.x86_64 2024-09-13T17:49:01.907 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T17:49:01.975 INFO:teuthology.orchestra.run.smithi111.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T17:49:02.011 INFO:teuthology.orchestra.run.smithi111.stderr:done 2024-09-13T17:49:02.013 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2024-09-13T17:49:02.029 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.qLeMcNiR54 2024-09-13T17:49:02.029 DEBUG:teuthology.orchestra.run.smithi111:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.qLeMcNiR54 2024-09-13T17:49:02.101 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.qLeMcNiR54 2024-09-13T17:49:02.243 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.qLeMcNiR54 is 6KB 2024-09-13T17:49:02.290 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.qLeMcNiR54 2024-09-13T17:49:02.305 DEBUG:teuthology.orchestra.run.smithi111:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T17:49:02.368 INFO:teuthology.orchestra.run.smithi111.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T17:49:02.368 INFO:teuthology.orchestra.run.smithi111.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T17:49:02.368 INFO:teuthology.orchestra.run.smithi111.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T17:49:02.368 INFO:teuthology.orchestra.run.smithi111.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T17:49:02.369 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T17:49:02.520 DEBUG:teuthology.orchestra.run.smithi111:> sudo shutdown -r now 2024-09-13T17:49:02.550 INFO:teuthology.orchestra.run.smithi188.stderr:Generating grub configuration file ... 2024-09-13T17:49:03.855 INFO:teuthology.orchestra.run.smithi188.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T17:49:03.890 INFO:teuthology.orchestra.run.smithi188.stderr:done 2024-09-13T17:49:03.893 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-09-13T17:49:03.908 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.V7IRlEHHTh 2024-09-13T17:49:03.908 DEBUG:teuthology.orchestra.run.smithi188:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.V7IRlEHHTh 2024-09-13T17:49:03.984 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.V7IRlEHHTh 2024-09-13T17:49:04.133 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.V7IRlEHHTh is 6KB 2024-09-13T17:49:04.154 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.V7IRlEHHTh 2024-09-13T17:49:04.169 DEBUG:teuthology.orchestra.run.smithi188:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T17:49:04.232 INFO:teuthology.orchestra.run.smithi188.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T17:49:04.232 INFO:teuthology.orchestra.run.smithi188.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T17:49:04.233 INFO:teuthology.orchestra.run.smithi188.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T17:49:04.233 INFO:teuthology.orchestra.run.smithi188.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T17:49:04.234 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T17:49:04.396 DEBUG:teuthology.orchestra.run.smithi188:> sudo shutdown -r now 2024-09-13T17:49:32.550 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T17:49:32.551 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-13T17:49:32.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:49:34.401 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T17:49:34.401 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-09-13T17:49:34.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:49:51.037 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-09-13T17:49:52.829 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2024-09-13T17:50:00.045 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-13T17:50:00.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:50:01.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-09-13T17:50:01.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:50:03.133 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-09-13T17:50:04.897 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2024-09-13T17:50:15.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-13T17:50:15.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:50:16.899 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-09-13T17:50:16.899 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:50:18.205 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-09-13T17:50:33.217 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-13T17:50:33.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:50:48.731 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-09-13T17:50:49.154 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-09-13T17:50:49.155 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-13T17:50:49.155 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-09-13T17:50:49.170 INFO:teuthology.orchestra.run.smithi188.stdout:5.14.0-505.el9.x86_64 2024-09-13T17:50:49.170 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T17:50:49.170 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T17:50:49.170 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T17:50:50.172 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T17:50:50.172 DEBUG:teuthology.orchestra.run.smithi188:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T17:50:50.234 INFO:teuthology.orchestra.run.smithi188.stdout:ttyS1 2024-09-13T17:50:50.251 DEBUG:teuthology.parallel:result is None 2024-09-13T17:50:51.709 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-09-13T17:51:09.710 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-13T17:51:09.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:51:10.049 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-09-13T17:51:10.426 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-13T17:51:10.426 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-13T17:51:10.426 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2024-09-13T17:51:10.443 INFO:teuthology.orchestra.run.smithi111.stdout:5.14.0-505.el9.x86_64 2024-09-13T17:51:10.443 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T17:51:10.443 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T17:51:10.443 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T17:51:11.444 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T17:51:11.445 DEBUG:teuthology.orchestra.run.smithi111:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T17:51:11.530 INFO:teuthology.orchestra.run.smithi111.stdout:ttyS1 2024-09-13T17:51:11.565 DEBUG:teuthology.parallel:result is None 2024-09-13T17:51:11.566 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T17:51:11.575 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T17:51:11.575 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T17:51:11.577 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T17:51:11.596 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T17:51:11.602 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T17:51:11.637 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T17:51:11.637 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T17:51:11.640 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T17:51:11.666 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T17:51:11.672 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T17:51:11.673 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T17:51:11.704 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T17:51:11.732 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:51:11.742 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:51:11.743 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:51:11.753 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:51:11.755 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T17:51:11.762 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T17:51:11.762 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T17:51:11.787 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T17:51:11.823 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T17:51:11.832 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T17:51:11.832 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T17:51:11.854 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T17:51:11.879 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T17:51:11.960 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T17:51:12.001 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T17:51:12.089 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T17:51:12.123 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:51:12.123 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T17:51:12.188 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T17:51:12.238 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T17:51:12.293 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T17:51:12.345 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T17:51:12.378 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:51:12.378 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T17:51:12.445 DEBUG:teuthology.orchestra.run.smithi111:> sudo service rsyslog restart 2024-09-13T17:51:12.447 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2024-09-13T17:51:12.490 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T17:51:12.526 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T17:51:12.802 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T17:51:12.810 INFO:teuthology.task.internal:Starting timer... 2024-09-13T17:51:12.810 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T17:51:12.849 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T17:51:12.881 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-13T17:51:12.882 DEBUG:teuthology.orchestra.run.smithi111:> sudo service auditd rotate 2024-09-13T17:51:12.930 INFO:teuthology.orchestra.run.smithi111.stdout:Rotating logs: 2024-09-13T17:51:12.932 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2024-09-13T17:51:13.001 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2024-09-13T17:51:13.003 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T17:51:13.003 DEBUG:teuthology.orchestra.run.smithi111:> /usr/sbin/getenforce 2024-09-13T17:51:13.032 INFO:teuthology.orchestra.run.smithi111.stdout:Permissive 2024-09-13T17:51:13.033 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2024-09-13T17:51:13.058 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2024-09-13T17:51:13.058 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi111.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2024-09-13T17:51:13.058 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T17:51:13.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:51:13.098 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T17:51:13.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:51:13.123 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T17:51:13.124 DEBUG:teuthology.orchestra.run.smithi111:> sudo /usr/sbin/setenforce permissive 2024-09-13T17:51:13.179 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2024-09-13T17:51:13.219 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T17:51:13.229 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T17:51:13.240 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T17:51:13.241 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi111.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T17:54:56.765 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi111.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2024-09-13T17:54:56.766 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-13T17:54:56.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:54:56.855 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-09-13T17:54:56.933 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-13T17:54:56.933 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-09-13T17:54:56.934 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:54:57.015 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-09-13T17:54:57.101 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-09-13T17:54:57.101 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T17:54:57.113 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T17:54:57.113 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T17:54:57.114 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:54:57.117 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T17:54:57.117 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:54:57.145 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T17:54:57.159 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T17:54:57.171 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T17:54:57.183 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: ntpd: command not found 2024-09-13T17:54:57.185 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T17:54:57.194 INFO:teuthology.orchestra.run.smithi111.stdout:506 Cannot talk to daemon 2024-09-13T17:54:57.208 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T17:54:57.209 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2024-09-13T17:54:57.221 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2024-09-13T17:54:57.222 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T17:54:57.235 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T17:54:57.249 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T17:54:57.278 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-09-13T17:54:57.282 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T17:54:57.282 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-09-13T17:54:57.282 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:54:57.282 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:54:57.282 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:54:57.282 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:54:57.306 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-09-13T17:54:57.310 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T17:54:57.311 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-09-13T17:54:57.311 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:54:57.311 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:54:57.311 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:54:57.311 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:54:57.312 INFO:teuthology.run_tasks:Running task install... 2024-09-13T17:54:57.319 DEBUG:teuthology.task.install:project ceph 2024-09-13T17:54:57.319 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}} 2024-09-13T17:54:57.320 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'} 2024-09-13T17:54:57.320 INFO:teuthology.task.install:Using flavor: default 2024-09-13T17:54:57.325 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T17:54:57.326 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-09-13T17:54:57.326 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'tag': None, 'wait_for_package': False} 2024-09-13T17:54:57.326 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:54:57.328 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'tag': None, 'wait_for_package': False} 2024-09-13T17:54:57.329 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:54:57.485 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/ 2024-09-13T17:54:57.486 INFO:teuthology.task.install.rpm:Package version is 18.2.4-860.g6c6b083f 2024-09-13T17:54:57.497 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/ 2024-09-13T17:54:57.497 INFO:teuthology.task.install.rpm:Package version is 18.2.4-860.g6c6b083f 2024-09-13T17:54:57.604 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T17:54:57.604 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:54:57.605 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T17:54:57.614 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T17:54:57.614 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:54:57.614 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T17:54:57.632 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-13T17:54:57.633 DEBUG:teuthology.orchestra.run.smithi111:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T17:54:57.641 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-13T17:54:57.642 DEBUG:teuthology.orchestra.run.smithi188:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T17:54:57.698 DEBUG:teuthology.orchestra.run.smithi111:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T17:54:57.708 DEBUG:teuthology.orchestra.run.smithi188:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T17:54:57.774 DEBUG:teuthology.orchestra.run.smithi111:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T17:54:57.783 DEBUG:teuthology.orchestra.run.smithi188:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T17:54:57.838 INFO:teuthology.orchestra.run.smithi111.stdout:check_obsoletes = 1 2024-09-13T17:54:57.839 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum clean all 2024-09-13T17:54:57.848 INFO:teuthology.orchestra.run.smithi188.stdout:check_obsoletes = 1 2024-09-13T17:54:57.849 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum clean all 2024-09-13T17:54:58.070 INFO:teuthology.orchestra.run.smithi111.stdout:45 files removed 2024-09-13T17:54:58.072 INFO:teuthology.orchestra.run.smithi188.stdout:45 files removed 2024-09-13T17:54:58.097 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-13T17:54:58.098 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-13T17:54:58.660 INFO:teuthology.orchestra.run.smithi111.stdout:ceph packages for x86_64 282 kB/s | 78 kB 00:00 2024-09-13T17:54:58.668 INFO:teuthology.orchestra.run.smithi188.stdout:ceph packages for x86_64 275 kB/s | 78 kB 00:00 2024-09-13T17:54:58.951 INFO:teuthology.orchestra.run.smithi111.stdout:ceph noarch packages 46 kB/s | 11 kB 00:00 2024-09-13T17:54:58.960 INFO:teuthology.orchestra.run.smithi188.stdout:ceph noarch packages 46 kB/s | 11 kB 00:00 2024-09-13T17:54:59.227 INFO:teuthology.orchestra.run.smithi111.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-13T17:54:59.236 INFO:teuthology.orchestra.run.smithi188.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-13T17:54:59.694 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.2 MB 00:00 2024-09-13T17:54:59.863 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-13T17:55:01.580 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 32 MB/s | 20 MB 00:00 2024-09-13T17:55:02.687 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-09-13T17:55:06.880 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.5 MB 00:01 2024-09-13T17:55:08.137 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-13T17:55:08.598 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 19 kB 00:00 2024-09-13T17:55:09.564 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 19 kB 00:00 2024-09-13T17:55:09.801 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-13T17:55:11.176 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 15 MB/s | 23 MB 00:01 2024-09-13T17:55:15.249 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-13T17:55:16.599 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-13T17:55:17.592 INFO:teuthology.orchestra.run.smithi188.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T17:55:17.592 INFO:teuthology.orchestra.run.smithi188.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T17:55:17.603 INFO:teuthology.orchestra.run.smithi188.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-13T17:55:17.603 INFO:teuthology.orchestra.run.smithi188.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-13T17:55:17.604 INFO:teuthology.orchestra.run.smithi188.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-13T17:55:17.605 INFO:teuthology.orchestra.run.smithi188.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-13T17:55:17.607 INFO:teuthology.orchestra.run.smithi188.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-13T17:55:17.609 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-13T17:55:17.610 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-13T17:55:17.670 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-09-13T17:55:17.689 INFO:teuthology.orchestra.run.smithi188.stdout:====================================================================================== 2024-09-13T17:55:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-09-13T17:55:17.689 INFO:teuthology.orchestra.run.smithi188.stdout:====================================================================================== 2024-09-13T17:55:17.689 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-09-13T17:55:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-13T17:55:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: ceph x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 6.5 k 2024-09-13T17:55:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 5.1 M 2024-09-13T17:55:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 852 k 2024-09-13T17:55:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 142 k 2024-09-13T17:55:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 1.5 M 2024-09-13T17:55:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 139 k 2024-09-13T17:55:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 3.5 M 2024-09-13T17:55:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 7.4 M 2024-09-13T17:55:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 49 k 2024-09-13T17:55:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 7.7 M 2024-09-13T17:55:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 40 M 2024-09-13T17:55:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 264 k 2024-09-13T17:55:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 226 k 2024-09-13T17:55:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 224 k 2024-09-13T17:55:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 18 k 2024-09-13T17:55:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-13T17:55:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-13T17:55:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-13T17:55:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 31 k 2024-09-13T17:55:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 712 k 2024-09-13T17:55:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 127 k 2024-09-13T17:55:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-13T17:55:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-13T17:55:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-13T17:55:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-13T17:55:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-13T17:55:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-13T17:55:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 162 k 2024-09-13T17:55:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 321 k 2024-09-13T17:55:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 301 k 2024-09-13T17:55:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 99 k 2024-09-13T17:55:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 87 k 2024-09-13T17:55:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T17:55:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 172 k 2024-09-13T17:55:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-13T17:55:17.692 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2024-09-13T17:55:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-13T17:55:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-13T17:55:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-13T17:55:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-13T17:55:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: librados2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.3 M 2024-09-13T17:55:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T17:55:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-13T17:55:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-13T17:55:17.693 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-09-13T17:55:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-13T17:55:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T17:55:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 18 M 2024-09-13T17:55:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 24 k 2024-09-13T17:55:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 2.1 M 2024-09-13T17:55:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 246 k 2024-09-13T17:55:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.7 M 2024-09-13T17:55:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 17 M 2024-09-13T17:55:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 16 k 2024-09-13T17:55:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 25 k 2024-09-13T17:55:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T17:55:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T17:55:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T17:55:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T17:55:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T17:55:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T17:55:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T17:55:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T17:55:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 166 k 2024-09-13T17:55:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T17:55:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T17:55:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T17:55:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T17:55:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T17:55:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 478 k 2024-09-13T17:55:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T17:55:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.5 M 2024-09-13T17:55:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T17:55:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-13T17:55:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T17:55:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T17:55:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T17:55:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T17:55:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-13T17:55:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T17:55:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T17:55:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T17:55:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T17:55:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 45 k 2024-09-13T17:55:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 129 k 2024-09-13T17:55:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T17:55:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T17:55:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T17:55:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T17:55:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T17:55:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T17:55:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T17:55:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T17:55:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T17:55:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T17:55:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T17:55:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T17:55:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T17:55:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T17:55:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T17:55:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T17:55:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T17:55:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T17:55:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T17:55:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T17:55:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T17:55:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T17:55:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T17:55:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T17:55:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T17:55:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T17:55:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T17:55:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T17:55:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T17:55:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T17:55:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T17:55:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T17:55:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T17:55:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T17:55:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T17:55:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T17:55:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T17:55:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T17:55:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T17:55:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T17:55:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T17:55:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T17:55:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T17:55:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T17:55:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T17:55:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T17:55:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T17:55:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T17:55:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T17:55:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T17:55:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T17:55:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-13T17:55:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T17:55:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-13T17:55:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:55:17.700 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-09-13T17:55:17.700 INFO:teuthology.orchestra.run.smithi188.stdout:====================================================================================== 2024-09-13T17:55:17.700 INFO:teuthology.orchestra.run.smithi188.stdout:Install 130 Packages 2024-09-13T17:55:17.700 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 8 Packages 2024-09-13T17:55:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:55:17.701 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 203 M 2024-09-13T17:55:17.701 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-09-13T17:55:18.317 INFO:teuthology.orchestra.run.smithi188.stdout:(1/138): ceph-18.2.4-860.g6c6b083f.el9.x86_64.r 47 kB/s | 6.5 kB 00:00 2024-09-13T17:55:18.501 INFO:teuthology.orchestra.run.smithi188.stdout:(2/138): ceph-fuse-18.2.4-860.g6c6b083f.el9.x86 4.5 MB/s | 852 kB 00:00 2024-09-13T17:55:18.609 INFO:teuthology.orchestra.run.smithi188.stdout:(3/138): ceph-base-18.2.4-860.g6c6b083f.el9.x86 12 MB/s | 5.1 MB 00:00 2024-09-13T17:55:18.634 INFO:teuthology.orchestra.run.smithi188.stdout:(4/138): ceph-immutable-object-cache-18.2.4-860 1.0 MB/s | 142 kB 00:00 2024-09-13T17:55:18.743 INFO:teuthology.orchestra.run.smithi188.stdout:(5/138): ceph-mds-18.2.4-860.g6c6b083f.el9.x86_ 16 MB/s | 2.1 MB 00:00 2024-09-13T17:55:18.785 INFO:teuthology.orchestra.run.smithi188.stdout:(6/138): ceph-mgr-18.2.4-860.g6c6b083f.el9.x86_ 9.7 MB/s | 1.5 MB 00:00 2024-09-13T17:55:18.961 INFO:teuthology.orchestra.run.smithi111.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T17:55:18.962 INFO:teuthology.orchestra.run.smithi111.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T17:55:18.973 INFO:teuthology.orchestra.run.smithi111.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-13T17:55:18.974 INFO:teuthology.orchestra.run.smithi111.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-13T17:55:18.975 INFO:teuthology.orchestra.run.smithi111.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-13T17:55:18.976 INFO:teuthology.orchestra.run.smithi111.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-13T17:55:18.978 INFO:teuthology.orchestra.run.smithi111.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-13T17:55:18.981 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-13T17:55:18.982 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-13T17:55:18.985 INFO:teuthology.orchestra.run.smithi188.stdout:(7/138): ceph-mon-18.2.4-860.g6c6b083f.el9.x86_ 19 MB/s | 4.7 MB 00:00 2024-09-13T17:55:19.043 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-13T17:55:19.064 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-13T17:55:19.064 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-09-13T17:55:19.064 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-13T17:55:19.064 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-13T17:55:19.064 INFO:teuthology.orchestra.run.smithi111.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-13T17:55:19.064 INFO:teuthology.orchestra.run.smithi111.stdout: ceph x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 6.5 k 2024-09-13T17:55:19.064 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 5.1 M 2024-09-13T17:55:19.065 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 852 k 2024-09-13T17:55:19.065 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 142 k 2024-09-13T17:55:19.065 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 1.5 M 2024-09-13T17:55:19.065 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 139 k 2024-09-13T17:55:19.065 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 3.5 M 2024-09-13T17:55:19.065 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 7.4 M 2024-09-13T17:55:19.065 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 49 k 2024-09-13T17:55:19.065 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 7.7 M 2024-09-13T17:55:19.065 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 40 M 2024-09-13T17:55:19.065 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 264 k 2024-09-13T17:55:19.065 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 226 k 2024-09-13T17:55:19.065 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 224 k 2024-09-13T17:55:19.066 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 18 k 2024-09-13T17:55:19.066 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-13T17:55:19.066 INFO:teuthology.orchestra.run.smithi111.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-13T17:55:19.066 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-13T17:55:19.066 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 31 k 2024-09-13T17:55:19.066 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 712 k 2024-09-13T17:55:19.066 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 127 k 2024-09-13T17:55:19.066 INFO:teuthology.orchestra.run.smithi111.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-13T17:55:19.066 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-13T17:55:19.066 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-13T17:55:19.066 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-13T17:55:19.067 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-13T17:55:19.067 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-13T17:55:19.067 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 162 k 2024-09-13T17:55:19.067 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 321 k 2024-09-13T17:55:19.067 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 301 k 2024-09-13T17:55:19.067 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 99 k 2024-09-13T17:55:19.067 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 87 k 2024-09-13T17:55:19.067 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T17:55:19.067 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 172 k 2024-09-13T17:55:19.068 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-13T17:55:19.068 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2024-09-13T17:55:19.068 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-13T17:55:19.068 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-13T17:55:19.068 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-13T17:55:19.068 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-13T17:55:19.068 INFO:teuthology.orchestra.run.smithi111.stdout: librados2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.3 M 2024-09-13T17:55:19.068 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T17:55:19.068 INFO:teuthology.orchestra.run.smithi111.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-13T17:55:19.068 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-13T17:55:19.068 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-09-13T17:55:19.068 INFO:teuthology.orchestra.run.smithi111.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-13T17:55:19.069 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T17:55:19.069 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 18 M 2024-09-13T17:55:19.069 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 24 k 2024-09-13T17:55:19.069 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 2.1 M 2024-09-13T17:55:19.069 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 246 k 2024-09-13T17:55:19.069 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.7 M 2024-09-13T17:55:19.069 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 17 M 2024-09-13T17:55:19.069 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 16 k 2024-09-13T17:55:19.069 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 25 k 2024-09-13T17:55:19.069 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T17:55:19.069 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T17:55:19.069 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T17:55:19.070 INFO:teuthology.orchestra.run.smithi111.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T17:55:19.070 INFO:teuthology.orchestra.run.smithi111.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T17:55:19.070 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T17:55:19.070 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T17:55:19.070 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T17:55:19.070 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 166 k 2024-09-13T17:55:19.070 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T17:55:19.070 INFO:teuthology.orchestra.run.smithi111.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T17:55:19.070 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T17:55:19.070 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T17:55:19.070 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T17:55:19.070 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 478 k 2024-09-13T17:55:19.070 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T17:55:19.071 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.5 M 2024-09-13T17:55:19.071 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T17:55:19.071 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-13T17:55:19.071 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T17:55:19.071 INFO:teuthology.orchestra.run.smithi111.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T17:55:19.071 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T17:55:19.071 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T17:55:19.071 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-13T17:55:19.071 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T17:55:19.071 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T17:55:19.071 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T17:55:19.071 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T17:55:19.071 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 45 k 2024-09-13T17:55:19.072 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 129 k 2024-09-13T17:55:19.072 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T17:55:19.072 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T17:55:19.072 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T17:55:19.072 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T17:55:19.072 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T17:55:19.072 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T17:55:19.072 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T17:55:19.072 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T17:55:19.072 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T17:55:19.072 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T17:55:19.072 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T17:55:19.073 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T17:55:19.073 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T17:55:19.073 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T17:55:19.073 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T17:55:19.073 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T17:55:19.073 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T17:55:19.073 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T17:55:19.073 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T17:55:19.073 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T17:55:19.073 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T17:55:19.073 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T17:55:19.073 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T17:55:19.073 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T17:55:19.074 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T17:55:19.074 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T17:55:19.074 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T17:55:19.074 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T17:55:19.074 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T17:55:19.074 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T17:55:19.074 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T17:55:19.074 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T17:55:19.074 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T17:55:19.074 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T17:55:19.074 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T17:55:19.074 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T17:55:19.075 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T17:55:19.075 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T17:55:19.075 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T17:55:19.075 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T17:55:19.075 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T17:55:19.075 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T17:55:19.075 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T17:55:19.075 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T17:55:19.075 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T17:55:19.075 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T17:55:19.075 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T17:55:19.075 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T17:55:19.075 INFO:teuthology.orchestra.run.smithi111.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T17:55:19.076 INFO:teuthology.orchestra.run.smithi111.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T17:55:19.076 INFO:teuthology.orchestra.run.smithi111.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T17:55:19.076 INFO:teuthology.orchestra.run.smithi111.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-13T17:55:19.076 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T17:55:19.076 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-13T17:55:19.076 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:55:19.076 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-13T17:55:19.076 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-13T17:55:19.076 INFO:teuthology.orchestra.run.smithi111.stdout:Install 130 Packages 2024-09-13T17:55:19.076 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 8 Packages 2024-09-13T17:55:19.076 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:55:19.077 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 203 M 2024-09-13T17:55:19.077 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-13T17:55:19.417 INFO:teuthology.orchestra.run.smithi111.stdout:(1/138): ceph-18.2.4-860.g6c6b083f.el9.x86_64.r 53 kB/s | 6.5 kB 00:00 2024-09-13T17:55:19.477 INFO:teuthology.orchestra.run.smithi188.stdout:(8/138): ceph-common-18.2.4-860.g6c6b083f.el9.x 14 MB/s | 18 MB 00:01 2024-09-13T17:55:19.569 INFO:teuthology.orchestra.run.smithi188.stdout:(9/138): ceph-radosgw-18.2.4-860.g6c6b083f.el9. 13 MB/s | 7.7 MB 00:00 2024-09-13T17:55:19.594 INFO:teuthology.orchestra.run.smithi188.stdout:(10/138): ceph-selinux-18.2.4-860.g6c6b083f.el9 216 kB/s | 25 kB 00:00 2024-09-13T17:55:19.617 INFO:teuthology.orchestra.run.smithi111.stdout:(2/138): ceph-fuse-18.2.4-860.g6c6b083f.el9.x86 4.2 MB/s | 852 kB 00:00 2024-09-13T17:55:19.644 INFO:teuthology.orchestra.run.smithi188.stdout:(11/138): cephfs-mirror-18.2.4-860.g6c6b083f.el 4.4 MB/s | 224 kB 00:00 2024-09-13T17:55:19.709 INFO:teuthology.orchestra.run.smithi111.stdout:(3/138): ceph-immutable-object-cache-18.2.4-860 1.5 MB/s | 142 kB 00:00 2024-09-13T17:55:19.736 INFO:teuthology.orchestra.run.smithi188.stdout:(12/138): libcephfs-devel-18.2.4-860.g6c6b083f. 340 kB/s | 31 kB 00:00 2024-09-13T17:55:19.992 INFO:teuthology.orchestra.run.smithi111.stdout:(4/138): ceph-mds-18.2.4-860.g6c6b083f.el9.x86_ 7.4 MB/s | 2.1 MB 00:00 2024-09-13T17:55:20.060 INFO:teuthology.orchestra.run.smithi111.stdout:(5/138): ceph-base-18.2.4-860.g6c6b083f.el9.x86 6.7 MB/s | 5.1 MB 00:00 2024-09-13T17:55:20.103 INFO:teuthology.orchestra.run.smithi188.stdout:(13/138): ceph-osd-18.2.4-860.g6c6b083f.el9.x86 13 MB/s | 17 MB 00:01 2024-09-13T17:55:20.160 INFO:teuthology.orchestra.run.smithi111.stdout:(6/138): ceph-mgr-18.2.4-860.g6c6b083f.el9.x86_ 8.7 MB/s | 1.5 MB 00:00 2024-09-13T17:55:20.170 INFO:teuthology.orchestra.run.smithi188.stdout:(14/138): libcephsqlite-18.2.4-860.g6c6b083f.el 2.4 MB/s | 166 kB 00:00 2024-09-13T17:55:20.254 INFO:teuthology.orchestra.run.smithi188.stdout:(15/138): librados-devel-18.2.4-860.g6c6b083f.e 1.5 MB/s | 127 kB 00:00 2024-09-13T17:55:20.346 INFO:teuthology.orchestra.run.smithi188.stdout:(16/138): libradosstriper1-18.2.4-860.g6c6b083f 5.1 MB/s | 478 kB 00:00 2024-09-13T17:55:20.479 INFO:teuthology.orchestra.run.smithi188.stdout:(17/138): libcephfs2-18.2.4-860.g6c6b083f.el9.x 959 kB/s | 712 kB 00:00 2024-09-13T17:55:20.485 INFO:teuthology.orchestra.run.smithi111.stdout:(7/138): ceph-mon-18.2.4-860.g6c6b083f.el9.x86_ 11 MB/s | 4.7 MB 00:00 2024-09-13T17:55:20.563 INFO:teuthology.orchestra.run.smithi188.stdout:(18/138): python3-ceph-argparse-18.2.4-860.g6c6 539 kB/s | 45 kB 00:00 2024-09-13T17:55:20.663 INFO:teuthology.orchestra.run.smithi188.stdout:(19/138): python3-ceph-common-18.2.4-860.g6c6b0 1.3 MB/s | 129 kB 00:00 2024-09-13T17:55:20.780 INFO:teuthology.orchestra.run.smithi188.stdout:(20/138): python3-cephfs-18.2.4-860.g6c6b083f.e 1.4 MB/s | 162 kB 00:00 2024-09-13T17:55:20.855 INFO:teuthology.orchestra.run.smithi188.stdout:(21/138): librgw2-18.2.4-860.g6c6b083f.el9.x86_ 8.8 MB/s | 4.5 MB 00:00 2024-09-13T17:55:20.881 INFO:teuthology.orchestra.run.smithi188.stdout:(22/138): python3-rados-18.2.4-860.g6c6b083f.el 3.1 MB/s | 321 kB 00:00 2024-09-13T17:55:20.948 INFO:teuthology.orchestra.run.smithi188.stdout:(23/138): python3-rbd-18.2.4-860.g6c6b083f.el9. 3.2 MB/s | 301 kB 00:00 2024-09-13T17:55:20.973 INFO:teuthology.orchestra.run.smithi188.stdout:(24/138): python3-rgw-18.2.4-860.g6c6b083f.el9. 1.1 MB/s | 99 kB 00:00 2024-09-13T17:55:21.025 INFO:teuthology.orchestra.run.smithi188.stdout:(25/138): rbd-fuse-18.2.4-860.g6c6b083f.el9.x86 1.1 MB/s | 87 kB 00:00 2024-09-13T17:55:21.118 INFO:teuthology.orchestra.run.smithi111.stdout:(8/138): ceph-common-18.2.4-860.g6c6b083f.el9.x 10 MB/s | 18 MB 00:01 2024-09-13T17:55:21.132 INFO:teuthology.orchestra.run.smithi188.stdout:(26/138): rbd-nbd-18.2.4-860.g6c6b083f.el9.x86_ 1.5 MB/s | 172 kB 00:00 2024-09-13T17:55:21.185 INFO:teuthology.orchestra.run.smithi111.stdout:(9/138): ceph-selinux-18.2.4-860.g6c6b083f.el9. 378 kB/s | 25 kB 00:00 2024-09-13T17:55:21.233 INFO:teuthology.orchestra.run.smithi188.stdout:(27/138): ceph-grafana-dashboards-18.2.4-860.g6 244 kB/s | 24 kB 00:00 2024-09-13T17:55:21.302 INFO:teuthology.orchestra.run.smithi111.stdout:(10/138): ceph-radosgw-18.2.4-860.g6c6b083f.el9 9.5 MB/s | 7.7 MB 00:00 2024-09-13T17:55:21.308 INFO:teuthology.orchestra.run.smithi188.stdout:(28/138): rbd-mirror-18.2.4-860.g6c6b083f.el9.x 8.9 MB/s | 3.0 MB 00:00 2024-09-13T17:55:21.333 INFO:teuthology.orchestra.run.smithi188.stdout:(29/138): ceph-mgr-cephadm-18.2.4-860.g6c6b083f 1.4 MB/s | 139 kB 00:00 2024-09-13T17:55:21.378 INFO:teuthology.orchestra.run.smithi111.stdout:(11/138): cephfs-mirror-18.2.4-860.g6c6b083f.el 2.9 MB/s | 224 kB 00:00 2024-09-13T17:55:21.478 INFO:teuthology.orchestra.run.smithi111.stdout:(12/138): libcephfs-devel-18.2.4-860.g6c6b083f. 310 kB/s | 31 kB 00:00 2024-09-13T17:55:21.725 INFO:teuthology.orchestra.run.smithi188.stdout:(30/138): ceph-mgr-dashboard-18.2.4-860.g6c6b08 8.5 MB/s | 3.5 MB 00:00 2024-09-13T17:55:21.937 INFO:teuthology.orchestra.run.smithi111.stdout:(13/138): libcephfs2-18.2.4-860.g6c6b083f.el9.x 1.5 MB/s | 712 kB 00:00 2024-09-13T17:55:22.004 INFO:teuthology.orchestra.run.smithi111.stdout:(14/138): libcephsqlite-18.2.4-860.g6c6b083f.el 2.4 MB/s | 166 kB 00:00 2024-09-13T17:55:22.079 INFO:teuthology.orchestra.run.smithi111.stdout:(15/138): librados-devel-18.2.4-860.g6c6b083f.e 1.7 MB/s | 127 kB 00:00 2024-09-13T17:55:22.109 INFO:teuthology.orchestra.run.smithi188.stdout:(31/138): ceph-test-18.2.4-860.g6c6b083f.el9.x8 16 MB/s | 40 MB 00:02 2024-09-13T17:55:22.134 INFO:teuthology.orchestra.run.smithi188.stdout:(32/138): ceph-mgr-modules-core-18.2.4-860.g6c6 602 kB/s | 246 kB 00:00 2024-09-13T17:55:22.163 INFO:teuthology.orchestra.run.smithi111.stdout:(16/138): libradosstriper1-18.2.4-860.g6c6b083f 5.6 MB/s | 478 kB 00:00 2024-09-13T17:55:22.184 INFO:teuthology.orchestra.run.smithi188.stdout:(33/138): ceph-mgr-rook-18.2.4-860.g6c6b083f.el 660 kB/s | 49 kB 00:00 2024-09-13T17:55:22.210 INFO:teuthology.orchestra.run.smithi188.stdout:(34/138): ceph-prometheus-alerts-18.2.4-860.g6c 219 kB/s | 16 kB 00:00 2024-09-13T17:55:22.243 INFO:teuthology.orchestra.run.smithi188.stdout:(35/138): ceph-volume-18.2.4-860.g6c6b083f.el9. 4.4 MB/s | 264 kB 00:00 2024-09-13T17:55:22.330 INFO:teuthology.orchestra.run.smithi111.stdout:(17/138): ceph-osd-18.2.4-860.g6c6b083f.el9.x86 7.6 MB/s | 17 MB 00:02 2024-09-13T17:55:22.352 INFO:teuthology.orchestra.run.smithi188.stdout:(36/138): ceph-mgr-diskprediction-local-18.2.4- 7.3 MB/s | 7.4 MB 00:01 2024-09-13T17:55:22.377 INFO:teuthology.orchestra.run.smithi188.stdout:(37/138): cephadm-18.2.4-860.g6c6b083f.el9.noar 1.3 MB/s | 226 kB 00:00 2024-09-13T17:55:22.397 INFO:teuthology.orchestra.run.smithi111.stdout:(18/138): python3-ceph-argparse-18.2.4-860.g6c6 678 kB/s | 45 kB 00:00 2024-09-13T17:55:22.402 INFO:teuthology.orchestra.run.smithi188.stdout:(38/138): cephfs-top-18.2.4-860.g6c6b083f.el9.n 115 kB/s | 18 kB 00:00 2024-09-13T17:55:22.464 INFO:teuthology.orchestra.run.smithi111.stdout:(19/138): python3-ceph-common-18.2.4-860.g6c6b0 1.9 MB/s | 129 kB 00:00 2024-09-13T17:55:22.539 INFO:teuthology.orchestra.run.smithi111.stdout:(20/138): python3-cephfs-18.2.4-860.g6c6b083f.e 2.1 MB/s | 162 kB 00:00 2024-09-13T17:55:22.622 INFO:teuthology.orchestra.run.smithi111.stdout:(21/138): librgw2-18.2.4-860.g6c6b083f.el9.x86_ 9.7 MB/s | 4.5 MB 00:00 2024-09-13T17:55:22.648 INFO:teuthology.orchestra.run.smithi111.stdout:(22/138): python3-rados-18.2.4-860.g6c6b083f.el 2.9 MB/s | 321 kB 00:00 2024-09-13T17:55:22.690 INFO:teuthology.orchestra.run.smithi111.stdout:(23/138): python3-rbd-18.2.4-860.g6c6b083f.el9. 4.4 MB/s | 301 kB 00:00 2024-09-13T17:55:22.715 INFO:teuthology.orchestra.run.smithi111.stdout:(24/138): python3-rgw-18.2.4-860.g6c6b083f.el9. 1.4 MB/s | 99 kB 00:00 2024-09-13T17:55:22.740 INFO:teuthology.orchestra.run.smithi111.stdout:(25/138): rbd-fuse-18.2.4-860.g6c6b083f.el9.x86 1.7 MB/s | 87 kB 00:00 2024-09-13T17:55:22.777 INFO:teuthology.orchestra.run.smithi188.stdout:(39/138): fuse-2.9.9-16.el9.x86_64.rpm 200 kB/s | 80 kB 00:00 2024-09-13T17:55:22.816 INFO:teuthology.orchestra.run.smithi111.stdout:(26/138): rbd-nbd-18.2.4-860.g6c6b083f.el9.x86_ 2.2 MB/s | 172 kB 00:00 2024-09-13T17:55:22.908 INFO:teuthology.orchestra.run.smithi111.stdout:(27/138): ceph-grafana-dashboards-18.2.4-860.g6 267 kB/s | 24 kB 00:00 2024-09-13T17:55:22.953 INFO:teuthology.orchestra.run.smithi188.stdout:(40/138): libquadmath-11.5.0-2.el9.x86_64.rpm 347 kB/s | 191 kB 00:00 2024-09-13T17:55:22.992 INFO:teuthology.orchestra.run.smithi111.stdout:(28/138): ceph-mgr-cephadm-18.2.4-860.g6c6b083f 1.6 MB/s | 139 kB 00:00 2024-09-13T17:55:23.070 INFO:teuthology.orchestra.run.smithi188.stdout:(41/138): python3-cffi-1.14.5-5.el9.x86_64.rpm 867 kB/s | 253 kB 00:00 2024-09-13T17:55:23.195 INFO:teuthology.orchestra.run.smithi188.stdout:(42/138): python3-ply-3.11-14.el9.noarch.rpm 852 kB/s | 106 kB 00:00 2024-09-13T17:55:23.229 INFO:teuthology.orchestra.run.smithi188.stdout:(43/138): libgfortran-11.5.0-2.el9.x86_64.rpm 942 kB/s | 801 kB 00:00 2024-09-13T17:55:23.304 INFO:teuthology.orchestra.run.smithi188.stdout:(44/138): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-13T17:55:23.337 INFO:teuthology.orchestra.run.smithi188.stdout:(45/138): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-13T17:55:23.375 INFO:teuthology.orchestra.run.smithi111.stdout:(29/138): rbd-mirror-18.2.4-860.g6c6b083f.el9.x 4.5 MB/s | 3.0 MB 00:00 2024-09-13T17:55:23.421 INFO:teuthology.orchestra.run.smithi188.stdout:(46/138): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-13T17:55:23.463 INFO:teuthology.orchestra.run.smithi188.stdout:(47/138): python3-cryptography-36.0.1-4.el9.x86 2.4 MB/s | 1.2 MB 00:00 2024-09-13T17:55:23.509 INFO:teuthology.orchestra.run.smithi111.stdout:(30/138): ceph-mgr-dashboard-18.2.4-860.g6c6b08 6.8 MB/s | 3.5 MB 00:00 2024-09-13T17:55:23.522 INFO:teuthology.orchestra.run.smithi188.stdout:(48/138): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-09-13T17:55:23.564 INFO:teuthology.orchestra.run.smithi188.stdout:(49/138): elfutils-libelf-devel-0.191-4.el9.x86 524 kB/s | 22 kB 00:00 2024-09-13T17:55:23.567 INFO:teuthology.orchestra.run.smithi111.stdout:(31/138): ceph-mgr-modules-core-18.2.4-860.g6c6 4.1 MB/s | 246 kB 00:00 2024-09-13T17:55:23.589 INFO:teuthology.orchestra.run.smithi188.stdout:(50/138): boost-program-options-1.75.0-8.el9.x8 851 kB/s | 107 kB 00:00 2024-09-13T17:55:23.614 INFO:teuthology.orchestra.run.smithi188.stdout:(51/138): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-13T17:55:23.617 INFO:teuthology.orchestra.run.smithi111.stdout:(32/138): ceph-mgr-rook-18.2.4-860.g6c6b083f.el 986 kB/s | 49 kB 00:00 2024-09-13T17:55:23.648 INFO:teuthology.orchestra.run.smithi188.stdout:(52/138): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-09-13T17:55:23.673 INFO:teuthology.orchestra.run.smithi188.stdout:(53/138): flexiblas-3.0.4-8.el9.x86_64.rpm 390 kB/s | 33 kB 00:00 2024-09-13T17:55:23.699 INFO:teuthology.orchestra.run.smithi188.stdout:(54/138): flexiblas-openblas-openmp-3.0.4-8.el9 361 kB/s | 18 kB 00:00 2024-09-13T17:55:23.724 INFO:teuthology.orchestra.run.smithi188.stdout:(55/138): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-13T17:55:23.749 INFO:teuthology.orchestra.run.smithi188.stdout:(56/138): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-13T17:55:23.774 INFO:teuthology.orchestra.run.smithi188.stdout:(57/138): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-09-13T17:55:23.776 INFO:teuthology.orchestra.run.smithi111.stdout:(33/138): ceph-prometheus-alerts-18.2.4-860.g6c 104 kB/s | 16 kB 00:00 2024-09-13T17:55:23.816 INFO:teuthology.orchestra.run.smithi188.stdout:(58/138): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-13T17:55:23.850 INFO:teuthology.orchestra.run.smithi188.stdout:(59/138): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-09-13T17:55:23.875 INFO:teuthology.orchestra.run.smithi188.stdout:(60/138): libuuid-devel-2.37.4-20.el9.x86_64.rp 417 kB/s | 24 kB 00:00 2024-09-13T17:55:23.901 INFO:teuthology.orchestra.run.smithi188.stdout:(61/138): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-13T17:55:23.926 INFO:teuthology.orchestra.run.smithi188.stdout:(62/138): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-13T17:55:23.951 INFO:teuthology.orchestra.run.smithi188.stdout:(63/138): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-13T17:55:24.010 INFO:teuthology.orchestra.run.smithi188.stdout:(64/138): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.6 MB/s | 3.0 MB 00:00 2024-09-13T17:55:24.094 INFO:teuthology.orchestra.run.smithi188.stdout:(65/138): postgresql-13.16-1.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-13T17:55:24.127 INFO:teuthology.orchestra.run.smithi111.stdout:(34/138): ceph-test-18.2.4-860.g6c6b083f.el9.x8 13 MB/s | 40 MB 00:02 2024-09-13T17:55:24.152 INFO:teuthology.orchestra.run.smithi111.stdout:(35/138): ceph-volume-18.2.4-860.g6c6b083f.el9. 705 kB/s | 264 kB 00:00 2024-09-13T17:55:24.160 INFO:teuthology.orchestra.run.smithi188.stdout:(66/138): openblas-openmp-0.3.26-2.el9.x86_64.r 21 MB/s | 4.9 MB 00:00 2024-09-13T17:55:24.202 INFO:teuthology.orchestra.run.smithi188.stdout:(67/138): postgresql-contrib-13.16-1.el9.x86_64 8.2 MB/s | 910 kB 00:00 2024-09-13T17:55:24.252 INFO:teuthology.orchestra.run.smithi111.stdout:(36/138): cephfs-top-18.2.4-860.g6c6b083f.el9.n 183 kB/s | 18 kB 00:00 2024-09-13T17:55:24.261 INFO:teuthology.orchestra.run.smithi188.stdout:(68/138): openssl-devel-3.2.2-6.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-13T17:55:24.286 INFO:teuthology.orchestra.run.smithi111.stdout:(37/138): cephadm-18.2.4-860.g6c6b083f.el9.noar 1.4 MB/s | 226 kB 00:00 2024-09-13T17:55:24.286 INFO:teuthology.orchestra.run.smithi188.stdout:(69/138): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-13T17:55:24.328 INFO:teuthology.orchestra.run.smithi188.stdout:(70/138): python3-devel-3.9.19-8.el9.x86_64.rpm 5.8 MB/s | 245 kB 00:00 2024-09-13T17:55:24.412 INFO:teuthology.orchestra.run.smithi188.stdout:(71/138): postgresql-server-13.16-1.el9.x86_64. 28 MB/s | 5.8 MB 00:00 2024-09-13T17:55:24.413 INFO:teuthology.orchestra.run.smithi111.stdout:(38/138): fuse-2.9.9-16.el9.x86_64.rpm 497 kB/s | 80 kB 00:00 2024-09-13T17:55:24.437 INFO:teuthology.orchestra.run.smithi188.stdout:(72/138): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-13T17:55:24.454 INFO:teuthology.orchestra.run.smithi188.stdout:(73/138): python3-jsonpatch-1.21-16.el9.noarch. 612 kB/s | 26 kB 00:00 2024-09-13T17:55:24.471 INFO:teuthology.orchestra.run.smithi188.stdout:(74/138): python3-jsonpointer-2.0-4.el9.noarch. 575 kB/s | 19 kB 00:00 2024-09-13T17:55:24.488 INFO:teuthology.orchestra.run.smithi111.stdout:(39/138): libquadmath-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-13T17:55:24.496 INFO:teuthology.orchestra.run.smithi188.stdout:(75/138): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-09-13T17:55:24.514 INFO:teuthology.orchestra.run.smithi111.stdout:(40/138): libgfortran-11.5.0-2.el9.x86_64.rpm 3.4 MB/s | 801 kB 00:00 2024-09-13T17:55:24.514 INFO:teuthology.orchestra.run.smithi188.stdout:(76/138): python3-markupsafe-1.1.1-12.el9.x86_6 829 kB/s | 35 kB 00:00 2024-09-13T17:55:24.539 INFO:teuthology.orchestra.run.smithi111.stdout:(41/138): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-09-13T17:55:24.581 INFO:teuthology.orchestra.run.smithi111.stdout:(42/138): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-13T17:55:24.597 INFO:teuthology.orchestra.run.smithi188.stdout:(77/138): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-09-13T17:55:24.622 INFO:teuthology.orchestra.run.smithi188.stdout:(78/138): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-13T17:55:24.648 INFO:teuthology.orchestra.run.smithi188.stdout:(79/138): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-13T17:55:24.673 INFO:teuthology.orchestra.run.smithi188.stdout:(80/138): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-13T17:55:24.673 INFO:teuthology.orchestra.run.smithi111.stdout:(43/138): ceph-mgr-diskprediction-local-18.2.4- 5.7 MB/s | 7.4 MB 00:01 2024-09-13T17:55:24.690 INFO:teuthology.orchestra.run.smithi188.stdout:(81/138): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-13T17:55:24.698 INFO:teuthology.orchestra.run.smithi111.stdout:(44/138): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-13T17:55:24.715 INFO:teuthology.orchestra.run.smithi188.stdout:(82/138): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-09-13T17:55:24.741 INFO:teuthology.orchestra.run.smithi188.stdout:(83/138): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-13T17:55:24.748 INFO:teuthology.orchestra.run.smithi111.stdout:(45/138): python3-cryptography-36.0.1-4.el9.x86 5.2 MB/s | 1.2 MB 00:00 2024-09-13T17:55:24.758 INFO:teuthology.orchestra.run.smithi188.stdout:(84/138): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-13T17:55:24.774 INFO:teuthology.orchestra.run.smithi111.stdout:(46/138): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-13T17:55:24.775 INFO:teuthology.orchestra.run.smithi188.stdout:(85/138): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-13T17:55:24.799 INFO:teuthology.orchestra.run.smithi111.stdout:(47/138): python3-requests-2.25.1-8.el9.noarch. 995 kB/s | 125 kB 00:00 2024-09-13T17:55:24.859 INFO:teuthology.orchestra.run.smithi188.stdout:(86/138): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-09-13T17:55:24.875 INFO:teuthology.orchestra.run.smithi188.stdout:(87/138): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-09-13T17:55:24.900 INFO:teuthology.orchestra.run.smithi188.stdout:(88/138): socat-1.7.4.1-6.el9.x86_64.rpm 7.2 MB/s | 304 kB 00:00 2024-09-13T17:55:24.917 INFO:teuthology.orchestra.run.smithi188.stdout:(89/138): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2024-09-13T17:55:24.941 INFO:teuthology.orchestra.run.smithi111.stdout:(48/138): boost-program-options-1.75.0-8.el9.x8 755 kB/s | 107 kB 00:00 2024-09-13T17:55:24.942 INFO:teuthology.orchestra.run.smithi188.stdout:(90/138): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 2.2 MB/s | 96 kB 00:00 2024-09-13T17:55:24.966 INFO:teuthology.orchestra.run.smithi111.stdout:(49/138): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-09-13T17:55:24.968 INFO:teuthology.orchestra.run.smithi188.stdout:(91/138): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-13T17:55:24.991 INFO:teuthology.orchestra.run.smithi111.stdout:(50/138): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-13T17:55:24.993 INFO:teuthology.orchestra.run.smithi188.stdout:(92/138): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-13T17:55:25.035 INFO:teuthology.orchestra.run.smithi188.stdout:(93/138): fmt-8.1.1-5.el9.x86_64.rpm 1.6 MB/s | 111 kB 00:00 2024-09-13T17:55:25.042 INFO:teuthology.orchestra.run.smithi111.stdout:(51/138): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-09-13T17:55:25.067 INFO:teuthology.orchestra.run.smithi111.stdout:(52/138): flexiblas-3.0.4-8.el9.x86_64.rpm 434 kB/s | 33 kB 00:00 2024-09-13T17:55:25.093 INFO:teuthology.orchestra.run.smithi111.stdout:(53/138): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-09-13T17:55:25.102 INFO:teuthology.orchestra.run.smithi188.stdout:(94/138): gperftools-libs-2.9.1-3.el9.x86_64.rp 2.8 MB/s | 308 kB 00:00 2024-09-13T17:55:25.121 INFO:teuthology.orchestra.run.smithi111.stdout:(54/138): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-13T17:55:25.127 INFO:teuthology.orchestra.run.smithi188.stdout:(95/138): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-13T17:55:25.143 INFO:teuthology.orchestra.run.smithi111.stdout:(55/138): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-13T17:55:25.153 INFO:teuthology.orchestra.run.smithi188.stdout:(96/138): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-13T17:55:25.168 INFO:teuthology.orchestra.run.smithi111.stdout:(56/138): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-13T17:55:25.178 INFO:teuthology.orchestra.run.smithi188.stdout:(97/138): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-13T17:55:25.194 INFO:teuthology.orchestra.run.smithi111.stdout:(57/138): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-09-13T17:55:25.252 INFO:teuthology.orchestra.run.smithi111.stdout:(58/138): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-09-13T17:55:25.278 INFO:teuthology.orchestra.run.smithi111.stdout:(59/138): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-09-13T17:55:25.303 INFO:teuthology.orchestra.run.smithi111.stdout:(60/138): libuuid-devel-2.37.4-20.el9.x86_64.rp 488 kB/s | 24 kB 00:00 2024-09-13T17:55:25.328 INFO:teuthology.orchestra.run.smithi111.stdout:(61/138): libzstd-devel-1.5.1-2.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-09-13T17:55:25.362 INFO:teuthology.orchestra.run.smithi111.stdout:(62/138): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-13T17:55:25.370 INFO:teuthology.orchestra.run.smithi188.stdout:(98/138): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-13T17:55:25.387 INFO:teuthology.orchestra.run.smithi111.stdout:(63/138): openblas-0.3.26-2.el9.x86_64.rpm 656 kB/s | 38 kB 00:00 2024-09-13T17:55:25.412 INFO:teuthology.orchestra.run.smithi188.stdout:(99/138): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.5 MB/s | 839 kB 00:00 2024-09-13T17:55:25.412 INFO:teuthology.orchestra.run.smithi111.stdout:(64/138): libtool-2.4.6-46.el9.x86_64.rpm 2.6 MB/s | 581 kB 00:00 2024-09-13T17:55:25.437 INFO:teuthology.orchestra.run.smithi188.stdout:(100/138): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.8 MB/s | 43 kB 00:00 2024-09-13T17:55:25.463 INFO:teuthology.orchestra.run.smithi188.stdout:(101/138): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-13T17:55:25.488 INFO:teuthology.orchestra.run.smithi188.stdout:(102/138): python3-certifi-2023.05.07-4.el9.noa 566 kB/s | 14 kB 00:00 2024-09-13T17:55:25.504 INFO:teuthology.orchestra.run.smithi111.stdout:(65/138): openblas-openmp-0.3.26-2.el9.x86_64.r 35 MB/s | 4.9 MB 00:00 2024-09-13T17:55:25.530 INFO:teuthology.orchestra.run.smithi188.stdout:(103/138): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-09-13T17:55:25.554 INFO:teuthology.orchestra.run.smithi111.stdout:(66/138): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-13T17:55:25.555 INFO:teuthology.orchestra.run.smithi188.stdout:(104/138): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-13T17:55:25.589 INFO:teuthology.orchestra.run.smithi188.stdout:(105/138): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-13T17:55:25.596 INFO:teuthology.orchestra.run.smithi111.stdout:(67/138): postgresql-contrib-13.16-1.el9.x86_64 9.7 MB/s | 910 kB 00:00 2024-09-13T17:55:25.622 INFO:teuthology.orchestra.run.smithi111.stdout:(68/138): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-13T17:55:25.656 INFO:teuthology.orchestra.run.smithi188.stdout:(106/138): libarrow-9.0.0-11.el9.x86_64.rpm 7.1 MB/s | 4.4 MB 00:00 2024-09-13T17:55:25.672 INFO:teuthology.orchestra.run.smithi188.stdout:(107/138): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-09-13T17:55:25.680 INFO:teuthology.orchestra.run.smithi111.stdout:(69/138): openssl-devel-3.2.2-6.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-13T17:55:25.698 INFO:teuthology.orchestra.run.smithi188.stdout:(108/138): python3-jaraco-classes-3.2.1-5.el9.n 423 kB/s | 18 kB 00:00 2024-09-13T17:55:25.723 INFO:teuthology.orchestra.run.smithi188.stdout:(109/138): python3-google-auth-2.34.0-1.el9.noa 1.3 MB/s | 220 kB 00:00 2024-09-13T17:55:25.731 INFO:teuthology.orchestra.run.smithi111.stdout:(70/138): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-13T17:55:25.749 INFO:teuthology.orchestra.run.smithi188.stdout:(110/138): python3-jaraco-collections-3.0.0-8.e 305 kB/s | 23 kB 00:00 2024-09-13T17:55:25.774 INFO:teuthology.orchestra.run.smithi188.stdout:(111/138): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-09-13T17:55:25.799 INFO:teuthology.orchestra.run.smithi188.stdout:(112/138): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-13T17:55:25.814 INFO:teuthology.orchestra.run.smithi111.stdout:(71/138): postgresql-server-13.16-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-09-13T17:55:25.824 INFO:teuthology.orchestra.run.smithi188.stdout:(113/138): python3-logutils-0.3.5-21.el9.noarch 924 kB/s | 46 kB 00:00 2024-09-13T17:55:25.839 INFO:teuthology.orchestra.run.smithi111.stdout:(72/138): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-13T17:55:25.850 INFO:teuthology.orchestra.run.smithi188.stdout:(114/138): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-13T17:55:25.865 INFO:teuthology.orchestra.run.smithi111.stdout:(73/138): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-09-13T17:55:25.875 INFO:teuthology.orchestra.run.smithi188.stdout:(115/138): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-13T17:55:25.890 INFO:teuthology.orchestra.run.smithi111.stdout:(74/138): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-09-13T17:55:25.900 INFO:teuthology.orchestra.run.smithi188.stdout:(116/138): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-13T17:55:25.915 INFO:teuthology.orchestra.run.smithi111.stdout:(75/138): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-13T17:55:25.925 INFO:teuthology.orchestra.run.smithi188.stdout:(117/138): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-13T17:55:25.941 INFO:teuthology.orchestra.run.smithi111.stdout:(76/138): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-13T17:55:25.951 INFO:teuthology.orchestra.run.smithi188.stdout:(118/138): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-13T17:55:25.976 INFO:teuthology.orchestra.run.smithi188.stdout:(119/138): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-13T17:55:26.001 INFO:teuthology.orchestra.run.smithi188.stdout:(120/138): python3-routes-2.5.1-5.el9.noarch.rp 7.5 MB/s | 188 kB 00:00 2024-09-13T17:55:26.024 INFO:teuthology.orchestra.run.smithi111.stdout:(77/138): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-13T17:55:26.027 INFO:teuthology.orchestra.run.smithi188.stdout:(121/138): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-13T17:55:26.050 INFO:teuthology.orchestra.run.smithi111.stdout:(78/138): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-13T17:55:26.052 INFO:teuthology.orchestra.run.smithi188.stdout:(122/138): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-13T17:55:26.075 INFO:teuthology.orchestra.run.smithi111.stdout:(79/138): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-13T17:55:26.094 INFO:teuthology.orchestra.run.smithi188.stdout:(123/138): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-09-13T17:55:26.100 INFO:teuthology.orchestra.run.smithi111.stdout:(80/138): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-13T17:55:26.119 INFO:teuthology.orchestra.run.smithi188.stdout:(124/138): python3-typing-extensions-4.12.2-2.e 809 kB/s | 74 kB 00:00 2024-09-13T17:55:26.125 INFO:teuthology.orchestra.run.smithi111.stdout:(81/138): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-13T17:55:26.144 INFO:teuthology.orchestra.run.smithi188.stdout:(125/138): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-13T17:55:26.151 INFO:teuthology.orchestra.run.smithi111.stdout:(82/138): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-13T17:55:26.170 INFO:teuthology.orchestra.run.smithi188.stdout:(126/138): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-13T17:55:26.176 INFO:teuthology.orchestra.run.smithi111.stdout:(83/138): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-13T17:55:26.195 INFO:teuthology.orchestra.run.smithi188.stdout:(127/138): python3-zc-lockfile-2.0-10.el9.noarc 396 kB/s | 20 kB 00:00 2024-09-13T17:55:26.201 INFO:teuthology.orchestra.run.smithi111.stdout:(84/138): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-13T17:55:26.220 INFO:teuthology.orchestra.run.smithi188.stdout:(128/138): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-13T17:55:26.226 INFO:teuthology.orchestra.run.smithi111.stdout:(85/138): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-13T17:55:26.246 INFO:teuthology.orchestra.run.smithi188.stdout:(129/138): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-13T17:55:26.313 INFO:teuthology.orchestra.run.smithi188.stdout:(130/138): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-13T17:55:26.319 INFO:teuthology.orchestra.run.smithi111.stdout:(86/138): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-13T17:55:26.345 INFO:teuthology.orchestra.run.smithi111.stdout:(87/138): python3-toml-0.10.2-6.el9.noarch.rpm 357 kB/s | 42 kB 00:00 2024-09-13T17:55:26.377 INFO:teuthology.orchestra.run.smithi111.stdout:(88/138): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-13T17:55:26.420 INFO:teuthology.orchestra.run.smithi111.stdout:(89/138): uuid-1.6.2-55.el9.x86_64.rpm 767 kB/s | 58 kB 00:00 2024-09-13T17:55:26.445 INFO:teuthology.orchestra.run.smithi111.stdout:(90/138): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.4 MB/s | 96 kB 00:00 2024-09-13T17:55:26.470 INFO:teuthology.orchestra.run.smithi111.stdout:(91/138): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-13T17:55:26.495 INFO:teuthology.orchestra.run.smithi111.stdout:(92/138): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-09-13T17:55:26.521 INFO:teuthology.orchestra.run.smithi188.stdout:(131/138): librbd1-18.2.4-860.g6c6b083f.el9.x86 11 MB/s | 3.0 MB 00:00 2024-09-13T17:55:26.554 INFO:teuthology.orchestra.run.smithi111.stdout:(93/138): fmt-8.1.1-5.el9.x86_64.rpm 1.3 MB/s | 111 kB 00:00 2024-09-13T17:55:26.580 INFO:teuthology.orchestra.run.smithi188.stdout:(132/138): librados2-18.2.4-860.g6c6b083f.el9.x 9.2 MB/s | 3.3 MB 00:00 2024-09-13T17:55:26.604 INFO:teuthology.orchestra.run.smithi111.stdout:(94/138): gperftools-libs-2.9.1-3.el9.x86_64.rp 2.8 MB/s | 308 kB 00:00 2024-09-13T17:55:26.638 INFO:teuthology.orchestra.run.smithi111.stdout:(95/138): libarrow-doc-9.0.0-11.el9.noarch.rpm 783 kB/s | 26 kB 00:00 2024-09-13T17:55:26.788 INFO:teuthology.orchestra.run.smithi188.stdout:(133/138): NetworkManager-team-1.49.90-1.el9.x8 196 kB/s | 41 kB 00:00 2024-09-13T17:55:26.805 INFO:teuthology.orchestra.run.smithi111.stdout:(96/138): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-13T17:55:26.830 INFO:teuthology.orchestra.run.smithi111.stdout:(97/138): liboath-2.6.7-2.el9.x86_64.rpm 253 kB/s | 49 kB 00:00 2024-09-13T17:55:26.881 INFO:teuthology.orchestra.run.smithi111.stdout:(98/138): libunwind-1.6.2-1.el9.x86_64.rpm 900 kB/s | 67 kB 00:00 2024-09-13T17:55:26.939 INFO:teuthology.orchestra.run.smithi111.stdout:(99/138): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 839 kB 00:00 2024-09-13T17:55:26.964 INFO:teuthology.orchestra.run.smithi111.stdout:(100/138): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-13T17:55:26.990 INFO:teuthology.orchestra.run.smithi111.stdout:(101/138): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-09-13T17:55:27.015 INFO:teuthology.orchestra.run.smithi111.stdout:(102/138): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-13T17:55:27.040 INFO:teuthology.orchestra.run.smithi111.stdout:(103/138): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-13T17:55:27.065 INFO:teuthology.orchestra.run.smithi111.stdout:(104/138): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-13T17:55:27.097 INFO:teuthology.orchestra.run.smithi188.stdout:(134/138): NetworkManager-tui-1.49.90-1.el9.x86 806 kB/s | 249 kB 00:00 2024-09-13T17:55:27.107 INFO:teuthology.orchestra.run.smithi111.stdout:(105/138): python3-google-auth-2.34.0-1.el9.noa 5.2 MB/s | 220 kB 00:00 2024-09-13T17:55:27.133 INFO:teuthology.orchestra.run.smithi111.stdout:(106/138): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-13T17:55:27.192 INFO:teuthology.orchestra.run.smithi111.stdout:(107/138): libarrow-9.0.0-11.el9.x86_64.rpm 6.9 MB/s | 4.4 MB 00:00 2024-09-13T17:55:27.217 INFO:teuthology.orchestra.run.smithi111.stdout:(108/138): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-09-13T17:55:27.242 INFO:teuthology.orchestra.run.smithi111.stdout:(109/138): python3-jaraco-classes-3.2.1-5.el9.n 162 kB/s | 18 kB 00:00 2024-09-13T17:55:27.267 INFO:teuthology.orchestra.run.smithi111.stdout:(110/138): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-09-13T17:55:27.292 INFO:teuthology.orchestra.run.smithi111.stdout:(111/138): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-09-13T17:55:27.318 INFO:teuthology.orchestra.run.smithi111.stdout:(112/138): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-09-13T17:55:27.343 INFO:teuthology.orchestra.run.smithi111.stdout:(113/138): python3-logutils-0.3.5-21.el9.noarch 924 kB/s | 46 kB 00:00 2024-09-13T17:55:27.355 INFO:teuthology.orchestra.run.smithi188.stdout:(135/138): NetworkManager-1.49.90-1.el9.x86_64. 2.2 MB/s | 2.3 MB 00:01 2024-09-13T17:55:27.368 INFO:teuthology.orchestra.run.smithi111.stdout:(114/138): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-13T17:55:27.393 INFO:teuthology.orchestra.run.smithi111.stdout:(115/138): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-13T17:55:27.419 INFO:teuthology.orchestra.run.smithi111.stdout:(116/138): python3-portend-3.1.0-2.el9.noarch.r 656 kB/s | 16 kB 00:00 2024-09-13T17:55:27.444 INFO:teuthology.orchestra.run.smithi111.stdout:(117/138): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-13T17:55:27.469 INFO:teuthology.orchestra.run.smithi111.stdout:(118/138): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-13T17:55:27.472 INFO:teuthology.orchestra.run.smithi188.stdout:(136/138): NetworkManager-libnm-1.49.90-1.el9.x 1.9 MB/s | 1.8 MB 00:00 2024-09-13T17:55:27.495 INFO:teuthology.orchestra.run.smithi111.stdout:(119/138): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-09-13T17:55:27.520 INFO:teuthology.orchestra.run.smithi111.stdout:(120/138): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-13T17:55:27.545 INFO:teuthology.orchestra.run.smithi111.stdout:(121/138): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-13T17:55:27.573 INFO:teuthology.orchestra.run.smithi111.stdout:(122/138): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-13T17:55:27.596 INFO:teuthology.orchestra.run.smithi111.stdout:(123/138): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-13T17:55:27.598 INFO:teuthology.orchestra.run.smithi188.stdout:(137/138): openssl-3.2.2-6.el9.x86_64.rpm 2.7 MB/s | 1.4 MB 00:00 2024-09-13T17:55:27.630 INFO:teuthology.orchestra.run.smithi111.stdout:(124/138): python3-kubernetes-26.1.0-2.el9.noar 2.8 MB/s | 1.0 MB 00:00 2024-09-13T17:55:27.655 INFO:teuthology.orchestra.run.smithi111.stdout:(125/138): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-13T17:55:27.681 INFO:teuthology.orchestra.run.smithi111.stdout:(126/138): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-13T17:55:27.705 INFO:teuthology.orchestra.run.smithi111.stdout:(127/138): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-13T17:55:27.739 INFO:teuthology.orchestra.run.smithi111.stdout:(128/138): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-09-13T17:55:27.764 INFO:teuthology.orchestra.run.smithi111.stdout:(129/138): re2-20211101-20.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-13T17:55:27.840 INFO:teuthology.orchestra.run.smithi111.stdout:(130/138): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-13T17:55:28.040 INFO:teuthology.orchestra.run.smithi111.stdout:(131/138): librbd1-18.2.4-860.g6c6b083f.el9.x86 11 MB/s | 3.0 MB 00:00 2024-09-13T17:55:28.099 INFO:teuthology.orchestra.run.smithi111.stdout:(132/138): NetworkManager-1.49.90-1.el9.x86_64. 9.0 MB/s | 2.3 MB 00:00 2024-09-13T17:55:28.149 INFO:teuthology.orchestra.run.smithi111.stdout:(133/138): librados2-18.2.4-860.g6c6b083f.el9.x 8.1 MB/s | 3.3 MB 00:00 2024-09-13T17:55:28.174 INFO:teuthology.orchestra.run.smithi111.stdout:(134/138): NetworkManager-team-1.49.90-1.el9.x8 543 kB/s | 41 kB 00:00 2024-09-13T17:55:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:(135/138): openssl-3.2.2-6.el9.x86_64.rpm 9.6 MB/s | 1.4 MB 00:00 2024-09-13T17:55:28.383 INFO:teuthology.orchestra.run.smithi111.stdout:(136/138): NetworkManager-libnm-1.49.90-1.el9.x 5.3 MB/s | 1.8 MB 00:00 2024-09-13T17:55:28.408 INFO:teuthology.orchestra.run.smithi111.stdout:(137/138): NetworkManager-tui-1.49.90-1.el9.x86 960 kB/s | 249 kB 00:00 2024-09-13T17:55:28.492 INFO:teuthology.orchestra.run.smithi111.stdout:(138/138): openssl-libs-3.2.2-6.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-09-13T17:55:28.498 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-13T17:55:28.498 INFO:teuthology.orchestra.run.smithi111.stdout:Total 21 MB/s | 203 MB 00:09 2024-09-13T17:55:28.556 INFO:teuthology.orchestra.run.smithi188.stdout:(138/138): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 2.4 MB 00:01 2024-09-13T17:55:28.562 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-09-13T17:55:28.562 INFO:teuthology.orchestra.run.smithi188.stdout:Total 19 MB/s | 203 MB 00:10 2024-09-13T17:55:29.608 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-13T17:55:29.644 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-09-13T17:55:29.752 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-13T17:55:29.752 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-13T17:55:29.785 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-09-13T17:55:29.785 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-09-13T17:55:31.795 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-13T17:55:31.795 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-13T17:55:31.850 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-09-13T17:55:31.850 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-09-13T17:55:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-13T17:55:37.587 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/146 2024-09-13T17:55:37.625 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/146 2024-09-13T17:55:37.688 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/146 2024-09-13T17:55:37.720 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-09-13T17:55:37.730 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/146 2024-09-13T17:55:37.755 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/146 2024-09-13T17:55:37.807 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/146 2024-09-13T17:55:37.826 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/146 2024-09-13T17:55:37.867 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/146 2024-09-13T17:55:37.926 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/146 2024-09-13T17:55:37.968 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/146 2024-09-13T17:55:37.997 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/146 2024-09-13T17:55:38.056 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/146 2024-09-13T17:55:38.206 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/146 2024-09-13T17:55:38.256 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T17:55:38.385 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T17:55:38.386 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/146 2024-09-13T17:55:38.416 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T17:55:38.506 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T17:55:38.507 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T17:55:38.546 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 10/146 2024-09-13T17:55:38.612 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T17:55:38.641 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T17:55:38.706 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/146 2024-09-13T17:55:38.749 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T17:55:38.768 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/146 2024-09-13T17:55:38.779 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 10/146 2024-09-13T17:55:38.795 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T17:55:38.896 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T17:55:38.921 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 14/146 2024-09-13T17:55:38.944 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/146 2024-09-13T17:55:39.001 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/146 2024-09-13T17:55:39.037 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T17:55:39.123 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 14/146 2024-09-13T17:55:39.129 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T17:55:39.163 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 14/146 2024-09-13T17:55:39.259 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 15/146 2024-09-13T17:55:39.373 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 14/146 2024-09-13T17:55:39.375 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T17:55:39.461 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T17:55:39.526 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 15/146 2024-09-13T17:55:39.640 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T17:55:39.711 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T17:55:39.756 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T17:55:39.799 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/146 2024-09-13T17:55:39.833 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/146 2024-09-13T17:55:39.907 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/146 2024-09-13T17:55:39.949 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/146 2024-09-13T17:55:39.977 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/146 2024-09-13T17:55:40.008 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T17:55:40.049 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/146 2024-09-13T17:55:40.055 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 22/146 2024-09-13T17:55:40.084 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/146 2024-09-13T17:55:40.107 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 23/146 2024-09-13T17:55:40.138 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 24/146 2024-09-13T17:55:40.149 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/146 2024-09-13T17:55:40.183 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/146 2024-09-13T17:55:40.190 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 25/146 2024-09-13T17:55:40.211 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/146 2024-09-13T17:55:40.278 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 26/146 2024-09-13T17:55:40.280 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 22/146 2024-09-13T17:55:40.317 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x 27/146 2024-09-13T17:55:40.333 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 23/146 2024-09-13T17:55:40.362 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9 28/146 2024-09-13T17:55:40.373 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 24/146 2024-09-13T17:55:40.425 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 25/146 2024-09-13T17:55:40.436 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 29/146 2024-09-13T17:55:40.500 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 26/146 2024-09-13T17:55:40.535 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x 27/146 2024-09-13T17:55:40.579 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9 28/146 2024-09-13T17:55:40.594 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/146 2024-09-13T17:55:40.646 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/146 2024-09-13T17:55:40.652 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 29/146 2024-09-13T17:55:40.786 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/146 2024-09-13T17:55:40.847 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/146 2024-09-13T17:55:41.183 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/146 2024-09-13T17:55:41.266 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/146 2024-09-13T17:55:41.305 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 34/146 2024-09-13T17:55:41.342 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 35/146 2024-09-13T17:55:41.372 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/146 2024-09-13T17:55:41.384 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/146 2024-09-13T17:55:41.458 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/146 2024-09-13T17:55:41.498 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 34/146 2024-09-13T17:55:41.535 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 35/146 2024-09-13T17:55:41.540 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 37/146 2024-09-13T17:55:41.576 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/146 2024-09-13T17:55:41.593 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 38/146 2024-09-13T17:55:41.707 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : re2-1:20211101-20.el9.x86_64 39/146 2024-09-13T17:55:41.739 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 37/146 2024-09-13T17:55:41.746 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 40/146 2024-09-13T17:55:41.793 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 38/146 2024-09-13T17:55:41.801 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/146 2024-09-13T17:55:41.875 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/146 2024-09-13T17:55:41.906 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 43/146 2024-09-13T17:55:41.910 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : re2-1:20211101-20.el9.x86_64 39/146 2024-09-13T17:55:41.942 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 44/146 2024-09-13T17:55:41.947 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 40/146 2024-09-13T17:55:41.988 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/146 2024-09-13T17:55:42.002 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/146 2024-09-13T17:55:42.034 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/146 2024-09-13T17:55:42.074 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/146 2024-09-13T17:55:42.084 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/146 2024-09-13T17:55:42.107 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 43/146 2024-09-13T17:55:42.143 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 44/146 2024-09-13T17:55:42.189 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/146 2024-09-13T17:55:42.221 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/146 2024-09-13T17:55:42.235 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/146 2024-09-13T17:55:42.260 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/146 2024-09-13T17:55:42.285 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/146 2024-09-13T17:55:42.304 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 50/146 2024-09-13T17:55:42.404 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/146 2024-09-13T17:55:42.429 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/146 2024-09-13T17:55:42.437 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 52/146 2024-09-13T17:55:42.471 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/146 2024-09-13T17:55:42.477 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/146 2024-09-13T17:55:42.510 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 54/146 2024-09-13T17:55:42.511 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 50/146 2024-09-13T17:55:42.537 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 55/146 2024-09-13T17:55:42.578 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fmt-8.1.1-5.el9.x86_64 56/146 2024-09-13T17:55:42.607 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 57/146 2024-09-13T17:55:42.614 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/146 2024-09-13T17:55:42.641 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 58/146 2024-09-13T17:55:42.647 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 52/146 2024-09-13T17:55:42.678 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : uuid-1.6.2-55.el9.x86_64 59/146 2024-09-13T17:55:42.687 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/146 2024-09-13T17:55:42.708 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 60/146 2024-09-13T17:55:42.720 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 54/146 2024-09-13T17:55:42.742 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 61/146 2024-09-13T17:55:42.747 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 55/146 2024-09-13T17:55:42.776 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 62/146 2024-09-13T17:55:42.789 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fmt-8.1.1-5.el9.x86_64 56/146 2024-09-13T17:55:42.817 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 63/146 2024-09-13T17:55:42.825 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 57/146 2024-09-13T17:55:42.852 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 64/146 2024-09-13T17:55:42.868 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 58/146 2024-09-13T17:55:42.886 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 65/146 2024-09-13T17:55:42.904 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : uuid-1.6.2-55.el9.x86_64 59/146 2024-09-13T17:55:42.933 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 66/146 2024-09-13T17:55:42.934 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 60/146 2024-09-13T17:55:42.969 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 61/146 2024-09-13T17:55:43.002 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 62/146 2024-09-13T17:55:43.030 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/146 2024-09-13T17:55:43.044 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 63/146 2024-09-13T17:55:43.078 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 64/146 2024-09-13T17:55:43.087 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 68/146 2024-09-13T17:55:43.113 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 65/146 2024-09-13T17:55:43.160 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 66/146 2024-09-13T17:55:43.257 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/146 2024-09-13T17:55:43.315 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 68/146 2024-09-13T17:55:44.006 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 69/146 2024-09-13T17:55:44.034 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 69/146 2024-09-13T17:55:44.060 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 70/146 2024-09-13T17:55:44.094 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 71/146 2024-09-13T17:55:44.095 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 70/146 2024-09-13T17:55:44.130 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 71/146 2024-09-13T17:55:44.204 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 72/146 2024-09-13T17:55:44.226 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 72/146 2024-09-13T17:55:44.263 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 73/146 2024-09-13T17:55:44.447 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-0.3.26-2.el9.x86_64 74/146 2024-09-13T17:55:44.478 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 75/146 2024-09-13T17:55:44.514 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 73/146 2024-09-13T17:55:44.560 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 76/146 2024-09-13T17:55:44.699 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-0.3.26-2.el9.x86_64 74/146 2024-09-13T17:55:44.729 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 75/146 2024-09-13T17:55:44.817 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 76/146 2024-09-13T17:55:45.210 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 77/146 2024-09-13T17:55:45.329 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T17:55:45.473 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 77/146 2024-09-13T17:55:45.585 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T17:55:46.567 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T17:55:46.653 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 80/146 2024-09-13T17:55:46.681 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 81/146 2024-09-13T17:55:46.709 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 82/146 2024-09-13T17:55:46.831 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T17:55:46.916 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 80/146 2024-09-13T17:55:46.944 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 81/146 2024-09-13T17:55:46.977 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 82/146 2024-09-13T17:55:47.009 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 83/146 2024-09-13T17:55:47.055 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 84/146 2024-09-13T17:55:47.138 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 84/146 2024-09-13T17:55:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 85/146 2024-09-13T17:55:47.213 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 86/146 2024-09-13T17:55:47.284 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 83/146 2024-09-13T17:55:47.332 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 84/146 2024-09-13T17:55:47.415 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 84/146 2024-09-13T17:55:47.445 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 85/146 2024-09-13T17:55:47.481 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 86/146 2024-09-13T17:55:47.619 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 87/146 2024-09-13T17:55:47.658 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 88/146 2024-09-13T17:55:47.729 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 88/146 2024-09-13T17:55:47.760 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 89/146 2024-09-13T17:55:47.906 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 87/146 2024-09-13T17:55:47.935 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 88/146 2024-09-13T17:55:48.005 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 88/146 2024-09-13T17:55:48.028 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 89/146 2024-09-13T17:55:49.828 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T17:55:49.865 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T17:55:50.031 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T17:55:50.066 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T17:55:50.389 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T17:55:50.428 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 91/146 2024-09-13T17:55:50.523 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 91/146 2024-09-13T17:55:50.523 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T17:55:50.523 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T17:55:50.524 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:55:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 92/146 2024-09-13T17:55:50.592 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T17:55:50.628 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 91/146 2024-09-13T17:55:50.717 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 91/146 2024-09-13T17:55:50.718 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T17:55:50.718 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T17:55:50.718 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:55:50.762 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 92/146 2024-09-13T17:56:02.050 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 92/146 2024-09-13T17:56:02.051 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /sys 2024-09-13T17:56:02.051 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /proc 2024-09-13T17:56:02.051 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /mnt 2024-09-13T17:56:02.051 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /var/tmp 2024-09-13T17:56:02.051 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /home 2024-09-13T17:56:02.051 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /root 2024-09-13T17:56:02.051 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /tmp 2024-09-13T17:56:02.051 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:02.070 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 92/146 2024-09-13T17:56:02.070 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /sys 2024-09-13T17:56:02.070 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /proc 2024-09-13T17:56:02.070 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /mnt 2024-09-13T17:56:02.070 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /var/tmp 2024-09-13T17:56:02.070 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /home 2024-09-13T17:56:02.071 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /root 2024-09-13T17:56:02.071 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /tmp 2024-09-13T17:56:02.071 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:02.206 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 93/146 2024-09-13T17:56:02.223 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 93/146 2024-09-13T17:56:02.223 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:02.223 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T17:56:02.223 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:02.233 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 93/146 2024-09-13T17:56:02.249 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 93/146 2024-09-13T17:56:02.250 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:02.250 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T17:56:02.250 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:03.676 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 94/146 2024-09-13T17:56:03.703 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 94/146 2024-09-13T17:56:03.703 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:03.703 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T17:56:03.703 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T17:56:03.703 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T17:56:03.703 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:03.720 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 94/146 2024-09-13T17:56:03.747 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 94/146 2024-09-13T17:56:03.748 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:03.748 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T17:56:03.748 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T17:56:03.748 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T17:56:03.748 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:03.901 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 95/146 2024-09-13T17:56:03.925 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 95/146 2024-09-13T17:56:03.925 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:03.925 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T17:56:03.925 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T17:56:03.925 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T17:56:03.925 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:03.954 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 95/146 2024-09-13T17:56:03.977 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 95/146 2024-09-13T17:56:03.978 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:03.978 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T17:56:03.978 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T17:56:03.978 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T17:56:03.978 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:04.361 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 96/146 2024-09-13T17:56:04.384 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 96/146 2024-09-13T17:56:04.384 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:04.384 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T17:56:04.385 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T17:56:04.385 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T17:56:04.385 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:04.413 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 96/146 2024-09-13T17:56:04.436 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 96/146 2024-09-13T17:56:04.437 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:04.437 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T17:56:04.437 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T17:56:04.437 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T17:56:04.437 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:04.490 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : automake-1.16.2-8.el9.noarch 97/146 2024-09-13T17:56:04.539 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ply-3.11-14.el9.noarch 98/146 2024-09-13T17:56:04.551 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : automake-1.16.2-8.el9.noarch 97/146 2024-09-13T17:56:04.602 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 99/146 2024-09-13T17:56:04.607 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ply-3.11-14.el9.noarch 98/146 2024-09-13T17:56:04.664 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 99/146 2024-09-13T17:56:04.789 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 100/146 2024-09-13T17:56:04.846 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 101/146 2024-09-13T17:56:04.849 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 100/146 2024-09-13T17:56:04.906 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 101/146 2024-09-13T17:56:04.915 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 102/146 2024-09-13T17:56:04.982 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 102/146 2024-09-13T17:56:05.003 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 103/146 2024-09-13T17:56:05.071 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 103/146 2024-09-13T17:56:05.142 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/146 2024-09-13T17:56:05.179 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 105/146 2024-09-13T17:56:05.211 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 106/146 2024-09-13T17:56:05.220 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/146 2024-09-13T17:56:05.255 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 105/146 2024-09-13T17:56:05.258 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fuse-2.9.9-16.el9.x86_64 107/146 2024-09-13T17:56:05.287 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 106/146 2024-09-13T17:56:05.315 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fuse-2.9.9-16.el9.x86_64 107/146 2024-09-13T17:56:05.645 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 108/146 2024-09-13T17:56:05.671 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 108/146 2024-09-13T17:56:05.671 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 108/146 2024-09-13T17:56:05.698 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 108/146 2024-09-13T17:56:05.699 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el 109/146 2024-09-13T17:56:05.726 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el 109/146 2024-09-13T17:56:05.755 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.e 110/146 2024-09-13T17:56:05.777 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 111/146 2024-09-13T17:56:05.789 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.e 110/146 2024-09-13T17:56:05.812 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 111/146 2024-09-13T17:56:06.811 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 111/146 2024-09-13T17:56:06.845 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 112/146 2024-09-13T17:56:06.846 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 111/146 2024-09-13T17:56:06.881 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 112/146 2024-09-13T17:56:07.804 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 112/146 2024-09-13T17:56:07.838 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 113/146 2024-09-13T17:56:07.845 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 112/146 2024-09-13T17:56:07.873 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 113/146 2024-09-13T17:56:07.938 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 113/146 2024-09-13T17:56:07.974 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 113/146 2024-09-13T17:56:08.099 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9 114/146 2024-09-13T17:56:08.123 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 115/146 2024-09-13T17:56:08.147 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 115/146 2024-09-13T17:56:08.147 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:08.147 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T17:56:08.147 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T17:56:08.147 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T17:56:08.148 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:08.175 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9 114/146 2024-09-13T17:56:08.198 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 116/146 2024-09-13T17:56:08.208 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 115/146 2024-09-13T17:56:08.210 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 116/146 2024-09-13T17:56:08.232 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 115/146 2024-09-13T17:56:08.232 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:08.233 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T17:56:08.233 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T17:56:08.233 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T17:56:08.233 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:08.284 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 116/146 2024-09-13T17:56:08.296 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 116/146 2024-09-13T17:56:08.296 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 117/146 2024-09-13T17:56:08.350 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 118/146 2024-09-13T17:56:08.391 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 117/146 2024-09-13T17:56:08.446 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 118/146 2024-09-13T17:56:09.067 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libtool-2.4.6-46.el9.x86_64 119/146 2024-09-13T17:56:09.093 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 120/146 2024-09-13T17:56:09.116 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 120/146 2024-09-13T17:56:09.117 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:09.117 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T17:56:09.117 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:56:09.117 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:56:09.117 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:09.160 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 121/146 2024-09-13T17:56:09.183 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 121/146 2024-09-13T17:56:09.184 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:09.184 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T17:56:09.184 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:09.192 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libtool-2.4.6-46.el9.x86_64 119/146 2024-09-13T17:56:09.220 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 120/146 2024-09-13T17:56:09.227 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 122/146 2024-09-13T17:56:09.243 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 120/146 2024-09-13T17:56:09.244 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:09.244 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T17:56:09.244 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:56:09.244 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:56:09.244 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:09.251 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 122/146 2024-09-13T17:56:09.251 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:09.251 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-13T17:56:09.251 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T17:56:09.251 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T17:56:09.251 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:09.287 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 121/146 2024-09-13T17:56:09.310 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 121/146 2024-09-13T17:56:09.310 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:09.310 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T17:56:09.310 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:09.354 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 122/146 2024-09-13T17:56:09.378 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 122/146 2024-09-13T17:56:09.378 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:09.378 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-13T17:56:09.378 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T17:56:09.378 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T17:56:09.378 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:09.545 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 123/146 2024-09-13T17:56:09.569 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 123/146 2024-09-13T17:56:09.569 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:09.569 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T17:56:09.569 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T17:56:09.569 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T17:56:09.569 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:09.680 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 123/146 2024-09-13T17:56:09.703 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 123/146 2024-09-13T17:56:09.703 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:56:09.703 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T17:56:09.704 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T17:56:09.704 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T17:56:09.704 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:13.232 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 124/146 2024-09-13T17:56:13.291 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 125/146 2024-09-13T17:56:13.347 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 126/146 2024-09-13T17:56:13.379 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 127/146 2024-09-13T17:56:13.401 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 124/146 2024-09-13T17:56:13.442 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 125/146 2024-09-13T17:56:13.482 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 126/146 2024-09-13T17:56:13.515 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 127/146 2024-09-13T17:56:13.564 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 128/146 2024-09-13T17:56:13.610 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 129/146 2024-09-13T17:56:13.641 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_6 130/146 2024-09-13T17:56:13.670 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 131/146 2024-09-13T17:56:13.703 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 128/146 2024-09-13T17:56:13.716 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 132/146 2024-09-13T17:56:13.746 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 129/146 2024-09-13T17:56:13.777 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_6 130/146 2024-09-13T17:56:13.815 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 131/146 2024-09-13T17:56:13.853 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 132/146 2024-09-13T17:56:14.338 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T17:56:14.373 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T17:56:14.394 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T17:56:14.437 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T17:56:14.485 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T17:56:14.506 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T17:56:15.235 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 134/146 2024-09-13T17:56:15.328 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 135/146 2024-09-13T17:56:15.340 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 134/146 2024-09-13T17:56:15.380 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 136/146 2024-09-13T17:56:15.415 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 135/146 2024-09-13T17:56:15.462 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 136/146 2024-09-13T17:56:15.477 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flex-2.6.4-9.el9.x86_64 137/146 2024-09-13T17:56:15.506 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : bison-3.7.4-5.el9.x86_64 138/146 2024-09-13T17:56:15.531 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 139/146 2024-09-13T17:56:15.549 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flex-2.6.4-9.el9.x86_64 137/146 2024-09-13T17:56:15.566 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 140/146 2024-09-13T17:56:15.566 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T17:56:15.577 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : bison-3.7.4-5.el9.x86_64 138/146 2024-09-13T17:56:15.610 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 139/146 2024-09-13T17:56:15.636 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 140/146 2024-09-13T17:56:15.637 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T17:56:15.649 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T17:56:15.650 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T17:56:15.721 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T17:56:15.721 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T17:56:15.734 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T17:56:15.759 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 143/146 2024-09-13T17:56:15.765 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T17:56:15.765 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T17:56:15.813 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T17:56:15.838 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 143/146 2024-09-13T17:56:15.844 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T17:56:15.844 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T17:56:15.944 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T17:56:15.944 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T17:56:16.023 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T17:56:16.023 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T17:56:16.028 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T17:56:16.099 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T17:56:16.101 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T17:56:16.163 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T17:56:21.445 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T17:56:21.445 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 1/146 2024-09-13T17:56:21.445 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2/146 2024-09-13T17:56:21.445 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 3/146 2024-09-13T17:56:21.445 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 4/146 2024-09-13T17:56:21.446 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 5/146 2024-09-13T17:56:21.446 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 6/146 2024-09-13T17:56:21.446 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 7/146 2024-09-13T17:56:21.446 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T17:56:21.446 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T17:56:21.446 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 10/146 2024-09-13T17:56:21.446 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 11/146 2024-09-13T17:56:21.446 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 12/146 2024-09-13T17:56:21.446 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T17:56:21.446 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_6 14/146 2024-09-13T17:56:21.446 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 15/146 2024-09-13T17:56:21.446 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 16/146 2024-09-13T17:56:21.446 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 17/146 2024-09-13T17:56:21.447 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 18/146 2024-09-13T17:56:21.447 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 19/146 2024-09-13T17:56:21.447 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9 20/146 2024-09-13T17:56:21.447 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x 21/146 2024-09-13T17:56:21.447 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 22/146 2024-09-13T17:56:21.447 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 23/146 2024-09-13T17:56:21.447 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 24/146 2024-09-13T17:56:21.447 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 25/146 2024-09-13T17:56:21.448 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 26/146 2024-09-13T17:56:21.448 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 27/146 2024-09-13T17:56:21.448 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 28/146 2024-09-13T17:56:21.448 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.e 29/146 2024-09-13T17:56:21.448 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 30/146 2024-09-13T17:56:21.448 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 31/146 2024-09-13T17:56:21.448 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 32/146 2024-09-13T17:56:21.448 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9 33/146 2024-09-13T17:56:21.448 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 34/146 2024-09-13T17:56:21.448 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el 35/146 2024-09-13T17:56:21.448 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 36/146 2024-09-13T17:56:21.448 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 37/146 2024-09-13T17:56:21.449 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 38/146 2024-09-13T17:56:21.449 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 39/146 2024-09-13T17:56:21.449 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 40/146 2024-09-13T17:56:21.449 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 41/146 2024-09-13T17:56:21.449 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 42/146 2024-09-13T17:56:21.449 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 43/146 2024-09-13T17:56:21.449 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ply-3.11-14.el9.noarch 44/146 2024-09-13T17:56:21.449 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 45/146 2024-09-13T17:56:21.449 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 46/146 2024-09-13T17:56:21.449 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 47/146 2024-09-13T17:56:21.449 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : automake-1.16.2-8.el9.noarch 48/146 2024-09-13T17:56:21.449 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : bison-3.7.4-5.el9.x86_64 49/146 2024-09-13T17:56:21.450 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 50/146 2024-09-13T17:56:21.450 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 51/146 2024-09-13T17:56:21.450 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flex-2.6.4-9.el9.x86_64 52/146 2024-09-13T17:56:21.450 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 53/146 2024-09-13T17:56:21.450 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 54/146 2024-09-13T17:56:21.450 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 55/146 2024-09-13T17:56:21.450 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 56/146 2024-09-13T17:56:21.450 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 57/146 2024-09-13T17:56:21.450 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 58/146 2024-09-13T17:56:21.450 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 59/146 2024-09-13T17:56:21.450 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 60/146 2024-09-13T17:56:21.450 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 61/146 2024-09-13T17:56:21.451 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 62/146 2024-09-13T17:56:21.451 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 63/146 2024-09-13T17:56:21.451 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 64/146 2024-09-13T17:56:21.451 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 65/146 2024-09-13T17:56:21.451 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 66/146 2024-09-13T17:56:21.451 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-13.16-1.el9.x86_64 67/146 2024-09-13T17:56:21.451 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 68/146 2024-09-13T17:56:21.451 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 69/146 2024-09-13T17:56:21.451 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 70/146 2024-09-13T17:56:21.451 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 71/146 2024-09-13T17:56:21.451 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 72/146 2024-09-13T17:56:21.451 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 73/146 2024-09-13T17:56:21.451 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 74/146 2024-09-13T17:56:21.451 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 75/146 2024-09-13T17:56:21.452 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 76/146 2024-09-13T17:56:21.452 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 77/146 2024-09-13T17:56:21.452 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T17:56:21.452 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T17:56:21.452 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 80/146 2024-09-13T17:56:21.452 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 81/146 2024-09-13T17:56:21.452 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 82/146 2024-09-13T17:56:21.452 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 83/146 2024-09-13T17:56:21.452 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 84/146 2024-09-13T17:56:21.452 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 85/146 2024-09-13T17:56:21.452 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/146 2024-09-13T17:56:21.452 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 87/146 2024-09-13T17:56:21.453 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 88/146 2024-09-13T17:56:21.453 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 89/146 2024-09-13T17:56:21.453 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 90/146 2024-09-13T17:56:21.453 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 91/146 2024-09-13T17:56:21.453 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 92/146 2024-09-13T17:56:21.453 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 93/146 2024-09-13T17:56:21.453 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 94/146 2024-09-13T17:56:21.453 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 95/146 2024-09-13T17:56:21.453 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 96/146 2024-09-13T17:56:21.453 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 97/146 2024-09-13T17:56:21.453 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/146 2024-09-13T17:56:21.453 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/146 2024-09-13T17:56:21.454 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 100/146 2024-09-13T17:56:21.454 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 101/146 2024-09-13T17:56:21.454 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/146 2024-09-13T17:56:21.454 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/146 2024-09-13T17:56:21.454 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 104/146 2024-09-13T17:56:21.454 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/146 2024-09-13T17:56:21.454 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/146 2024-09-13T17:56:21.454 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 107/146 2024-09-13T17:56:21.454 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/146 2024-09-13T17:56:21.454 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/146 2024-09-13T17:56:21.454 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/146 2024-09-13T17:56:21.454 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/146 2024-09-13T17:56:21.455 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/146 2024-09-13T17:56:21.455 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 113/146 2024-09-13T17:56:21.455 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 114/146 2024-09-13T17:56:21.455 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 115/146 2024-09-13T17:56:21.455 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 116/146 2024-09-13T17:56:21.455 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 117/146 2024-09-13T17:56:21.455 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 118/146 2024-09-13T17:56:21.455 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/146 2024-09-13T17:56:21.455 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 120/146 2024-09-13T17:56:21.455 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 121/146 2024-09-13T17:56:21.455 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 122/146 2024-09-13T17:56:21.455 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 123/146 2024-09-13T17:56:21.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 124/146 2024-09-13T17:56:21.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 125/146 2024-09-13T17:56:21.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 126/146 2024-09-13T17:56:21.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 127/146 2024-09-13T17:56:21.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 128/146 2024-09-13T17:56:21.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : re2-1:20211101-20.el9.x86_64 129/146 2024-09-13T17:56:21.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 130/146 2024-09-13T17:56:21.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 131/146 2024-09-13T17:56:21.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 132/146 2024-09-13T17:56:21.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 133/146 2024-09-13T17:56:21.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 134/146 2024-09-13T17:56:21.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 135/146 2024-09-13T17:56:21.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 136/146 2024-09-13T17:56:21.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 137/146 2024-09-13T17:56:21.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 138/146 2024-09-13T17:56:21.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 139/146 2024-09-13T17:56:21.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 140/146 2024-09-13T17:56:21.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 141/146 2024-09-13T17:56:21.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 142/146 2024-09-13T17:56:21.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 143/146 2024-09-13T17:56:21.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 144/146 2024-09-13T17:56:21.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 145/146 2024-09-13T17:56:21.554 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T17:56:21.554 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 1/146 2024-09-13T17:56:21.554 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2/146 2024-09-13T17:56:21.554 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 3/146 2024-09-13T17:56:21.554 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 4/146 2024-09-13T17:56:21.555 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 5/146 2024-09-13T17:56:21.555 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 6/146 2024-09-13T17:56:21.555 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 7/146 2024-09-13T17:56:21.555 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T17:56:21.555 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T17:56:21.555 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 10/146 2024-09-13T17:56:21.555 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 11/146 2024-09-13T17:56:21.555 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 12/146 2024-09-13T17:56:21.555 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T17:56:21.555 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_6 14/146 2024-09-13T17:56:21.555 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 15/146 2024-09-13T17:56:21.555 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 16/146 2024-09-13T17:56:21.555 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 17/146 2024-09-13T17:56:21.555 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 18/146 2024-09-13T17:56:21.556 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 19/146 2024-09-13T17:56:21.556 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9 20/146 2024-09-13T17:56:21.556 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x 21/146 2024-09-13T17:56:21.556 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 22/146 2024-09-13T17:56:21.556 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 23/146 2024-09-13T17:56:21.556 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 24/146 2024-09-13T17:56:21.556 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 25/146 2024-09-13T17:56:21.556 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 26/146 2024-09-13T17:56:21.556 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 27/146 2024-09-13T17:56:21.556 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 28/146 2024-09-13T17:56:21.556 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.e 29/146 2024-09-13T17:56:21.556 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 30/146 2024-09-13T17:56:21.556 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 31/146 2024-09-13T17:56:21.557 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 32/146 2024-09-13T17:56:21.557 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9 33/146 2024-09-13T17:56:21.557 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 34/146 2024-09-13T17:56:21.557 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el 35/146 2024-09-13T17:56:21.557 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 36/146 2024-09-13T17:56:21.557 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 37/146 2024-09-13T17:56:21.557 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 38/146 2024-09-13T17:56:21.557 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 39/146 2024-09-13T17:56:21.557 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 40/146 2024-09-13T17:56:21.557 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 41/146 2024-09-13T17:56:21.557 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 42/146 2024-09-13T17:56:21.557 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 43/146 2024-09-13T17:56:21.557 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ply-3.11-14.el9.noarch 44/146 2024-09-13T17:56:21.558 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 45/146 2024-09-13T17:56:21.558 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 46/146 2024-09-13T17:56:21.558 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 47/146 2024-09-13T17:56:21.558 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : automake-1.16.2-8.el9.noarch 48/146 2024-09-13T17:56:21.558 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : bison-3.7.4-5.el9.x86_64 49/146 2024-09-13T17:56:21.558 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 50/146 2024-09-13T17:56:21.558 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 51/146 2024-09-13T17:56:21.558 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flex-2.6.4-9.el9.x86_64 52/146 2024-09-13T17:56:21.558 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 53/146 2024-09-13T17:56:21.558 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 54/146 2024-09-13T17:56:21.558 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 55/146 2024-09-13T17:56:21.558 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 56/146 2024-09-13T17:56:21.558 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 57/146 2024-09-13T17:56:21.559 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 58/146 2024-09-13T17:56:21.559 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 59/146 2024-09-13T17:56:21.559 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 60/146 2024-09-13T17:56:21.559 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 61/146 2024-09-13T17:56:21.559 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 62/146 2024-09-13T17:56:21.559 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 63/146 2024-09-13T17:56:21.559 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 64/146 2024-09-13T17:56:21.559 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 65/146 2024-09-13T17:56:21.559 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 66/146 2024-09-13T17:56:21.559 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-13.16-1.el9.x86_64 67/146 2024-09-13T17:56:21.559 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 68/146 2024-09-13T17:56:21.559 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 69/146 2024-09-13T17:56:21.559 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 70/146 2024-09-13T17:56:21.560 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 71/146 2024-09-13T17:56:21.560 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 72/146 2024-09-13T17:56:21.560 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 73/146 2024-09-13T17:56:21.560 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 74/146 2024-09-13T17:56:21.560 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 75/146 2024-09-13T17:56:21.560 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 76/146 2024-09-13T17:56:21.560 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 77/146 2024-09-13T17:56:21.560 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T17:56:21.560 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T17:56:21.560 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 80/146 2024-09-13T17:56:21.560 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 81/146 2024-09-13T17:56:21.560 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 82/146 2024-09-13T17:56:21.560 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 83/146 2024-09-13T17:56:21.561 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 84/146 2024-09-13T17:56:21.561 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 85/146 2024-09-13T17:56:21.561 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/146 2024-09-13T17:56:21.561 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 87/146 2024-09-13T17:56:21.561 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 88/146 2024-09-13T17:56:21.561 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 89/146 2024-09-13T17:56:21.561 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 90/146 2024-09-13T17:56:21.561 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 91/146 2024-09-13T17:56:21.561 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 92/146 2024-09-13T17:56:21.561 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 93/146 2024-09-13T17:56:21.561 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 94/146 2024-09-13T17:56:21.561 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 95/146 2024-09-13T17:56:21.561 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 96/146 2024-09-13T17:56:21.562 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 97/146 2024-09-13T17:56:21.562 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/146 2024-09-13T17:56:21.562 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/146 2024-09-13T17:56:21.562 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 100/146 2024-09-13T17:56:21.562 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 101/146 2024-09-13T17:56:21.562 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/146 2024-09-13T17:56:21.562 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/146 2024-09-13T17:56:21.562 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 104/146 2024-09-13T17:56:21.562 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/146 2024-09-13T17:56:21.562 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/146 2024-09-13T17:56:21.562 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 107/146 2024-09-13T17:56:21.562 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/146 2024-09-13T17:56:21.562 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/146 2024-09-13T17:56:21.562 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/146 2024-09-13T17:56:21.563 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/146 2024-09-13T17:56:21.563 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/146 2024-09-13T17:56:21.563 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 113/146 2024-09-13T17:56:21.563 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 114/146 2024-09-13T17:56:21.563 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 115/146 2024-09-13T17:56:21.563 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 116/146 2024-09-13T17:56:21.563 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 117/146 2024-09-13T17:56:21.563 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 118/146 2024-09-13T17:56:21.563 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/146 2024-09-13T17:56:21.563 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 120/146 2024-09-13T17:56:21.563 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 121/146 2024-09-13T17:56:21.563 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 122/146 2024-09-13T17:56:21.563 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 123/146 2024-09-13T17:56:21.564 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 124/146 2024-09-13T17:56:21.564 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 125/146 2024-09-13T17:56:21.564 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 126/146 2024-09-13T17:56:21.564 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 127/146 2024-09-13T17:56:21.564 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 128/146 2024-09-13T17:56:21.564 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : re2-1:20211101-20.el9.x86_64 129/146 2024-09-13T17:56:21.564 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 130/146 2024-09-13T17:56:21.564 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 131/146 2024-09-13T17:56:21.564 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 132/146 2024-09-13T17:56:21.564 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 133/146 2024-09-13T17:56:21.564 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 134/146 2024-09-13T17:56:21.564 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 135/146 2024-09-13T17:56:21.564 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 136/146 2024-09-13T17:56:21.564 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 137/146 2024-09-13T17:56:21.565 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 138/146 2024-09-13T17:56:21.565 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 139/146 2024-09-13T17:56:21.565 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 140/146 2024-09-13T17:56:21.565 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 141/146 2024-09-13T17:56:21.565 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 142/146 2024-09-13T17:56:21.565 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 143/146 2024-09-13T17:56:21.565 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 144/146 2024-09-13T17:56:21.565 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 145/146 2024-09-13T17:56:23.849 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T17:56:23.849 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:23.849 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2024-09-13T17:56:23.849 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-13T17:56:23.849 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-13T17:56:23.849 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-13T17:56:23.849 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-13T17:56:23.849 INFO:teuthology.orchestra.run.smithi188.stdout: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.849 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.849 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-13T17:56:23.850 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-13T17:56:23.850 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-09-13T17:56:23.850 INFO:teuthology.orchestra.run.smithi188.stdout: automake-1.16.2-8.el9.noarch 2024-09-13T17:56:23.850 INFO:teuthology.orchestra.run.smithi188.stdout: bison-3.7.4-5.el9.x86_64 2024-09-13T17:56:23.850 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T17:56:23.850 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.850 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.850 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.850 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.850 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:23.850 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.850 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.851 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.851 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:23.851 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:23.851 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:23.851 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:23.851 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:23.851 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.851 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.851 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:23.851 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.851 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.851 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.851 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:23.851 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:23.852 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.852 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:23.852 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-13T17:56:23.852 INFO:teuthology.orchestra.run.smithi188.stdout: flex-2.6.4-9.el9.x86_64 2024-09-13T17:56:23.852 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T17:56:23.852 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T17:56:23.852 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T17:56:23.852 INFO:teuthology.orchestra.run.smithi188.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T17:56:23.852 INFO:teuthology.orchestra.run.smithi188.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-13T17:56:23.852 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T17:56:23.852 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-13T17:56:23.852 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T17:56:23.853 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T17:56:23.853 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.853 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.853 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.853 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T17:56:23.853 INFO:teuthology.orchestra.run.smithi188.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T17:56:23.853 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T17:56:23.853 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T17:56:23.853 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T17:56:23.853 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.853 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.853 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T17:56:23.853 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.854 INFO:teuthology.orchestra.run.smithi188.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-13T17:56:23.854 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T17:56:23.854 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-13T17:56:23.854 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-13T17:56:23.854 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T17:56:23.854 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T17:56:23.854 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T17:56:23.854 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-13T17:56:23.854 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T17:56:23.854 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-13.16-1.el9.x86_64 2024-09-13T17:56:23.854 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-13T17:56:23.854 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-13T17:56:23.854 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-13T17:56:23.855 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T17:56:23.855 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T17:56:23.855 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T17:56:23.855 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T17:56:23.855 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.855 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.855 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.855 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T17:56:23.855 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T17:56:23.855 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T17:56:23.855 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T17:56:23.855 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T17:56:23.856 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T17:56:23.856 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T17:56:23.856 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T17:56:23.856 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T17:56:23.856 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T17:56:23.856 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T17:56:23.856 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T17:56:23.856 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T17:56:23.856 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T17:56:23.856 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T17:56:23.856 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T17:56:23.856 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T17:56:23.856 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T17:56:23.857 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T17:56:23.857 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T17:56:23.857 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T17:56:23.857 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T17:56:23.857 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T17:56:23.857 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T17:56:23.857 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T17:56:23.857 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T17:56:23.857 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T17:56:23.857 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T17:56:23.857 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T17:56:23.857 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T17:56:23.857 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T17:56:23.858 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T17:56:23.858 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T17:56:23.858 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T17:56:23.858 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.858 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.858 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T17:56:23.858 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T17:56:23.858 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T17:56:23.858 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.858 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T17:56:23.858 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T17:56:23.858 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T17:56:23.858 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T17:56:23.858 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T17:56:23.859 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T17:56:23.859 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T17:56:23.859 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T17:56:23.859 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T17:56:23.859 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T17:56:23.859 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T17:56:23.859 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.859 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.859 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:23.859 INFO:teuthology.orchestra.run.smithi188.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T17:56:23.859 INFO:teuthology.orchestra.run.smithi188.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T17:56:23.859 INFO:teuthology.orchestra.run.smithi188.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T17:56:23.859 INFO:teuthology.orchestra.run.smithi188.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-13T17:56:23.859 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-13T17:56:23.860 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T17:56:23.860 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-13T17:56:23.860 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:23.860 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-09-13T17:56:24.129 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T17:56:24.129 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:24.129 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2024-09-13T17:56:24.129 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-13T17:56:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-13T17:56:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-13T17:56:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-13T17:56:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-13T17:56:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-13T17:56:24.130 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-13T17:56:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: automake-1.16.2-8.el9.noarch 2024-09-13T17:56:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: bison-3.7.4-5.el9.x86_64 2024-09-13T17:56:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T17:56:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:56:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-13T17:56:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: flex-2.6.4-9.el9.x86_64 2024-09-13T17:56:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T17:56:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T17:56:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T17:56:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T17:56:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-13T17:56:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T17:56:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-13T17:56:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T17:56:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T17:56:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T17:56:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T17:56:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T17:56:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T17:56:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T17:56:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T17:56:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-13T17:56:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T17:56:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-13T17:56:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-13T17:56:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T17:56:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T17:56:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T17:56:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-13T17:56:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T17:56:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-13.16-1.el9.x86_64 2024-09-13T17:56:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-13T17:56:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-13T17:56:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-13T17:56:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T17:56:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T17:56:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T17:56:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T17:56:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T17:56:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T17:56:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T17:56:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T17:56:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T17:56:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T17:56:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T17:56:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T17:56:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T17:56:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T17:56:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T17:56:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T17:56:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T17:56:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T17:56:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T17:56:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T17:56:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T17:56:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T17:56:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T17:56:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T17:56:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T17:56:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T17:56:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T17:56:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T17:56:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T17:56:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T17:56:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T17:56:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T17:56:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T17:56:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T17:56:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T17:56:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T17:56:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T17:56:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T17:56:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T17:56:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T17:56:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T17:56:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T17:56:24.139 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T17:56:24.139 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T17:56:24.139 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T17:56:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T17:56:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T17:56:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T17:56:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T17:56:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T17:56:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T17:56:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T17:56:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:56:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T17:56:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T17:56:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T17:56:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-13T17:56:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-13T17:56:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T17:56:24.142 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-13T17:56:24.142 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:24.142 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-13T17:56:24.247 DEBUG:teuthology.parallel:result is None 2024-09-13T17:56:24.534 DEBUG:teuthology.parallel:result is None 2024-09-13T17:56:24.535 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:56:24.692 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T17:56:24.715 INFO:teuthology.orchestra.run.smithi111.stdout:18.2.4-860.g6c6b083f.el9 2024-09-13T17:56:24.716 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-860.g6c6b083f.el9 2024-09-13T17:56:24.716 INFO:teuthology.task.install:The correct ceph version 18.2.4-860.g6c6b083f is installed. 2024-09-13T17:56:24.718 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:56:24.875 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T17:56:24.899 INFO:teuthology.orchestra.run.smithi188.stdout:18.2.4-860.g6c6b083f.el9 2024-09-13T17:56:24.899 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-860.g6c6b083f.el9 2024-09-13T17:56:24.899 INFO:teuthology.task.install:The correct ceph version 18.2.4-860.g6c6b083f is installed. 2024-09-13T17:56:24.901 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T17:56:24.901 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:24.902 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T17:56:24.930 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:56:24.930 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T17:56:24.968 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T17:56:24.968 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:24.969 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T17:56:24.996 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T17:56:25.060 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:56:25.060 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T17:56:25.086 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T17:56:25.152 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T17:56:25.152 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:25.153 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T17:56:25.177 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T17:56:25.242 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:56:25.242 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T17:56:25.268 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T17:56:25.334 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T17:56:25.334 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:25.334 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T17:56:25.358 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T17:56:25.423 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:56:25.423 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T17:56:25.448 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T17:56:25.513 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T17:56:25.598 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T17:56:25.598 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 777 /var/log/ceph 2024-09-13T17:56:25.600 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /var/log/ceph 2024-09-13T17:56:25.625 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T17:56:25.625 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T17:56:25.665 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T17:56:25.690 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T17:56:25.690 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T17:56:25.730 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T17:56:25.760 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-13T17:56:25.760 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-13T17:56:25.760 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-13T17:56:25.760 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:25.760 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-13T17:56:25.808 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-13T17:56:25.926 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-13T17:56:25.951 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:56:25.951 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-13T17:56:25.967 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-13T17:56:26.087 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-13T17:56:26.117 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T17:56:26.117 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Replacing daemon mds.a'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T17:56:26.118 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903942', 'branch': 'wip-rishabh-testing-20240910.100730-reef', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/cap-flush}', 'email': None, 'first_in_suite': False, 'job_id': '7903942', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240910.100730-reef'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Replacing daemon mds.a'], 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240910.100730-reef', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6439, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'sleep_before_teardown': 0, 'subset': '964/1024', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240910.100730-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi111.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP0JsIhiNS3NXAbO+jMiWn0QT19TOhQQ+r/RePFMw4DAJN0UUT8lX21bUFN1pJtgItbRoSfOJTGKpszyUtFlFg0=', 'smithi188.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKYIxogs95ToiLZ0PfQzz59A/nPna3cQsJ0F8Md5EE6RRoh1gCwY7RSudkr8Ze6RBagvtNLwfeAhQyIacRAWxvM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Replacing daemon mds.a'], 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_cap_flush']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-13_14:42:16', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T17:56:26.118 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T17:56:26.121 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T17:56:26.166 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T17:56:26.179 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T17:56:26.231 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:26.232 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T17:56:26.258 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T17:56:26.258 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_1 2024-09-13T17:56:26.313 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T17:56:26.313 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:56:26.314 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-13T17:56:26.314 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:56:26.314 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:56:26.314 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-13 17:56:16.453481052 +0000 2024-09-13T17:56:26.314 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-13 17:54:37.491328041 +0000 2024-09-13T17:56:26.314 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-13 17:54:37.491328041 +0000 2024-09-13T17:56:26.314 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-13 17:54:37.491328041 +0000 2024-09-13T17:56:26.314 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T17:56:26.376 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-13T17:56:26.377 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-13T17:56:26.377 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000231558 s, 2.2 MB/s 2024-09-13T17:56:26.378 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T17:56:26.435 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_2 2024-09-13T17:56:26.489 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T17:56:26.490 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:56:26.490 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-13T17:56:26.490 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:56:26.490 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:56:26.490 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-13 17:56:16.453481052 +0000 2024-09-13T17:56:26.490 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-13 17:54:37.815322861 +0000 2024-09-13T17:56:26.490 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-13 17:54:37.815322861 +0000 2024-09-13T17:56:26.490 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-13 17:54:37.815322861 +0000 2024-09-13T17:56:26.490 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T17:56:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-13T17:56:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-13T17:56:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000196523 s, 2.6 MB/s 2024-09-13T17:56:26.554 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T17:56:26.610 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_3 2024-09-13T17:56:26.665 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T17:56:26.665 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:56:26.666 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-13T17:56:26.666 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:56:26.666 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:56:26.666 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-13 17:56:16.454481035 +0000 2024-09-13T17:56:26.666 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-13 17:54:38.138317699 +0000 2024-09-13T17:56:26.666 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-13 17:54:38.138317699 +0000 2024-09-13T17:56:26.666 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-13 17:54:38.138317699 +0000 2024-09-13T17:56:26.666 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T17:56:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-13T17:56:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-13T17:56:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000219664 s, 2.3 MB/s 2024-09-13T17:56:26.730 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T17:56:26.786 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_4 2024-09-13T17:56:26.842 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T17:56:26.842 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:56:26.842 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-13T17:56:26.842 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:56:26.842 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:56:26.842 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-13 17:56:16.454481035 +0000 2024-09-13T17:56:26.842 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-13 17:54:38.461312537 +0000 2024-09-13T17:56:26.842 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-13 17:54:38.461312537 +0000 2024-09-13T17:56:26.842 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-13 17:54:38.461312537 +0000 2024-09-13T17:56:26.842 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T17:56:26.905 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-13T17:56:26.905 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-13T17:56:26.905 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000164026 s, 3.1 MB/s 2024-09-13T17:56:26.906 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T17:56:26.961 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T17:56:26.962 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:56:26.962 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T17:56:26.980 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T17:56:26.980 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2024-09-13T17:56:27.038 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T17:56:27.038 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:56:27.038 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-13T17:56:27.038 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:56:27.038 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:56:27.039 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-09-13 17:56:16.525467689 +0000 2024-09-13T17:56:27.039 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-09-13 17:54:37.499331126 +0000 2024-09-13T17:56:27.039 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-09-13 17:54:37.499331126 +0000 2024-09-13T17:56:27.039 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-09-13 17:54:37.499331126 +0000 2024-09-13T17:56:27.039 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T17:56:27.104 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-09-13T17:56:27.104 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-09-13T17:56:27.104 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000216164 s, 2.4 MB/s 2024-09-13T17:56:27.105 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T17:56:27.164 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2024-09-13T17:56:27.220 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T17:56:27.220 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:56:27.220 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-13T17:56:27.220 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:56:27.220 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:56:27.220 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-09-13 17:56:16.525467689 +0000 2024-09-13T17:56:27.220 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-09-13 17:54:37.864324300 +0000 2024-09-13T17:56:27.220 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-09-13 17:54:37.864324300 +0000 2024-09-13T17:56:27.220 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-09-13 17:54:37.864324300 +0000 2024-09-13T17:56:27.221 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T17:56:27.288 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-09-13T17:56:27.288 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-09-13T17:56:27.288 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000201163 s, 2.5 MB/s 2024-09-13T17:56:27.288 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T17:56:27.345 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2024-09-13T17:56:27.401 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T17:56:27.401 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:56:27.401 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-13T17:56:27.402 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:56:27.402 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:56:27.402 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-09-13 17:56:16.525467689 +0000 2024-09-13T17:56:27.402 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-09-13 17:54:38.181318371 +0000 2024-09-13T17:56:27.402 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-09-13 17:54:38.181318371 +0000 2024-09-13T17:56:27.402 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-09-13 17:54:38.181318371 +0000 2024-09-13T17:56:27.402 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T17:56:27.465 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-09-13T17:56:27.465 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-09-13T17:56:27.465 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000215953 s, 2.4 MB/s 2024-09-13T17:56:27.466 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T17:56:27.522 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2024-09-13T17:56:27.578 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T17:56:27.578 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:56:27.578 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-13T17:56:27.578 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:56:27.578 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:56:27.578 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-09-13 17:56:16.526467671 +0000 2024-09-13T17:56:27.578 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-09-13 17:54:38.502312367 +0000 2024-09-13T17:56:27.578 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-09-13 17:54:38.502312367 +0000 2024-09-13T17:56:27.579 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-09-13 17:54:38.502312367 +0000 2024-09-13T17:56:27.579 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T17:56:27.642 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-09-13T17:56:27.642 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-09-13T17:56:27.642 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.00018653 s, 2.7 MB/s 2024-09-13T17:56:27.643 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T17:56:27.699 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T17:56:27.700 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi111.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T17:56:27.700 INFO:tasks.ceph:Generating config... 2024-09-13T17:56:27.701 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-13T17:56:27.701 INFO:tasks.ceph:[client] debug client = 20 2024-09-13T17:56:27.701 INFO:tasks.ceph:[client] debug ms = 1 2024-09-13T17:56:27.702 INFO:tasks.ceph:[client] pid file = 2024-09-13T17:56:27.702 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-13T17:56:27.702 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-13T17:56:27.702 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-13T17:56:27.702 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-13T17:56:27.702 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-13T17:56:27.702 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-13T17:56:27.702 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-13T17:56:27.702 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-13T17:56:27.702 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-13T17:56:27.702 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-13T17:56:27.702 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-13T17:56:27.702 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-13T17:56:27.703 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-13T17:56:27.703 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T17:56:27.703 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T17:56:27.703 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T17:56:27.703 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T17:56:27.703 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T17:56:27.703 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-13T17:56:27.703 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-13T17:56:27.703 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T17:56:27.703 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T17:56:27.703 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-13T17:56:27.703 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-13T17:56:27.703 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T17:56:27.703 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T17:56:27.704 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-13T17:56:27.704 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T17:56:27.704 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T17:56:27.704 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T17:56:27.704 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T17:56:27.704 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T17:56:27.704 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-13T17:56:27.704 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T17:56:27.705 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T17:56:27.760 INFO:teuthology.orchestra.run.smithi111.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T17:56:27.763 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T17:56:27.807 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T17:56:27.873 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.111'), ('mon.b', '172.21.15.188'), ('mon.c', '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]')] 2024-09-13T17:56:27.874 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.111,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-13T17:56:27.875 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:27.875 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T17:56:27.929 DEBUG:teuthology.orchestra.run.smithi111:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.111 --add b 172.21.15.188 --addv c '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:56:28.015 INFO:teuthology.orchestra.run.smithi111.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:56:28.015 INFO:teuthology.orchestra.run.smithi111.stdout:monmaptool: generated fsid 2a8c13ef-d427-4b48-90b3-65459ec79a47 2024-09-13T17:56:28.015 INFO:teuthology.orchestra.run.smithi111.stdout:setting min_mon_release = pacific 2024-09-13T17:56:28.015 INFO:teuthology.orchestra.run.smithi111.stdout:epoch 0 2024-09-13T17:56:28.015 INFO:teuthology.orchestra.run.smithi111.stdout:fsid 2a8c13ef-d427-4b48-90b3-65459ec79a47 2024-09-13T17:56:28.015 INFO:teuthology.orchestra.run.smithi111.stdout:last_changed 2024-09-13T17:56:28.014181+0000 2024-09-13T17:56:28.015 INFO:teuthology.orchestra.run.smithi111.stdout:created 2024-09-13T17:56:28.014181+0000 2024-09-13T17:56:28.015 INFO:teuthology.orchestra.run.smithi111.stdout:min_mon_release 16 (pacific) 2024-09-13T17:56:28.015 INFO:teuthology.orchestra.run.smithi111.stdout:election_strategy: 1 2024-09-13T17:56:28.016 INFO:teuthology.orchestra.run.smithi111.stdout:0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-09-13T17:56:28.016 INFO:teuthology.orchestra.run.smithi111.stdout:1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.b 2024-09-13T17:56:28.016 INFO:teuthology.orchestra.run.smithi111.stdout:2: [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] mon.c 2024-09-13T17:56:28.016 INFO:teuthology.orchestra.run.smithi111.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T17:56:28.017 DEBUG:teuthology.orchestra.run.smithi111:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T17:56:28.032 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2a8c13ef-d427-4b48-90b3-65459ec79a47... 2024-09-13T17:56:28.033 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T17:56:28.075 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T17:56:28.114 INFO:teuthology.orchestra.run.smithi111.stdout:[global] 2024-09-13T17:56:28.115 INFO:teuthology.orchestra.run.smithi111.stdout: chdir = "" 2024-09-13T17:56:28.115 INFO:teuthology.orchestra.run.smithi111.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T17:56:28.115 INFO:teuthology.orchestra.run.smithi111.stdout: auth supported = cephx 2024-09-13T17:56:28.115 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.115 INFO:teuthology.orchestra.run.smithi111.stdout: filestore xattr use omap = true 2024-09-13T17:56:28.115 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.115 INFO:teuthology.orchestra.run.smithi111.stdout: mon clock drift allowed = 1.000 2024-09-13T17:56:28.115 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.115 INFO:teuthology.orchestra.run.smithi111.stdout: osd crush chooseleaf type = 0 2024-09-13T17:56:28.115 INFO:teuthology.orchestra.run.smithi111.stdout: auth debug = true 2024-09-13T17:56:28.115 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.116 INFO:teuthology.orchestra.run.smithi111.stdout: ms die on old message = true 2024-09-13T17:56:28.116 INFO:teuthology.orchestra.run.smithi111.stdout: ms die on bug = true 2024-09-13T17:56:28.116 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.116 INFO:teuthology.orchestra.run.smithi111.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T17:56:28.116 INFO:teuthology.orchestra.run.smithi111.stdout: mon pg warn max object skew = 0 2024-09-13T17:56:28.116 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.116 INFO:teuthology.orchestra.run.smithi111.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T17:56:28.116 INFO:teuthology.orchestra.run.smithi111.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T17:56:28.116 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.116 INFO:teuthology.orchestra.run.smithi111.stdout: osd pool default size = 2 2024-09-13T17:56:28.116 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.116 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd allow primary affinity = true 2024-09-13T17:56:28.116 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd allow pg remap = true 2024-09-13T17:56:28.117 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on legacy crush tunables = false 2024-09-13T17:56:28.117 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on crush straw calc version zero = false 2024-09-13T17:56:28.117 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on no sortbitwise = false 2024-09-13T17:56:28.117 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on osd down out interval zero = false 2024-09-13T17:56:28.117 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on too few osds = false 2024-09-13T17:56:28.117 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T17:56:28.117 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T17:56:28.117 INFO:teuthology.orchestra.run.smithi111.stdout: mon_allow_pool_size_one = true 2024-09-13T17:56:28.117 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.117 INFO:teuthology.orchestra.run.smithi111.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T17:56:28.117 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.117 INFO:teuthology.orchestra.run.smithi111.stdout: osd default data pool replay window = 5 2024-09-13T17:56:28.117 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.118 INFO:teuthology.orchestra.run.smithi111.stdout: mon allow pool delete = true 2024-09-13T17:56:28.118 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.118 INFO:teuthology.orchestra.run.smithi111.stdout: mon cluster log file level = debug 2024-09-13T17:56:28.118 INFO:teuthology.orchestra.run.smithi111.stdout: debug asserts on shutdown = true 2024-09-13T17:56:28.118 INFO:teuthology.orchestra.run.smithi111.stdout: mon health detail to clog = false 2024-09-13T17:56:28.118 INFO:teuthology.orchestra.run.smithi111.stdout: mon host = "172.21.15.111,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-09-13T17:56:28.118 INFO:teuthology.orchestra.run.smithi111.stdout: fsid = 2a8c13ef-d427-4b48-90b3-65459ec79a47 2024-09-13T17:56:28.118 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.118 INFO:teuthology.orchestra.run.smithi111.stdout:[osd] 2024-09-13T17:56:28.118 INFO:teuthology.orchestra.run.smithi111.stdout: osd journal size = 100 2024-09-13T17:56:28.118 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.118 INFO:teuthology.orchestra.run.smithi111.stdout: osd scrub load threshold = 5.0 2024-09-13T17:56:28.118 INFO:teuthology.orchestra.run.smithi111.stdout: osd scrub max interval = 600 2024-09-13T17:56:28.118 INFO:teuthology.orchestra.run.smithi111.stdout: osd mclock profile = high_recovery_ops 2024-09-13T17:56:28.118 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.119 INFO:teuthology.orchestra.run.smithi111.stdout: osd recover clone overlap = true 2024-09-13T17:56:28.119 INFO:teuthology.orchestra.run.smithi111.stdout: osd recovery max chunk = 1048576 2024-09-13T17:56:28.119 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.119 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug shutdown = true 2024-09-13T17:56:28.119 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug op order = true 2024-09-13T17:56:28.119 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug verify stray on activate = true 2024-09-13T17:56:28.119 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.119 INFO:teuthology.orchestra.run.smithi111.stdout: osd open classes on start = true 2024-09-13T17:56:28.119 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug pg log writeout = true 2024-09-13T17:56:28.119 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.119 INFO:teuthology.orchestra.run.smithi111.stdout: osd deep scrub update digest min age = 30 2024-09-13T17:56:28.119 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.119 INFO:teuthology.orchestra.run.smithi111.stdout: osd map max advance = 10 2024-09-13T17:56:28.119 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.119 INFO:teuthology.orchestra.run.smithi111.stdout: journal zero on create = true 2024-09-13T17:56:28.120 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.120 INFO:teuthology.orchestra.run.smithi111.stdout: filestore ondisk finisher threads = 3 2024-09-13T17:56:28.120 INFO:teuthology.orchestra.run.smithi111.stdout: filestore apply finisher threads = 3 2024-09-13T17:56:28.120 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.120 INFO:teuthology.orchestra.run.smithi111.stdout: bdev debug aio = true 2024-09-13T17:56:28.120 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug misdirected ops = true 2024-09-13T17:56:28.120 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore block size = 96636764160 2024-09-13T17:56:28.120 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore fsck on mount = True 2024-09-13T17:56:28.120 INFO:teuthology.orchestra.run.smithi111.stdout: debug bluefs = 20 2024-09-13T17:56:28.120 INFO:teuthology.orchestra.run.smithi111.stdout: debug bluestore = 20 2024-09-13T17:56:28.120 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-09-13T17:56:28.120 INFO:teuthology.orchestra.run.smithi111.stdout: debug osd = 20 2024-09-13T17:56:28.120 INFO:teuthology.orchestra.run.smithi111.stdout: debug rocksdb = 10 2024-09-13T17:56:28.121 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T17:56:28.121 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd full ratio = 0.9 2024-09-13T17:56:28.121 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd nearfull ratio = 0.8 2024-09-13T17:56:28.121 INFO:teuthology.orchestra.run.smithi111.stdout: osd failsafe full ratio = 0.95 2024-09-13T17:56:28.121 INFO:teuthology.orchestra.run.smithi111.stdout: osd objectstore = bluestore 2024-09-13T17:56:28.121 INFO:teuthology.orchestra.run.smithi111.stdout: osd op complaint time = 180 2024-09-13T17:56:28.121 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.121 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr] 2024-09-13T17:56:28.121 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-09-13T17:56:28.121 INFO:teuthology.orchestra.run.smithi111.stdout: debug mgr = 20 2024-09-13T17:56:28.121 INFO:teuthology.orchestra.run.smithi111.stdout: debug mon = 20 2024-09-13T17:56:28.121 INFO:teuthology.orchestra.run.smithi111.stdout: debug auth = 20 2024-09-13T17:56:28.121 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min pgs per osd = 4 2024-09-13T17:56:28.121 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min bytes per osd = 10 2024-09-13T17:56:28.122 INFO:teuthology.orchestra.run.smithi111.stdout: mgr/telemetry/nag = false 2024-09-13T17:56:28.122 INFO:teuthology.orchestra.run.smithi111.stdout: debug client = 20 2024-09-13T17:56:28.122 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.122 INFO:teuthology.orchestra.run.smithi111.stdout:[mon] 2024-09-13T17:56:28.122 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-09-13T17:56:28.122 INFO:teuthology.orchestra.run.smithi111.stdout: debug mon = 20 2024-09-13T17:56:28.122 INFO:teuthology.orchestra.run.smithi111.stdout: debug paxos = 20 2024-09-13T17:56:28.122 INFO:teuthology.orchestra.run.smithi111.stdout: debug auth = 20 2024-09-13T17:56:28.122 INFO:teuthology.orchestra.run.smithi111.stdout: mon data avail warn = 5 2024-09-13T17:56:28.122 INFO:teuthology.orchestra.run.smithi111.stdout: mon mgr mkfs grace = 240 2024-09-13T17:56:28.122 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min pgs per osd = 4 2024-09-13T17:56:28.122 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd reporter subtree level = osd 2024-09-13T17:56:28.122 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd prime pg temp = true 2024-09-13T17:56:28.122 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min bytes per osd = 10 2024-09-13T17:56:28.123 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.123 INFO:teuthology.orchestra.run.smithi111.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T17:56:28.123 INFO:teuthology.orchestra.run.smithi111.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T17:56:28.123 INFO:teuthology.orchestra.run.smithi111.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T17:56:28.123 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.123 INFO:teuthology.orchestra.run.smithi111.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T17:56:28.123 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T17:56:28.123 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T17:56:28.123 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.123 INFO:teuthology.orchestra.run.smithi111.stdout: # 1m isn't quite enough 2024-09-13T17:56:28.123 INFO:teuthology.orchestra.run.smithi111.stdout: mon_down_mkfs_grace = 2m 2024-09-13T17:56:28.123 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.123 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_filestore_osds = false 2024-09-13T17:56:28.123 INFO:teuthology.orchestra.run.smithi111.stdout: mon op complaint time = 120 2024-09-13T17:56:28.124 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on pool no app = False 2024-09-13T17:56:28.124 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:56:28.124 INFO:teuthology.orchestra.run.smithi111.stdout:[client] 2024-09-13T17:56:28.124 INFO:teuthology.orchestra.run.smithi111.stdout: rgw cache enabled = true 2024-09-13T17:56:28.124 INFO:teuthology.orchestra.run.smithi111.stdout: rgw enable ops log = true 2024-09-13T17:56:28.124 INFO:teuthology.orchestra.run.smithi111.stdout: rgw enable usage log = true 2024-09-13T17:56:28.124 INFO:teuthology.orchestra.run.smithi111.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T17:56:28.124 INFO:teuthology.orchestra.run.smithi111.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T17:56:28.124 INFO:teuthology.orchestra.run.smithi111.stdout: client mount timeout = 600 2024-09-13T17:56:28.124 INFO:teuthology.orchestra.run.smithi111.stdout: debug client = 20 2024-09-13T17:56:28.124 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-09-13T17:56:28.124 INFO:teuthology.orchestra.run.smithi111.stdout: pid file = "" 2024-09-13T17:56:28.124 INFO:teuthology.orchestra.run.smithi111.stdout: rados mon op timeout = 900 2024-09-13T17:56:28.124 INFO:teuthology.orchestra.run.smithi111.stdout: rados osd op timeout = 900 2024-09-13T17:56:28.125 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.a] 2024-09-13T17:56:28.125 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.b] 2024-09-13T17:56:28.125 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.c] 2024-09-13T17:56:28.125 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.a] 2024-09-13T17:56:28.125 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.c] 2024-09-13T17:56:28.125 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.b] 2024-09-13T17:56:28.125 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.d] 2024-09-13T17:56:28.125 INFO:teuthology.orchestra.run.smithi111.stdout:[mds] 2024-09-13T17:56:28.125 INFO:teuthology.orchestra.run.smithi111.stdout: debug mds = 20 2024-09-13T17:56:28.125 INFO:teuthology.orchestra.run.smithi111.stdout: debug mds balancer = 20 2024-09-13T17:56:28.125 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-09-13T17:56:28.125 INFO:teuthology.orchestra.run.smithi111.stdout: mds debug frag = True 2024-09-13T17:56:28.125 INFO:teuthology.orchestra.run.smithi111.stdout: mds debug scatterstat = True 2024-09-13T17:56:28.125 INFO:teuthology.orchestra.run.smithi111.stdout: mds op complaint time = 180 2024-09-13T17:56:28.125 INFO:teuthology.orchestra.run.smithi111.stdout: mds verify scatter = True 2024-09-13T17:56:28.126 INFO:teuthology.orchestra.run.smithi111.stdout: osd op complaint time = 180 2024-09-13T17:56:28.126 INFO:teuthology.orchestra.run.smithi111.stdout: rados mon op timeout = 900 2024-09-13T17:56:28.126 INFO:teuthology.orchestra.run.smithi111.stdout: rados osd op timeout = 900 2024-09-13T17:56:28.126 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-09-13T17:56:28.126 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-09-13T17:56:28.126 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T17:56:28.127 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-09-13T17:56:28.127 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.127 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-09-13T17:56:28.127 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.127 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-09-13T17:56:28.127 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.127 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-09-13T17:56:28.127 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-09-13T17:56:28.127 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.127 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-09-13T17:56:28.127 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-09-13T17:56:28.127 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.127 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T17:56:28.127 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-09-13T17:56:28.127 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.128 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T17:56:28.128 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T17:56:28.128 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.128 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-09-13T17:56:28.128 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.128 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-09-13T17:56:28.128 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-09-13T17:56:28.128 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-09-13T17:56:28.128 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-09-13T17:56:28.128 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-09-13T17:56:28.128 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-09-13T17:56:28.128 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-09-13T17:56:28.128 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T17:56:28.129 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T17:56:28.129 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-09-13T17:56:28.129 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.129 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T17:56:28.129 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.129 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-09-13T17:56:28.129 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.129 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-09-13T17:56:28.129 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.129 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-09-13T17:56:28.129 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-09-13T17:56:28.129 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-09-13T17:56:28.129 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.111,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-09-13T17:56:28.129 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = 2a8c13ef-d427-4b48-90b3-65459ec79a47 2024-09-13T17:56:28.129 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.130 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-09-13T17:56:28.130 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-09-13T17:56:28.130 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.130 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-09-13T17:56:28.130 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-09-13T17:56:28.130 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-09-13T17:56:28.130 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.130 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-09-13T17:56:28.130 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-09-13T17:56:28.130 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.130 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-09-13T17:56:28.130 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-09-13T17:56:28.130 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-09-13T17:56:28.130 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.131 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-09-13T17:56:28.131 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-09-13T17:56:28.131 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.131 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-09-13T17:56:28.131 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.131 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-09-13T17:56:28.131 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.131 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-09-13T17:56:28.131 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.131 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-09-13T17:56:28.131 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-09-13T17:56:28.131 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.131 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-09-13T17:56:28.131 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-09-13T17:56:28.131 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2024-09-13T17:56:28.132 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2024-09-13T17:56:28.132 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 20 2024-09-13T17:56:28.132 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 20 2024-09-13T17:56:28.132 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-09-13T17:56:28.132 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-09-13T17:56:28.132 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 10 2024-09-13T17:56:28.132 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T17:56:28.132 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2024-09-13T17:56:28.132 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2024-09-13T17:56:28.132 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2024-09-13T17:56:28.132 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2024-09-13T17:56:28.132 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-09-13T17:56:28.132 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.132 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-09-13T17:56:28.133 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-09-13T17:56:28.133 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-09-13T17:56:28.133 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-09-13T17:56:28.133 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-09-13T17:56:28.133 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-09-13T17:56:28.133 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-09-13T17:56:28.133 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-09-13T17:56:28.133 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-09-13T17:56:28.133 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.133 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-09-13T17:56:28.133 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-09-13T17:56:28.133 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-09-13T17:56:28.133 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-09-13T17:56:28.133 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-09-13T17:56:28.134 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-09-13T17:56:28.134 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-09-13T17:56:28.134 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-09-13T17:56:28.134 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-09-13T17:56:28.134 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-09-13T17:56:28.134 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-09-13T17:56:28.134 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.134 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T17:56:28.134 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T17:56:28.134 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T17:56:28.134 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.134 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T17:56:28.134 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T17:56:28.134 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T17:56:28.134 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.135 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-09-13T17:56:28.135 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-09-13T17:56:28.135 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.135 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-09-13T17:56:28.135 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2024-09-13T17:56:28.135 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on pool no app = False 2024-09-13T17:56:28.135 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-09-13T17:56:28.135 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-09-13T17:56:28.135 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-09-13T17:56:28.135 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-09-13T17:56:28.135 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-09-13T17:56:28.135 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T17:56:28.135 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T17:56:28.135 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2024-09-13T17:56:28.135 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-09-13T17:56:28.136 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-09-13T17:56:28.136 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2024-09-13T17:56:28.136 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-09-13T17:56:28.136 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-09-13T17:56:28.136 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-09-13T17:56:28.136 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2024-09-13T17:56:28.136 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2024-09-13T17:56:28.136 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-09-13T17:56:28.136 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-09-13T17:56:28.136 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-09-13T17:56:28.136 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-09-13T17:56:28.136 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2024-09-13T17:56:28.136 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2024-09-13T17:56:28.136 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2024-09-13T17:56:28.137 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-09-13T17:56:28.137 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2024-09-13T17:56:28.137 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = True 2024-09-13T17:56:28.137 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2024-09-13T17:56:28.137 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = True 2024-09-13T17:56:28.137 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-09-13T17:56:28.137 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-09-13T17:56:28.137 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-09-13T17:56:28.138 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T17:56:28.138 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T17:56:28.232 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T17:56:28.232 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:28.232 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T17:56:28.248 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:28.248 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T17:56:28.302 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T17:56:28.303 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:28.303 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T17:56:28.303 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T17:56:28.377 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:28.377 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:56:28.433 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi188.front.sepia.ceph.com 2024-09-13T17:56:28.433 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:56:28.433 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T17:56:28.433 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T17:56:28.467 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:56:28.467 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:56:28.523 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T17:56:28.523 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T17:56:28.523 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T17:56:28.574 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T17:56:28.577 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T17:56:28.652 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T17:56:28.654 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T17:56:28.655 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T17:56:28.705 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T17:56:28.708 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-13T17:56:28.731 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-13T17:56:28.822 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-13T17:56:28.824 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-13T17:56:28.849 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-13T17:56:28.899 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-13T17:56:28.902 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-13T17:56:28.927 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-13T17:56:29.017 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-13T17:56:29.020 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-13T17:56:29.044 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T17:56:29.044 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T17:56:29.085 INFO:teuthology.orchestra.run.smithi111.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T17:56:29.098 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-13T17:56:29.179 INFO:teuthology.orchestra.run.smithi111.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-13T17:56:29.192 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-13T17:56:29.233 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-13T17:56:29.245 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-13T17:56:29.326 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-13T17:56:29.338 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T17:56:29.339 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi111.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T17:56:29.339 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T17:56:29.363 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T17:56:29.363 INFO:tasks.ceph:role: osd.0 2024-09-13T17:56:29.363 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T17:56:29.364 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T17:56:29.436 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:56:29.436 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:56:29.436 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:56:29.436 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T17:56:29.436 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:56:29.436 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:56:29.436 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:56:29.436 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T17:56:29.436 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:56:29.437 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:56:29.926 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2024-09-13T17:56:29.928 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2024-09-13T17:56:29.928 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T17:56:30.056 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T17:56:30.088 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T17:56:30.153 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T17:56:30.153 INFO:tasks.ceph:role: osd.1 2024-09-13T17:56:30.153 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T17:56:30.154 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T17:56:30.225 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:56:30.225 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:56:30.225 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:56:30.225 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T17:56:30.225 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:56:30.225 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:56:30.225 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:56:30.225 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T17:56:30.226 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:56:30.226 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:56:30.686 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2024-09-13T17:56:30.688 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2024-09-13T17:56:30.688 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T17:56:30.781 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T17:56:30.811 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T17:56:30.876 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T17:56:30.876 INFO:tasks.ceph:role: osd.2 2024-09-13T17:56:30.876 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T17:56:30.877 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T17:56:30.948 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:56:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:56:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:56:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T17:56:30.949 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:56:30.950 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:56:30.950 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:56:30.950 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T17:56:30.950 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:56:30.950 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:56:31.439 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2024-09-13T17:56:31.441 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2024-09-13T17:56:31.441 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T17:56:31.582 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T17:56:31.615 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T17:56:31.682 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T17:56:31.682 INFO:tasks.ceph:role: osd.3 2024-09-13T17:56:31.682 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T17:56:31.682 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T17:56:31.755 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:56:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:56:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:56:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T17:56:31.756 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:56:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:56:31.756 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:56:31.756 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T17:56:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:56:31.756 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:56:32.270 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2024-09-13T17:56:32.272 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2024-09-13T17:56:32.272 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-13T17:56:32.426 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T17:56:32.459 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:56:32.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:32.557+0000 7f0a9ed56640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T17:56:32.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:32.557+0000 7f0a9ed56640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T17:56:32.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:32.557+0000 7f0a9ed56640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T17:56:32.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:32.557+0000 7f0a9ed56640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T17:56:32.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:32.557+0000 7f0a9ed56640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T17:56:32.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:32.558+0000 7f0a9ed56640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T17:56:35.410 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T17:56:35.438 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:56:35.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:35.523+0000 7fb525cb2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T17:56:35.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:35.523+0000 7fb525cb2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T17:56:35.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:35.523+0000 7fb525cb2640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T17:56:35.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:35.523+0000 7fb525cb2640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T17:56:35.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:35.523+0000 7fb525cb2640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T17:56:35.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:35.524+0000 7fb525cb2640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T17:56:38.376 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T17:56:38.406 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:56:38.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:38.496+0000 7f6f185a2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T17:56:38.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:38.497+0000 7f6f185a2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T17:56:38.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:38.497+0000 7f6f185a2640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T17:56:38.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:38.497+0000 7f6f185a2640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T17:56:38.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:38.497+0000 7f6f185a2640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T17:56:38.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:38.497+0000 7f6f185a2640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T17:56:41.380 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T17:56:41.410 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:56:41.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:41.500+0000 7fe0e6bc5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T17:56:41.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:41.500+0000 7fe0e6bc5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T17:56:41.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:41.500+0000 7fe0e6bc5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T17:56:41.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:41.500+0000 7fe0e6bc5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T17:56:41.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:41.500+0000 7fe0e6bc5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T17:56:41.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:56:41.501+0000 7fe0e6bc5640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T17:56:44.368 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T17:56:44.397 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-13T17:56:44.428 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T17:56:44.428 INFO:tasks.ceph:role: osd.4 2024-09-13T17:56:44.428 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com 2024-09-13T17:56:44.428 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T17:56:44.503 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:56:44.503 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:56:44.504 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:56:44.504 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T17:56:44.504 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:56:44.504 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:56:44.504 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:56:44.504 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T17:56:44.504 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:56:44.504 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:56:45.013 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-09-13T17:56:45.016 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-09-13T17:56:45.016 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-13T17:56:45.161 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-13T17:56:45.193 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-13T17:56:45.258 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T17:56:45.258 INFO:tasks.ceph:role: osd.5 2024-09-13T17:56:45.258 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com 2024-09-13T17:56:45.258 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T17:56:45.330 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:56:45.330 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:56:45.330 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:56:45.330 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T17:56:45.330 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:56:45.330 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:56:45.330 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:56:45.330 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T17:56:45.330 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:56:45.330 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:56:45.772 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-09-13T17:56:45.774 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-09-13T17:56:45.774 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-13T17:56:45.865 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-13T17:56:45.894 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-13T17:56:45.959 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T17:56:45.959 INFO:tasks.ceph:role: osd.6 2024-09-13T17:56:45.959 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com 2024-09-13T17:56:45.959 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T17:56:46.031 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:56:46.031 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:56:46.031 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:56:46.031 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T17:56:46.031 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:56:46.031 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:56:46.031 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:56:46.031 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T17:56:46.031 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:56:46.031 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:56:46.522 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-09-13T17:56:46.524 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-09-13T17:56:46.524 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-13T17:56:46.655 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-13T17:56:46.686 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-13T17:56:46.751 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T17:56:46.752 INFO:tasks.ceph:role: osd.7 2024-09-13T17:56:46.752 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com 2024-09-13T17:56:46.752 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T17:56:46.824 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:56:46.824 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:56:46.824 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:56:46.824 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T17:56:46.824 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:56:46.824 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:56:46.824 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:56:46.824 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T17:56:46.824 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:56:46.825 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:56:47.319 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-09-13T17:56:47.321 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-09-13T17:56:47.322 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-13T17:56:47.440 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-13T17:56:47.474 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:56:47.580 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:47.578+0000 7fcd9fc28640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-13T17:56:47.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:47.579+0000 7fcd9fc28640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-13T17:56:47.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:47.579+0000 7fcd9fc28640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T17:56:47.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:47.579+0000 7fcd9fc28640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T17:56:47.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:47.579+0000 7fcd9fc28640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T17:56:47.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:47.580+0000 7fcd9fc28640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-13T17:56:50.435 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-13T17:56:50.464 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:56:50.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:50.550+0000 7f8e51033640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-13T17:56:50.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:50.550+0000 7f8e51033640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-13T17:56:50.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:50.550+0000 7f8e51033640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T17:56:50.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:50.550+0000 7f8e51033640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T17:56:50.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:50.550+0000 7f8e51033640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T17:56:50.555 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:50.551+0000 7f8e51033640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-13T17:56:53.400 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-13T17:56:53.427 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:56:53.515 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:53.512+0000 7f0f13014640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-13T17:56:53.515 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:53.512+0000 7f0f13014640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-13T17:56:53.515 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:53.512+0000 7f0f13014640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T17:56:53.515 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:53.512+0000 7f0f13014640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T17:56:53.515 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:53.512+0000 7f0f13014640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T17:56:53.516 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:53.513+0000 7f0f13014640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-13T17:56:56.120 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:56:56.123 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:56:56.365 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-13T17:56:56.391 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:56:56.478 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:56.475+0000 7fc578213640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-13T17:56:56.478 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:56.475+0000 7fc578213640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-13T17:56:56.478 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:56.475+0000 7fc578213640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T17:56:56.478 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:56.475+0000 7fc578213640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T17:56:56.478 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:56.475+0000 7fc578213640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T17:56:56.479 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:56:56.476+0000 7fc578213640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-13T17:56:59.327 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-13T17:56:59.357 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T17:56:59.357 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:59.357 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-13T17:56:59.387 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:59.387 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-13T17:56:59.456 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:59.456 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-13T17:56:59.524 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:59.524 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T17:56:59.591 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:59.591 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T17:56:59.657 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:59.657 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T17:56:59.723 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:56:59.723 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T17:56:59.789 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:56:59.789 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T17:56:59.819 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:56:59.819 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-13T17:56:59.888 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:56:59.888 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-13T17:56:59.955 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:56:59.956 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-13T17:57:00.022 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:57:00.022 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-13T17:57:00.090 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:57:00.090 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-13T17:57:00.156 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:57:00.156 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-13T17:57:00.222 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:57:00.222 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T17:57:00.239 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:57:00.239 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-13T17:57:00.295 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:57:00.295 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-13T17:57:00.311 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:57:00.311 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-13T17:57:00.367 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T17:57:00.367 DEBUG:teuthology.orchestra.run.smithi111:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T17:57:00.369 DEBUG:teuthology.orchestra.run.smithi188:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T17:57:00.410 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr.y] 2024-09-13T17:57:00.411 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDMfORmMfQwIhAAnGHtvQPIerUgf+C7YMXBrA== 2024-09-13T17:57:00.411 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.a] 2024-09-13T17:57:00.411 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDMfORmGhsAKhAACDZXT4HDQrOTTulTPDDwBQ== 2024-09-13T17:57:00.411 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.c] 2024-09-13T17:57:00.411 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDMfORmab3yMBAAVENS+Dd1iEHaLhSJs0+dgQ== 2024-09-13T17:57:00.411 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.0] 2024-09-13T17:57:00.411 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDQfORm2zZEIRAAZVyIO+oraTl7fReOjRPFIg== 2024-09-13T17:57:00.411 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.1] 2024-09-13T17:57:00.411 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDTfORm2Cw+HxAAEblep5UnrnIjGE0ne/uhbA== 2024-09-13T17:57:00.411 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.2] 2024-09-13T17:57:00.411 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDWfORm60erHRAAznKL8gIlxUuPRJs71ibQgQ== 2024-09-13T17:57:00.411 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.3] 2024-09-13T17:57:00.412 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDZfORmXg3iHRAA1CDnJjVEw3IHgQbwO4X9kw== 2024-09-13T17:57:00.412 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr.x] 2024-09-13T17:57:00.412 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDMfORmHLHQJhAAVOix4lyIfaxaQb/3LUO5Kw== 2024-09-13T17:57:00.412 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.b] 2024-09-13T17:57:00.412 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDMfORmlVqTNRAA/v5yZiPCYiORGmfwU5DqmQ== 2024-09-13T17:57:00.412 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.d] 2024-09-13T17:57:00.412 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDNfORmP/oEARAABOBxvI5yMvBLW27cxx/dTg== 2024-09-13T17:57:00.412 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.4] 2024-09-13T17:57:00.412 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDffORm7eaQIhAAyZXNz0WPphUV7UgUgXEQVA== 2024-09-13T17:57:00.412 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.5] 2024-09-13T17:57:00.412 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDifORmT4/lIBAAtD/epRe+TthTciIrVS1QPw== 2024-09-13T17:57:00.412 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.6] 2024-09-13T17:57:00.412 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDlfORmhdihHhAAiskdva4E09iEu5trygpobQ== 2024-09-13T17:57:00.413 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.7] 2024-09-13T17:57:00.413 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDofORm4CluHBAAQRzgHsJ2GEeReXRun+slyg== 2024-09-13T17:57:00.413 INFO:teuthology.orchestra.run.smithi111.stdout:[client.0] 2024-09-13T17:57:00.413 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDNfORmzMgJBRAAEOeFOXxEvbtjqaxzRhfWtw== 2024-09-13T17:57:00.413 INFO:teuthology.orchestra.run.smithi111.stdout:[client.2] 2024-09-13T17:57:00.413 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDNfORmx5GlChAAH81QLf1bQUV2pIXYoRAwJg== 2024-09-13T17:57:00.413 INFO:teuthology.orchestra.run.smithi111.stdout:[client.1] 2024-09-13T17:57:00.413 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDNfORm1XzaDRAAyfCz1uhL9jnJ8WlZGbaspQ== 2024-09-13T17:57:00.413 INFO:teuthology.orchestra.run.smithi111.stdout:[client.3] 2024-09-13T17:57:00.413 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDNfORmm7hpExAAQtZIpkmyDzo/cqH6N+nhZQ== 2024-09-13T17:57:00.430 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.y] 2024-09-13T17:57:00.430 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDMfORmMfQwIhAAnGHtvQPIerUgf+C7YMXBrA== 2024-09-13T17:57:00.431 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-09-13T17:57:00.431 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDMfORmGhsAKhAACDZXT4HDQrOTTulTPDDwBQ== 2024-09-13T17:57:00.431 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-09-13T17:57:00.431 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDMfORmab3yMBAAVENS+Dd1iEHaLhSJs0+dgQ== 2024-09-13T17:57:00.431 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.0] 2024-09-13T17:57:00.431 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDQfORm2zZEIRAAZVyIO+oraTl7fReOjRPFIg== 2024-09-13T17:57:00.431 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.1] 2024-09-13T17:57:00.431 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDTfORm2Cw+HxAAEblep5UnrnIjGE0ne/uhbA== 2024-09-13T17:57:00.431 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.2] 2024-09-13T17:57:00.431 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDWfORm60erHRAAznKL8gIlxUuPRJs71ibQgQ== 2024-09-13T17:57:00.431 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.3] 2024-09-13T17:57:00.431 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDZfORmXg3iHRAA1CDnJjVEw3IHgQbwO4X9kw== 2024-09-13T17:57:00.431 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.x] 2024-09-13T17:57:00.432 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDMfORmHLHQJhAAVOix4lyIfaxaQb/3LUO5Kw== 2024-09-13T17:57:00.432 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-09-13T17:57:00.432 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDMfORmlVqTNRAA/v5yZiPCYiORGmfwU5DqmQ== 2024-09-13T17:57:00.432 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-09-13T17:57:00.432 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDNfORmP/oEARAABOBxvI5yMvBLW27cxx/dTg== 2024-09-13T17:57:00.432 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.4] 2024-09-13T17:57:00.432 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDffORm7eaQIhAAyZXNz0WPphUV7UgUgXEQVA== 2024-09-13T17:57:00.432 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.5] 2024-09-13T17:57:00.432 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDifORmT4/lIBAAtD/epRe+TthTciIrVS1QPw== 2024-09-13T17:57:00.432 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.6] 2024-09-13T17:57:00.432 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDlfORmhdihHhAAiskdva4E09iEu5trygpobQ== 2024-09-13T17:57:00.432 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.7] 2024-09-13T17:57:00.432 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDofORm4CluHBAAQRzgHsJ2GEeReXRun+slyg== 2024-09-13T17:57:00.432 INFO:teuthology.orchestra.run.smithi188.stdout:[client.0] 2024-09-13T17:57:00.433 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDNfORmzMgJBRAAEOeFOXxEvbtjqaxzRhfWtw== 2024-09-13T17:57:00.433 INFO:teuthology.orchestra.run.smithi188.stdout:[client.2] 2024-09-13T17:57:00.433 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDNfORmx5GlChAAH81QLf1bQUV2pIXYoRAwJg== 2024-09-13T17:57:00.433 INFO:teuthology.orchestra.run.smithi188.stdout:[client.1] 2024-09-13T17:57:00.433 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDNfORm1XzaDRAAyfCz1uhL9jnJ8WlZGbaspQ== 2024-09-13T17:57:00.433 INFO:teuthology.orchestra.run.smithi188.stdout:[client.3] 2024-09-13T17:57:00.433 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDNfORmm7hpExAAQtZIpkmyDzo/cqH6N+nhZQ== 2024-09-13T17:57:00.434 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T17:57:00.457 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T17:57:00.524 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:57:00.527 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:57:00.579 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:57:00.582 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:57:00.635 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:57:00.637 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:57:00.687 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:57:00.690 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:57:00.739 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:57:00.742 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:57:00.792 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:57:00.794 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:57:00.844 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T17:57:00.846 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T17:57:00.896 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:57:00.899 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:57:00.951 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:57:00.953 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:57:01.006 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:57:01.008 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:57:01.059 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:57:01.061 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:57:01.111 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:57:01.113 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:57:01.163 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:57:01.166 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:57:01.215 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T17:57:01.218 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T17:57:01.270 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T17:57:01.272 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T17:57:01.325 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T17:57:01.327 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T17:57:01.380 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T17:57:01.383 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T17:57:01.436 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T17:57:01.437 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T17:57:01.461 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T17:57:01.558 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T17:57:01.585 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T17:57:01.611 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T17:57:01.706 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T17:57:01.731 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T17:57:01.796 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T17:57:01.890 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T17:57:01.915 DEBUG:teuthology.orchestra.run.smithi111:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:57:01.917 DEBUG:teuthology.orchestra.run.smithi188:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:57:01.969 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T17:57:01.970 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T17:57:01.970 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T17:57:01.973 INFO:tasks.ceph.mon.a:Started 2024-09-13T17:57:01.973 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T17:57:01.973 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T17:57:02.012 INFO:tasks.ceph.mon.b:Started 2024-09-13T17:57:02.012 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T17:57:02.012 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T17:57:02.014 INFO:tasks.ceph.mon.c:Started 2024-09-13T17:57:02.014 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T17:57:02.014 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T17:57:02.014 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T17:57:02.017 INFO:tasks.ceph.mgr.y:Started 2024-09-13T17:57:02.017 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T17:57:02.017 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T17:57:02.019 INFO:tasks.ceph.mgr.x:Started 2024-09-13T17:57:02.019 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T17:57:02.019 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T17:57:02.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.115+0000 7f0297c20640 1 Processor -- start 2024-09-13T17:57:02.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.115+0000 7f0297c20640 1 -- start start 2024-09-13T17:57:02.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.116+0000 7f0297c20640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f029015a7f0 0x7f02900ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:02.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.116+0000 7f0297c20640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0290099570 0x7f0290164220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:02.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.116+0000 7f0297c20640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02900cc300 con 0x7f02900989d0 2024-09-13T17:57:02.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.116+0000 7f0297c20640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02900b3d20 con 0x7f029015a7f0 2024-09-13T17:57:02.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.116+0000 7f0297c20640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02900b33f0 con 0x7f0290099570 2024-09-13T17:57:02.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.116+0000 7f02959bc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f029015a7f0 0x7f02900ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:02.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.116+0000 7f02959bc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f029015a7f0 0x7f02900ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:32900/0 (socket says 172.21.15.111:32900) 2024-09-13T17:57:02.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.116+0000 7f02959bc640 1 -- 172.21.15.111:0/2078708 learned_addr learned my addr 172.21.15.111:0/2078708 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:02.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.116+0000 7f02959bc640 1 -- 172.21.15.111:0/2078708 >> v2:172.21.15.111:3300/0 conn(0x7f029015a7f0 msgr2=0x7f02900ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:57:02.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.116+0000 7f02961bd640 1 --2- 172.21.15.111:0/2078708 >> v2:172.21.15.188:3300/0 conn(0x7f0290099570 0x7f0290164220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:02.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.116+0000 7f02959bc640 1 -- 172.21.15.111:0/2078708 >> v2:172.21.15.111:3300/0 conn(0x7f029015a7f0 msgr2=0x7f02900ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:57:02.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.116+0000 7f02959bc640 1 --2- 172.21.15.111:0/2078708 >> v2:172.21.15.111:3300/0 conn(0x7f029015a7f0 0x7f02900ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:57:02.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.116+0000 7f02959bc640 1 --2- 172.21.15.111:0/2078708 >> v2:172.21.15.111:3300/0 conn(0x7f029015a7f0 0x7f02900ce3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:02.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.116+0000 7f02961bd640 1 -- 172.21.15.111:0/2078708 >> v2:172.21.15.188:3300/0 conn(0x7f0290099570 msgr2=0x7f0290164220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:57:02.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.116+0000 7f02961bd640 1 -- 172.21.15.111:0/2078708 >> v2:172.21.15.188:3300/0 conn(0x7f0290099570 msgr2=0x7f0290164220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:57:02.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.116+0000 7f02961bd640 1 --2- 172.21.15.111:0/2078708 >> v2:172.21.15.188:3300/0 conn(0x7f0290099570 0x7f0290164220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:57:02.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.116+0000 7f02961bd640 1 --2- 172.21.15.111:0/2078708 >> v2:172.21.15.188:3300/0 conn(0x7f0290099570 0x7f0290164220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:02.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.286+0000 7f02949ba640 1 -- 172.21.15.111:0/2078708 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3208357110 0 0) 0x7f02900cc300 con 0x7f02900989d0 2024-09-13T17:57:02.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.286+0000 7f02949ba640 1 -- 172.21.15.111:0/2078708 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f027c003520 con 0x7f02900989d0 2024-09-13T17:57:02.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.291+0000 7f02949ba640 1 -- 172.21.15.111:0/2078708 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 725113699 0 0) 0x7f027c003520 con 0x7f02900989d0 2024-09-13T17:57:02.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.291+0000 7f02949ba640 1 -- 172.21.15.111:0/2078708 >> v2:172.21.15.188:3300/0 conn(0x7f0290099570 msgr2=0x7f0290164220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:57:02.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.291+0000 7f02949ba640 1 --2- 172.21.15.111:0/2078708 >> v2:172.21.15.188:3300/0 conn(0x7f0290099570 0x7f0290164220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.291+0000 7f02949ba640 1 -- 172.21.15.111:0/2078708 >> v2:172.21.15.111:3300/0 conn(0x7f029015a7f0 msgr2=0x7f02900ce3e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:57:02.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.291+0000 7f02949ba640 1 --2- 172.21.15.111:0/2078708 >> v2:172.21.15.111:3300/0 conn(0x7f029015a7f0 0x7f02900ce3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.291+0000 7f02949ba640 1 -- 172.21.15.111:0/2078708 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02900b36d0 con 0x7f02900989d0 2024-09-13T17:57:02.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.291+0000 7f02949ba640 1 -- 172.21.15.111:0/2078708 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f0280001670 con 0x7f02900989d0 2024-09-13T17:57:02.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.291+0000 7f02949ba640 1 -- 172.21.15.111:0/2078708 >> v1:172.21.15.188:6789/0 conn(0x7f02900989d0 legacy=0x7f0290098e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:02.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.291+0000 7f02949ba640 1 --2- 172.21.15.111:0/2078708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027c003970 0x7f027c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:02.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.291+0000 7f02949ba640 1 --2- 172.21.15.111:0/2078708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f027c0047c0 0x7f027c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:02.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.291+0000 7f02949ba640 1 --2- 172.21.15.111:0/2078708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f027c0051f0 0x7f027c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:02.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.291+0000 7f02949ba640 1 -- 172.21.15.111:0/2078708 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02900b3d20 con 0x7f027c0047c0 2024-09-13T17:57:02.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.291+0000 7f02949ba640 1 -- 172.21.15.111:0/2078708 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02900b33f0 con 0x7f027c003970 2024-09-13T17:57:02.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.291+0000 7f02949ba640 1 -- 172.21.15.111:0/2078708 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027c00c1d0 con 0x7f027c0051f0 2024-09-13T17:57:02.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.291+0000 7f02951bb640 1 --2- 172.21.15.111:0/2078708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f027c0047c0 0x7f027c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:02.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.291+0000 7f02961bd640 1 --2- 172.21.15.111:0/2078708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f027c0051f0 0x7f027c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:02.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.291+0000 7f02959bc640 1 --2- 172.21.15.111:0/2078708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027c003970 0x7f027c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:02.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.292+0000 7f02951bb640 1 -- 172.21.15.111:0/2078708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f027c0051f0 msgr2=0x7f027c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:02.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.292+0000 7f02951bb640 1 --2- 172.21.15.111:0/2078708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f027c0051f0 0x7f027c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.292+0000 7f02951bb640 1 -- 172.21.15.111:0/2078708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027c003970 msgr2=0x7f027c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:02.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.292+0000 7f02951bb640 1 --2- 172.21.15.111:0/2078708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027c003970 0x7f027c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.292+0000 7f02951bb640 1 -- 172.21.15.111:0/2078708 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027c003520 con 0x7f027c0047c0 2024-09-13T17:57:02.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.292+0000 7f02961bd640 1 -- 172.21.15.111:0/2078708 reap_dead start 2024-09-13T17:57:02.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.292+0000 7f02951bb640 1 --2- 172.21.15.111:0/2078708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f027c0047c0 0x7f027c004c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f028000c110 tx=0x7f027c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b99d53c7296596c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:02.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.292+0000 7f02949ba640 1 -- 172.21.15.111:0/2078708 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0280031600 con 0x7f027c0047c0 2024-09-13T17:57:02.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.292+0000 7f02949ba640 1 -- 172.21.15.111:0/2078708 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028002d050 con 0x7f027c0047c0 2024-09-13T17:57:02.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.292+0000 7f02949ba640 1 -- 172.21.15.111:0/2078708 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02800319c0 con 0x7f027c0047c0 2024-09-13T17:57:02.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.293+0000 7f0297c20640 1 -- 172.21.15.111:0/2078708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f027c0047c0 msgr2=0x7f027c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:02.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.293+0000 7f0297c20640 1 --2- 172.21.15.111:0/2078708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f027c0047c0 0x7f027c004c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f028000c110 tx=0x7f027c0057f0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.293+0000 7f0297c20640 1 -- 172.21.15.111:0/2078708 shutdown_connections 2024-09-13T17:57:02.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.293+0000 7f0297c20640 1 --2- 172.21.15.111:0/2078708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f027c0047c0 0x7f027c004c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.293+0000 7f0297c20640 1 -- 172.21.15.111:0/2078708 >> 172.21.15.111:0/2078708 conn(0x7f02900b1f60 msgr2=0x7f02900b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:02.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.293+0000 7f0297c20640 1 -- 172.21.15.111:0/2078708 shutdown_connections 2024-09-13T17:57:02.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.293+0000 7f0297c20640 1 -- 172.21.15.111:0/2078708 wait complete. 2024-09-13T17:57:02.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.295+0000 7f0297c20640 1 Processor -- start 2024-09-13T17:57:02.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.295+0000 7f0297c20640 1 -- start start 2024-09-13T17:57:02.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.295+0000 7f0297c20640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027c0047c0 0x7f0290156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:02.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.295+0000 7f0297c20640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02900ca570 0x7f02900b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:02.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f0297c20640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02900b5d20 0x7f029009a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:02.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f0297c20640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02900b61d0 con 0x7f02900b5d20 2024-09-13T17:57:02.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f0297c20640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0290157040 con 0x7f027c0047c0 2024-09-13T17:57:02.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f0297c20640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02900c3cf0 con 0x7f02900ca570 2024-09-13T17:57:02.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f02961bd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02900b5d20 0x7f029009a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f02959bc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027c0047c0 0x7f0290156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f02959bc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027c0047c0 0x7f0290156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:58694/0 (socket says 172.21.15.111:58694) 2024-09-13T17:57:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f02961bd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02900b5d20 0x7f029009a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54246/0 (socket says 172.21.15.111:54246) 2024-09-13T17:57:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f02959bc640 1 -- 172.21.15.111:0/2779826233 learned_addr learned my addr 172.21.15.111:0/2779826233 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f02951bb640 1 --2- 172.21.15.111:0/2779826233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02900ca570 0x7f02900b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f02959bc640 1 -- 172.21.15.111:0/2779826233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02900ca570 msgr2=0x7f02900b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f02959bc640 1 --2- 172.21.15.111:0/2779826233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02900ca570 0x7f02900b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f02959bc640 1 -- 172.21.15.111:0/2779826233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02900b5d20 msgr2=0x7f029009a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f02959bc640 1 --2- 172.21.15.111:0/2779826233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02900b5d20 0x7f029009a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f02959bc640 1 -- 172.21.15.111:0/2779826233 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02900c5c90 con 0x7f027c0047c0 2024-09-13T17:57:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f02961bd640 1 --2- 172.21.15.111:0/2779826233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02900b5d20 0x7f029009a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f02959bc640 1 --2- 172.21.15.111:0/2779826233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027c0047c0 0x7f0290156b00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f028c0108c0 tx=0x7f028c010d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f02867fc640 1 -- 172.21.15.111:0/2779826233 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c00e9c0 con 0x7f027c0047c0 2024-09-13T17:57:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f02867fc640 1 -- 172.21.15.111:0/2779826233 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028c00eb20 con 0x7f027c0047c0 2024-09-13T17:57:02.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f02867fc640 1 -- 172.21.15.111:0/2779826233 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c00eed0 con 0x7f027c0047c0 2024-09-13T17:57:02.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.296+0000 7f0297c20640 1 -- 172.21.15.111:0/2779826233 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02900c3f40 con 0x7f027c0047c0 2024-09-13T17:57:02.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.297+0000 7f0297c20640 1 -- 172.21.15.111:0/2779826233 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02900c4500 con 0x7f027c0047c0 2024-09-13T17:57:02.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.297+0000 7f02867fc640 1 -- 172.21.15.111:0/2779826233 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f028c015470 con 0x7f027c0047c0 2024-09-13T17:57:02.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.297+0000 7f02867fc640 1 -- 172.21.15.111:0/2779826233 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f028c016970 con 0x7f027c0047c0 2024-09-13T17:57:02.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.297+0000 7f0297c20640 1 -- 172.21.15.111:0/2779826233 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02900c4690 con 0x7f027c0047c0 2024-09-13T17:57:02.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.300+0000 7f02867fc640 1 -- 172.21.15.111:0/2779826233 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f028c01a030 con 0x7f027c0047c0 2024-09-13T17:57:02.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.361+0000 7f0297c20640 1 -- 172.21.15.111:0/2779826233 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f0290098e60 con 0x7f027c0047c0 2024-09-13T17:57:02.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.370+0000 7f02867fc640 1 -- 172.21.15.111:0/2779826233 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f028c00f040 con 0x7f027c0047c0 2024-09-13T17:57:02.372 INFO:teuthology.orchestra.run.smithi111.stderr:adjusted tunables profile to default 2024-09-13T17:57:02.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.370+0000 7f0297c20640 1 -- 172.21.15.111:0/2779826233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027c0047c0 msgr2=0x7f0290156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:02.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.370+0000 7f0297c20640 1 --2- 172.21.15.111:0/2779826233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027c0047c0 0x7f0290156b00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f028c0108c0 tx=0x7f028c010d80 comp rx=0 tx=0).stop 2024-09-13T17:57:02.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.371+0000 7f0297c20640 1 -- 172.21.15.111:0/2779826233 shutdown_connections 2024-09-13T17:57:02.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.371+0000 7f0297c20640 1 --2- 172.21.15.111:0/2779826233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02900b5d20 0x7f029009a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.371+0000 7f0297c20640 1 --2- 172.21.15.111:0/2779826233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02900ca570 0x7f02900b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.371+0000 7f0297c20640 1 --2- 172.21.15.111:0/2779826233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027c0047c0 0x7f0290156b00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.371+0000 7f0297c20640 1 -- 172.21.15.111:0/2779826233 >> 172.21.15.111:0/2779826233 conn(0x7f02900b1f60 msgr2=0x7f02900b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:02.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.371+0000 7f0297c20640 1 -- 172.21.15.111:0/2779826233 shutdown_connections 2024-09-13T17:57:02.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:02.371+0000 7f0297c20640 1 -- 172.21.15.111:0/2779826233 wait complete. 2024-09-13T17:57:02.382 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T17:57:02.383 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T17:57:02.383 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:57:02.383 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T17:57:02.387 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:02.385+0000 7f6a9ba15040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:57:02.393 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:02.391+0000 7f8156532040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:57:02.407 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:57:02.407 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T17:57:02.430 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:02.428+0000 7f6a9ba15040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:57:02.433 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:02.431+0000 7f8156532040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:57:02.454 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:57:02.454 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T17:57:02.518 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:57:02.519 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T17:57:02.582 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:02.580+0000 7f8156532040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:57:02.584 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:57:02.584 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-13T17:57:02.587 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:02.585+0000 7f6a9ba15040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:57:02.609 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:57:02.609 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-13T17:57:02.634 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:02.633+0000 7f8156532040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:57:02.638 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:02.636+0000 7f6a9ba15040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:57:02.658 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:57:02.659 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-13T17:57:02.723 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:57:02.723 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-13T17:57:02.788 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 20368658-282f-4098-bfcc-4c95d0bf0def 0 2024-09-13T17:57:02.799 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:02.797+0000 7f8156532040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:57:02.801 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:02.799+0000 7f6a9ba15040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:57:02.897 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.895+0000 7fb5e608e640 1 Processor -- start 2024-09-13T17:57:02.897 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.895+0000 7fb5e608e640 1 -- start start 2024-09-13T17:57:02.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.896+0000 7fb5e608e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e0069ed0 0x7fb5e00b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:02.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.896+0000 7fb5e608e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb5e0145460 0x7fb5e006d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:02.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.896+0000 7fb5e608e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e00b3c60 con 0x7fb5e006a2b0 2024-09-13T17:57:02.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.896+0000 7fb5e608e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e00ac8f0 con 0x7fb5e0145460 2024-09-13T17:57:02.899 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.896+0000 7fb5e608e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e00b27e0 con 0x7fb5e0069ed0 2024-09-13T17:57:02.899 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.896+0000 7fb5df7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e0069ed0 0x7fb5e00b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:02.899 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.896+0000 7fb5df7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e0069ed0 0x7fb5e00b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:52052/0 (socket says 172.21.15.188:52052) 2024-09-13T17:57:02.899 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.896+0000 7fb5df7fe640 1 -- 172.21.15.188:0/2187580587 learned_addr learned my addr 172.21.15.188:0/2187580587 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-09-13T17:57:02.899 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.896+0000 7fb5dffff640 1 --2- 172.21.15.188:0/2187580587 >> v2:172.21.15.111:3300/0 conn(0x7fb5e0145460 0x7fb5e006d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:02.899 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.896+0000 7fb5df7fe640 1 -- 172.21.15.188:0/2187580587 >> v2:172.21.15.111:3300/0 conn(0x7fb5e0145460 msgr2=0x7fb5e006d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:02.899 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.896+0000 7fb5df7fe640 1 --2- 172.21.15.188:0/2187580587 >> v2:172.21.15.111:3300/0 conn(0x7fb5e0145460 0x7fb5e006d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.899 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.896+0000 7fb5df7fe640 1 -- 172.21.15.188:0/2187580587 >> v1:172.21.15.111:6789/0 conn(0x7fb5e006a2b0 legacy=0x7fb5e00ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:02.899 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.896+0000 7fb5df7fe640 1 -- 172.21.15.188:0/2187580587 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e006d290 con 0x7fb5e0069ed0 2024-09-13T17:57:02.900 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.897+0000 7fb5df7fe640 1 --2- 172.21.15.188:0/2187580587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e0069ed0 0x7fb5e00b2200 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009870 tx=0x7fb5d002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7649805bce2712a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:02.900 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.897+0000 7fb5deffd640 1 -- 172.21.15.188:0/2187580587 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0032440 con 0x7fb5e0069ed0 2024-09-13T17:57:02.900 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.897+0000 7fb5deffd640 1 -- 172.21.15.188:0/2187580587 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d00325a0 con 0x7fb5e0069ed0 2024-09-13T17:57:02.900 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.897+0000 7fb5deffd640 1 -- 172.21.15.188:0/2187580587 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0032930 con 0x7fb5e0069ed0 2024-09-13T17:57:02.901 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.898+0000 7fb5e608e640 1 -- 172.21.15.188:0/2187580587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e0069ed0 msgr2=0x7fb5e00b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:02.901 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.898+0000 7fb5e608e640 1 --2- 172.21.15.188:0/2187580587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e0069ed0 0x7fb5e00b2200 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009870 tx=0x7fb5d002a1c0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.901 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.898+0000 7fb5e608e640 1 -- 172.21.15.188:0/2187580587 shutdown_connections 2024-09-13T17:57:02.901 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.898+0000 7fb5e608e640 1 --2- 172.21.15.188:0/2187580587 >> v2:172.21.15.111:3300/0 conn(0x7fb5e0145460 0x7fb5e006d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.901 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.898+0000 7fb5e608e640 1 --2- 172.21.15.188:0/2187580587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e0069ed0 0x7fb5e00b2200 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.901 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.898+0000 7fb5e608e640 1 -- 172.21.15.188:0/2187580587 >> 172.21.15.188:0/2187580587 conn(0x7fb5e00b4720 msgr2=0x7fb5e00b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:02.901 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.898+0000 7fb5e608e640 1 -- 172.21.15.188:0/2187580587 shutdown_connections 2024-09-13T17:57:02.901 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.898+0000 7fb5e608e640 1 -- 172.21.15.188:0/2187580587 wait complete. 2024-09-13T17:57:02.903 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.899+0000 7fb5e608e640 1 Processor -- start 2024-09-13T17:57:02.903 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.899+0000 7fb5e608e640 1 -- start start 2024-09-13T17:57:02.903 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.900+0000 7fb5e608e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e0069ed0 0x7fb5e015f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:02.904 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.900+0000 7fb5e608e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e006a2b0 0x7fb5e013b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:02.904 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.900+0000 7fb5e608e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5e0145460 0x7fb5e01422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:02.904 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.900+0000 7fb5e608e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e015cac0 con 0x7fb5e0145460 2024-09-13T17:57:02.904 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.900+0000 7fb5e608e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e015d360 con 0x7fb5e006a2b0 2024-09-13T17:57:02.904 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.900+0000 7fb5e608e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e013b640 con 0x7fb5e0069ed0 2024-09-13T17:57:02.904 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.900+0000 7fb5df7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e0069ed0 0x7fb5e015f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:02.904 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.900+0000 7fb5df7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e0069ed0 0x7fb5e015f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:52056/0 (socket says 172.21.15.188:52056) 2024-09-13T17:57:02.904 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.900+0000 7fb5df7fe640 1 -- 172.21.15.188:0/3675808462 learned_addr learned my addr 172.21.15.188:0/3675808462 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-09-13T17:57:02.904 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.900+0000 7fb5d6ffd640 1 --2- 172.21.15.188:0/3675808462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e006a2b0 0x7fb5e013b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:02.904 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.900+0000 7fb5dffff640 1 --2- 172.21.15.188:0/3675808462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5e0145460 0x7fb5e01422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:02.904 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.901+0000 7fb5df7fe640 1 -- 172.21.15.188:0/3675808462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e006a2b0 msgr2=0x7fb5e013b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:02.904 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.901+0000 7fb5df7fe640 1 --2- 172.21.15.188:0/3675808462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e006a2b0 0x7fb5e013b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.904 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.901+0000 7fb5df7fe640 1 -- 172.21.15.188:0/3675808462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5e0145460 msgr2=0x7fb5e01422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:02.904 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.901+0000 7fb5df7fe640 1 --2- 172.21.15.188:0/3675808462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5e0145460 0x7fb5e01422f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.905 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.901+0000 7fb5df7fe640 1 -- 172.21.15.188:0/3675808462 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d00094d0 con 0x7fb5e0069ed0 2024-09-13T17:57:02.905 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.901+0000 7fb5df7fe640 1 --2- 172.21.15.188:0/3675808462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e0069ed0 0x7fb5e015f810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0000c00 tx=0x7fb5d00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:02.905 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.901+0000 7fb5dcff9640 1 -- 172.21.15.188:0/3675808462 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0032440 con 0x7fb5e0069ed0 2024-09-13T17:57:02.905 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.901+0000 7fb5dcff9640 1 -- 172.21.15.188:0/3675808462 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d0030980 con 0x7fb5e0069ed0 2024-09-13T17:57:02.905 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.901+0000 7fb5dcff9640 1 -- 172.21.15.188:0/3675808462 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0030d30 con 0x7fb5e0069ed0 2024-09-13T17:57:02.905 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.901+0000 7fb5e608e640 1 -- 172.21.15.188:0/3675808462 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e013b8a0 con 0x7fb5e0069ed0 2024-09-13T17:57:02.905 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.901+0000 7fb5e608e640 1 -- 172.21.15.188:0/3675808462 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e015cff0 con 0x7fb5e0069ed0 2024-09-13T17:57:02.905 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.901+0000 7fb5dcff9640 1 -- 172.21.15.188:0/3675808462 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fb5d0039040 con 0x7fb5e0069ed0 2024-09-13T17:57:02.905 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.901+0000 7fb5dcff9640 1 -- 172.21.15.188:0/3675808462 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb5d0046440 con 0x7fb5e0069ed0 2024-09-13T17:57:02.905 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.902+0000 7fb5e608e640 1 -- 172.21.15.188:0/3675808462 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e015fd50 con 0x7fb5e0069ed0 2024-09-13T17:57:02.906 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.904+0000 7fb5dcff9640 1 -- 172.21.15.188:0/3675808462 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fb5d0041020 con 0x7fb5e0069ed0 2024-09-13T17:57:02.909 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:02.907+0000 7f6a9ba15040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:57:02.913 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:02.912+0000 7f8156532040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:57:02.972 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.971+0000 7fb5e608e640 1 -- 172.21.15.188:0/3675808462 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "20368658-282f-4098-bfcc-4c95d0bf0def", "id": 0} v 0) v1 -- 0x7fb5e006a710 con 0x7fb5e0069ed0 2024-09-13T17:57:02.978 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.976+0000 7fb5dcff9640 1 -- 172.21.15.188:0/3675808462 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "20368658-282f-4098-bfcc-4c95d0bf0def", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb5d0037070 con 0x7fb5e0069ed0 2024-09-13T17:57:02.978 INFO:teuthology.orchestra.run.smithi188.stdout:0 2024-09-13T17:57:02.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.977+0000 7fb5e608e640 1 -- 172.21.15.188:0/3675808462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e0069ed0 msgr2=0x7fb5e015f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:02.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.977+0000 7fb5e608e640 1 --2- 172.21.15.188:0/3675808462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e0069ed0 0x7fb5e015f810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0000c00 tx=0x7fb5d00046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.977+0000 7fb5e608e640 1 -- 172.21.15.188:0/3675808462 shutdown_connections 2024-09-13T17:57:02.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.977+0000 7fb5e608e640 1 --2- 172.21.15.188:0/3675808462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5e0145460 0x7fb5e01422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.977+0000 7fb5e608e640 1 --2- 172.21.15.188:0/3675808462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e006a2b0 0x7fb5e013b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.977+0000 7fb5e608e640 1 --2- 172.21.15.188:0/3675808462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e0069ed0 0x7fb5e015f810 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:02.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.977+0000 7fb5e608e640 1 -- 172.21.15.188:0/3675808462 >> 172.21.15.188:0/3675808462 conn(0x7fb5e00b4720 msgr2=0x7fb5e015ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:02.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.978+0000 7fb5e608e640 1 -- 172.21.15.188:0/3675808462 shutdown_connections 2024-09-13T17:57:02.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:02.978+0000 7fb5e608e640 1 -- 172.21.15.188:0/3675808462 wait complete. 2024-09-13T17:57:02.989 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 46c9d9c2-beda-4b6d-bdf4-62b24fe98259 1 2024-09-13T17:57:03.086 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.084+0000 7f001c918640 1 Processor -- start 2024-09-13T17:57:03.086 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.084+0000 7f001c918640 1 -- start start 2024-09-13T17:57:03.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.085+0000 7f001c918640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f001815fdc0 0x7f001806afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.085+0000 7f001c918640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001806b520 0x7f001806b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.085+0000 7f001c918640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f00180989d0 0x7f0018098e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.085+0000 7f001c918640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f001806bea0 con 0x7f00180989d0 2024-09-13T17:57:03.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.085+0000 7f001c918640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00180cf590 con 0x7f001815fdc0 2024-09-13T17:57:03.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.085+0000 7f001c918640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00180cef80 con 0x7f001806b520 2024-09-13T17:57:03.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.085+0000 7f001659c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f001815fdc0 0x7f001806afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.085+0000 7f0015d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001806b520 0x7f001806b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.085+0000 7f001659c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f001815fdc0 0x7f001806afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:41172/0 (socket says 172.21.15.188:41172) 2024-09-13T17:57:03.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.085+0000 7f001659c640 1 -- 172.21.15.188:0/2156024605 learned_addr learned my addr 172.21.15.188:0/2156024605 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-09-13T17:57:03.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.085+0000 7f0015d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001806b520 0x7f001806b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:52066/0 (socket says 172.21.15.188:52066) 2024-09-13T17:57:03.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.086+0000 7f0016d9d640 1 --2- 172.21.15.188:0/2156024605 >> v2:172.21.15.111:3300/0 conn(0x7f00180989d0 0x7f0018098e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.086+0000 7f0015d9b640 1 -- 172.21.15.188:0/2156024605 >> v2:172.21.15.188:3300/0 conn(0x7f001815fdc0 msgr2=0x7f001806afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:03.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.086+0000 7f0015d9b640 1 --2- 172.21.15.188:0/2156024605 >> v2:172.21.15.188:3300/0 conn(0x7f001815fdc0 0x7f001806afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.086+0000 7f0015d9b640 1 -- 172.21.15.188:0/2156024605 >> v2:172.21.15.111:3300/0 conn(0x7f00180989d0 msgr2=0x7f0018098e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:03.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.086+0000 7f0015d9b640 1 --2- 172.21.15.188:0/2156024605 >> v2:172.21.15.111:3300/0 conn(0x7f00180989d0 0x7f0018098e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.086+0000 7f0015d9b640 1 -- 172.21.15.188:0/2156024605 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00180b11f0 con 0x7f001806b520 2024-09-13T17:57:03.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.086+0000 7f001659c640 1 --2- 172.21.15.188:0/2156024605 >> v2:172.21.15.188:3300/0 conn(0x7f001815fdc0 0x7f001806afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:03.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.086+0000 7f0015d9b640 1 --2- 172.21.15.188:0/2156024605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001806b520 0x7f001806b960 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f001806e150 tx=0x7f000002a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5abe4f439dcb7d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:03.090 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.086+0000 7f001559a640 1 -- 172.21.15.188:0/2156024605 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000002add0 con 0x7f001806b520 2024-09-13T17:57:03.090 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.086+0000 7f001559a640 1 -- 172.21.15.188:0/2156024605 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0000032440 con 0x7f001806b520 2024-09-13T17:57:03.090 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.087+0000 7f001559a640 1 -- 172.21.15.188:0/2156024605 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00000327d0 con 0x7f001806b520 2024-09-13T17:57:03.093 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.087+0000 7f001c918640 1 -- 172.21.15.188:0/2156024605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001806b520 msgr2=0x7f001806b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.093 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.087+0000 7f001c918640 1 --2- 172.21.15.188:0/2156024605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001806b520 0x7f001806b960 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f001806e150 tx=0x7f000002a080 comp rx=0 tx=0).stop 2024-09-13T17:57:03.093 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.087+0000 7f001c918640 1 -- 172.21.15.188:0/2156024605 shutdown_connections 2024-09-13T17:57:03.093 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.087+0000 7f001c918640 1 --2- 172.21.15.188:0/2156024605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001806b520 0x7f001806b960 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.093 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.087+0000 7f001c918640 1 --2- 172.21.15.188:0/2156024605 >> v2:172.21.15.111:3300/0 conn(0x7f00180989d0 0x7f0018098e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.093 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.087+0000 7f001c918640 1 --2- 172.21.15.188:0/2156024605 >> v2:172.21.15.188:3300/0 conn(0x7f001815fdc0 0x7f001806afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.087+0000 7f001c918640 1 -- 172.21.15.188:0/2156024605 >> 172.21.15.188:0/2156024605 conn(0x7f00180b1e40 msgr2=0x7f00180b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:03.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.087+0000 7f001c918640 1 -- 172.21.15.188:0/2156024605 shutdown_connections 2024-09-13T17:57:03.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.087+0000 7f001c918640 1 -- 172.21.15.188:0/2156024605 wait complete. 2024-09-13T17:57:03.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.089+0000 7f001c918640 1 Processor -- start 2024-09-13T17:57:03.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.089+0000 7f001c918640 1 -- start start 2024-09-13T17:57:03.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.089+0000 7f001c918640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f001806b520 0x7f0018153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.089+0000 7f001c918640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00180989d0 0x7f001815a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.089+0000 7f001c918640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f001815fdc0 0x7f001809d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.089+0000 7f001c918640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0018154520 con 0x7f001815fdc0 2024-09-13T17:57:03.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.089+0000 7f001c918640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0018155380 con 0x7f001806b520 2024-09-13T17:57:03.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.089+0000 7f001c918640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00181570c0 con 0x7f00180989d0 2024-09-13T17:57:03.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.089+0000 7f001659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f001806b520 0x7f0018153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.089+0000 7f0015d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00180989d0 0x7f001815a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.095 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.090+0000 7f001659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f001806b520 0x7f0018153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:41178/0 (socket says 172.21.15.188:41178) 2024-09-13T17:57:03.095 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.090+0000 7f001659c640 1 -- 172.21.15.188:0/2120348935 learned_addr learned my addr 172.21.15.188:0/2120348935 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-09-13T17:57:03.095 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.090+0000 7f0016d9d640 1 --2- 172.21.15.188:0/2120348935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f001815fdc0 0x7f001809d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.095 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.090+0000 7f001659c640 1 -- 172.21.15.188:0/2120348935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00180989d0 msgr2=0x7f001815a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.095 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.090+0000 7f001659c640 1 --2- 172.21.15.188:0/2120348935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00180989d0 0x7f001815a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.095 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.090+0000 7f001659c640 1 -- 172.21.15.188:0/2120348935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f001815fdc0 msgr2=0x7f001809d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.095 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.090+0000 7f001659c640 1 --2- 172.21.15.188:0/2120348935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f001815fdc0 0x7f001809d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.095 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.090+0000 7f001659c640 1 -- 172.21.15.188:0/2120348935 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00080094d0 con 0x7f001806b520 2024-09-13T17:57:03.095 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.090+0000 7f0015d9b640 1 --2- 172.21.15.188:0/2120348935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00180989d0 0x7f001815a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:03.095 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.090+0000 7f001659c640 1 --2- 172.21.15.188:0/2120348935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f001806b520 0x7f0018153ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f000800e9b0 tx=0x7f000800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:03.095 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.090+0000 7f000effd640 1 -- 172.21.15.188:0/2120348935 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000800c9a0 con 0x7f001806b520 2024-09-13T17:57:03.095 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.090+0000 7f000effd640 1 -- 172.21.15.188:0/2120348935 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000800cb00 con 0x7f001806b520 2024-09-13T17:57:03.095 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.090+0000 7f001c918640 1 -- 172.21.15.188:0/2120348935 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0000009400 con 0x7f001806b520 2024-09-13T17:57:03.096 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.090+0000 7f000effd640 1 -- 172.21.15.188:0/2120348935 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000800ce70 con 0x7f001806b520 2024-09-13T17:57:03.096 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.090+0000 7f001c918640 1 -- 172.21.15.188:0/2120348935 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0018154a50 con 0x7f001806b520 2024-09-13T17:57:03.096 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.090+0000 7f000effd640 1 -- 172.21.15.188:0/2120348935 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0008005c90 con 0x7f001806b520 2024-09-13T17:57:03.096 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.090+0000 7f000effd640 1 -- 172.21.15.188:0/2120348935 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f0008013820 con 0x7f001806b520 2024-09-13T17:57:03.096 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.091+0000 7f001c918640 1 -- 172.21.15.188:0/2120348935 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001806cbc0 con 0x7f001806b520 2024-09-13T17:57:03.096 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.093+0000 7f000effd640 1 -- 172.21.15.188:0/2120348935 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f000803be30 con 0x7f001806b520 2024-09-13T17:57:03.163 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.161+0000 7f001c918640 1 -- 172.21.15.188:0/2120348935 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "46c9d9c2-beda-4b6d-bdf4-62b24fe98259", "id": 1} v 0) v1 -- 0x7f001806b960 con 0x7f001806b520 2024-09-13T17:57:03.171 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.170+0000 7f000effd640 1 -- 172.21.15.188:0/2120348935 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "46c9d9c2-beda-4b6d-bdf4-62b24fe98259", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f000802a030 con 0x7f001806b520 2024-09-13T17:57:03.171 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-09-13T17:57:03.172 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.170+0000 7f001c918640 1 -- 172.21.15.188:0/2120348935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f001806b520 msgr2=0x7f0018153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.173 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.171+0000 7f001c918640 1 --2- 172.21.15.188:0/2120348935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f001806b520 0x7f0018153ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f000800e9b0 tx=0x7f000800ee70 comp rx=0 tx=0).stop 2024-09-13T17:57:03.173 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.171+0000 7f001c918640 1 -- 172.21.15.188:0/2120348935 shutdown_connections 2024-09-13T17:57:03.173 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.171+0000 7f001c918640 1 --2- 172.21.15.188:0/2120348935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f001815fdc0 0x7f001809d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.173 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.171+0000 7f001c918640 1 --2- 172.21.15.188:0/2120348935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00180989d0 0x7f001815a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.173 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.171+0000 7f001c918640 1 --2- 172.21.15.188:0/2120348935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f001806b520 0x7f0018153ce0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.173 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.171+0000 7f001c918640 1 -- 172.21.15.188:0/2120348935 >> 172.21.15.188:0/2120348935 conn(0x7f00180b1e40 msgr2=0x7f001806fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:03.173 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.171+0000 7f001c918640 1 -- 172.21.15.188:0/2120348935 shutdown_connections 2024-09-13T17:57:03.173 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.171+0000 7f001c918640 1 -- 172.21.15.188:0/2120348935 wait complete. 2024-09-13T17:57:03.182 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279 2 2024-09-13T17:57:03.224 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:03.221+0000 7f6a9ba15040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:57:03.241 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:03.240+0000 7f8156532040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:57:03.275 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:03.274+0000 7f6a9ba15040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:57:03.280 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.278+0000 7fcc0f57d640 1 Processor -- start 2024-09-13T17:57:03.280 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.278+0000 7fcc0f57d640 1 -- start start 2024-09-13T17:57:03.281 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.279+0000 7fcc0f57d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcc080d2760 0x7fcc080cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.282 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.279+0000 7fcc0f57d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc080cde20 0x7fcc080d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.282 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.279+0000 7fcc0f57d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc080cd450 con 0x7fcc0815c500 2024-09-13T17:57:03.282 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.279+0000 7fcc0f57d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc080d0cf0 con 0x7fcc080d2760 2024-09-13T17:57:03.282 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.279+0000 7fcc0f57d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc080d0fd0 con 0x7fcc080cde20 2024-09-13T17:57:03.282 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.279+0000 7fcc0db1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc080cde20 0x7fcc080d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.282 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.279+0000 7fcc0d319640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fcc0815c500 0x7fcc080d2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:38510/0 (socket says 172.21.15.188:38510) 2024-09-13T17:57:03.282 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.279+0000 7fcc0d319640 1 -- 172.21.15.188:0/2371998521 learned_addr learned my addr 172.21.15.188:0/2371998521 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-09-13T17:57:03.282 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.279+0000 7fcc0cb18640 1 --2- 172.21.15.188:0/2371998521 >> v2:172.21.15.111:3300/0 conn(0x7fcc080d2760 0x7fcc080cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.282 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.280+0000 7fcc0db1a640 1 -- 172.21.15.188:0/2371998521 >> v2:172.21.15.111:3300/0 conn(0x7fcc080d2760 msgr2=0x7fcc080cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:03.282 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.280+0000 7fcc0db1a640 1 --2- 172.21.15.188:0/2371998521 >> v2:172.21.15.111:3300/0 conn(0x7fcc080d2760 0x7fcc080cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.282 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.280+0000 7fcc0db1a640 1 -- 172.21.15.188:0/2371998521 >> v1:172.21.15.188:6789/0 conn(0x7fcc0815c500 legacy=0x7fcc080d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.282 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.280+0000 7fcc0db1a640 1 -- 172.21.15.188:0/2371998521 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc080cd0e0 con 0x7fcc080cde20 2024-09-13T17:57:03.283 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.281+0000 7fcc0db1a640 1 --2- 172.21.15.188:0/2371998521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc080cde20 0x7fcc080d0000 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009d90 tx=0x7fcc0002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96df3d22d8d2fb6b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:03.283 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.282+0000 7fcbfffff640 1 -- 172.21.15.188:0/2371998521 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00007740 con 0x7fcc080cde20 2024-09-13T17:57:03.283 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.282+0000 7fcbfffff640 1 -- 172.21.15.188:0/2371998521 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0002d050 con 0x7fcc080cde20 2024-09-13T17:57:03.283 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.282+0000 7fcbfffff640 1 -- 172.21.15.188:0/2371998521 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00007ac0 con 0x7fcc080cde20 2024-09-13T17:57:03.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.283+0000 7fcc0f57d640 1 -- 172.21.15.188:0/2371998521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc080cde20 msgr2=0x7fcc080d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.283+0000 7fcc0f57d640 1 --2- 172.21.15.188:0/2371998521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc080cde20 0x7fcc080d0000 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009d90 tx=0x7fcc0002a660 comp rx=0 tx=0).stop 2024-09-13T17:57:03.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.283+0000 7fcc0f57d640 1 -- 172.21.15.188:0/2371998521 shutdown_connections 2024-09-13T17:57:03.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.283+0000 7fcc0f57d640 1 --2- 172.21.15.188:0/2371998521 >> v2:172.21.15.111:3300/0 conn(0x7fcc080d2760 0x7fcc080cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.283+0000 7fcc0f57d640 1 --2- 172.21.15.188:0/2371998521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc080cde20 0x7fcc080d0000 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.283+0000 7fcc0f57d640 1 -- 172.21.15.188:0/2371998521 >> 172.21.15.188:0/2371998521 conn(0x7fcc08089f60 msgr2=0x7fcc0809b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:03.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.283+0000 7fcc0f57d640 1 -- 172.21.15.188:0/2371998521 shutdown_connections 2024-09-13T17:57:03.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.283+0000 7fcc0f57d640 1 -- 172.21.15.188:0/2371998521 wait complete. 2024-09-13T17:57:03.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.285+0000 7fcc0f57d640 1 Processor -- start 2024-09-13T17:57:03.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.285+0000 7fcc0f57d640 1 -- start start 2024-09-13T17:57:03.289 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.286+0000 7fcc0f57d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc080cde20 0x7fcc080c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.289 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.286+0000 7fcc0f57d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc080d2760 0x7fcc080af080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.290 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.286+0000 7fcc0f57d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc0815c500 0x7fcc080af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.290 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.286+0000 7fcc0f57d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc081f2ad0 con 0x7fcc080d2760 2024-09-13T17:57:03.290 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.286+0000 7fcc0f57d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc080cc420 con 0x7fcc080cde20 2024-09-13T17:57:03.290 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.286+0000 7fcc0f57d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc081f2480 con 0x7fcc0815c500 2024-09-13T17:57:03.290 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.286+0000 7fcc0db1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc0815c500 0x7fcc080af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.290 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.286+0000 7fcc0d319640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc080cde20 0x7fcc080c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.290 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.286+0000 7fcc0db1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc0815c500 0x7fcc080af5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:52102/0 (socket says 172.21.15.188:52102) 2024-09-13T17:57:03.290 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.286+0000 7fcc0db1a640 1 -- 172.21.15.188:0/943886341 learned_addr learned my addr 172.21.15.188:0/943886341 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-09-13T17:57:03.290 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.286+0000 7fcc0cb18640 1 --2- 172.21.15.188:0/943886341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc080d2760 0x7fcc080af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.290 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.287+0000 7fcc0d319640 1 -- 172.21.15.188:0/943886341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc0815c500 msgr2=0x7fcc080af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.290 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.287+0000 7fcc0d319640 1 --2- 172.21.15.188:0/943886341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc0815c500 0x7fcc080af5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.290 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.287+0000 7fcc0d319640 1 -- 172.21.15.188:0/943886341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc080d2760 msgr2=0x7fcc080af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.290 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.287+0000 7fcc0d319640 1 --2- 172.21.15.188:0/943886341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc080d2760 0x7fcc080af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.287+0000 7fcc0d319640 1 -- 172.21.15.188:0/943886341 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc00009a70 con 0x7fcc080cde20 2024-09-13T17:57:03.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.287+0000 7fcc0db1a640 1 --2- 172.21.15.188:0/943886341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc0815c500 0x7fcc080af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:03.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.287+0000 7fcc0cb18640 1 --2- 172.21.15.188:0/943886341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc080d2760 0x7fcc080af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:03.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.287+0000 7fcc0d319640 1 --2- 172.21.15.188:0/943886341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc080cde20 0x7fcc080c8570 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400c440 tx=0x7fcc0400c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:03.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.287+0000 7fcbfdffb640 1 -- 172.21.15.188:0/943886341 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04005ce0 con 0x7fcc080cde20 2024-09-13T17:57:03.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.287+0000 7fcbfdffb640 1 -- 172.21.15.188:0/943886341 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc04005e40 con 0x7fcc080cde20 2024-09-13T17:57:03.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.287+0000 7fcc0f57d640 1 -- 172.21.15.188:0/943886341 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc081f2760 con 0x7fcc080cde20 2024-09-13T17:57:03.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.287+0000 7fcbfdffb640 1 -- 172.21.15.188:0/943886341 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc040072d0 con 0x7fcc080cde20 2024-09-13T17:57:03.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.287+0000 7fcc0f57d640 1 -- 172.21.15.188:0/943886341 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc081f2f60 con 0x7fcc080cde20 2024-09-13T17:57:03.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.287+0000 7fcbfdffb640 1 -- 172.21.15.188:0/943886341 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fcc0401c440 con 0x7fcc080cde20 2024-09-13T17:57:03.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.287+0000 7fcbfdffb640 1 -- 172.21.15.188:0/943886341 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fcc04012820 con 0x7fcc080cde20 2024-09-13T17:57:03.292 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.288+0000 7fcc0f57d640 1 -- 172.21.15.188:0/943886341 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc081f3180 con 0x7fcc080cde20 2024-09-13T17:57:03.292 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.290+0000 7fcbfdffb640 1 -- 172.21.15.188:0/943886341 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fcc04021060 con 0x7fcc080cde20 2024-09-13T17:57:03.303 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:03.302+0000 7f8156532040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:57:03.360 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.358+0000 7fcc0f57d640 1 -- 172.21.15.188:0/943886341 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279", "id": 2} v 0) v1 -- 0x7fcc080d08c0 con 0x7fcc080cde20 2024-09-13T17:57:03.368 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.366+0000 7fcbfdffb640 1 -- 172.21.15.188:0/943886341 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcc04017070 con 0x7fcc080cde20 2024-09-13T17:57:03.368 INFO:teuthology.orchestra.run.smithi188.stdout:2 2024-09-13T17:57:03.369 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.367+0000 7fcc0f57d640 1 -- 172.21.15.188:0/943886341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc080cde20 msgr2=0x7fcc080c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.369 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.367+0000 7fcc0f57d640 1 --2- 172.21.15.188:0/943886341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc080cde20 0x7fcc080c8570 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400c440 tx=0x7fcc0400c900 comp rx=0 tx=0).stop 2024-09-13T17:57:03.369 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.367+0000 7fcc0f57d640 1 -- 172.21.15.188:0/943886341 shutdown_connections 2024-09-13T17:57:03.370 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.367+0000 7fcc0f57d640 1 --2- 172.21.15.188:0/943886341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc0815c500 0x7fcc080af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.370 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.367+0000 7fcc0f57d640 1 --2- 172.21.15.188:0/943886341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc080d2760 0x7fcc080af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.370 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.367+0000 7fcc0f57d640 1 --2- 172.21.15.188:0/943886341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc080cde20 0x7fcc080c8570 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.370 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.367+0000 7fcc0f57d640 1 -- 172.21.15.188:0/943886341 >> 172.21.15.188:0/943886341 conn(0x7fcc08089f60 msgr2=0x7fcc0806c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:03.370 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.367+0000 7fcc0f57d640 1 -- 172.21.15.188:0/943886341 shutdown_connections 2024-09-13T17:57:03.370 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.367+0000 7fcc0f57d640 1 -- 172.21.15.188:0/943886341 wait complete. 2024-09-13T17:57:03.383 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 9bc539c8-40b1-4070-9967-891d80f266a4 3 2024-09-13T17:57:03.424 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:03.422+0000 7f6a9ba15040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:57:03.463 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:03.461+0000 7f8156532040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:57:03.480 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.479+0000 7fa7d2e44640 1 Processor -- start 2024-09-13T17:57:03.481 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.479+0000 7fa7d2e44640 1 -- start start 2024-09-13T17:57:03.482 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.479+0000 7fa7d2e44640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7cc0ad360 0x7fa7cc0b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.482 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.479+0000 7fa7d2e44640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa7cc06e5e0 0x7fa7cc0b3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.482 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.480+0000 7fa7d2e44640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7cc0ad7f0 con 0x7fa7cc0b2200 2024-09-13T17:57:03.482 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.480+0000 7fa7d2e44640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7cc0adad0 con 0x7fa7cc06e5e0 2024-09-13T17:57:03.482 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.480+0000 7fa7d2e44640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc0addb0 con 0x7fa7cc0ad360 2024-09-13T17:57:03.482 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.480+0000 7fa7cbfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7cc0ad360 0x7fa7cc0b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.482 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.480+0000 7fa7cbfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7cc0ad360 0x7fa7cc0b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:52110/0 (socket says 172.21.15.188:52110) 2024-09-13T17:57:03.482 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.480+0000 7fa7cbfff640 1 -- 172.21.15.188:0/1787479574 learned_addr learned my addr 172.21.15.188:0/1787479574 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-09-13T17:57:03.482 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.480+0000 7fa7d0be0640 1 --1- 172.21.15.188:0/1787479574 >> v1:172.21.15.188:6789/0 conn(0x7fa7cc0b2200 0x7fa7cc0b25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:38514/0 (socket says 172.21.15.188:38514) 2024-09-13T17:57:03.482 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.480+0000 7fa7d13e1640 1 --2- 172.21.15.188:0/1787479574 >> v2:172.21.15.111:3300/0 conn(0x7fa7cc06e5e0 0x7fa7cc0b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.483 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.480+0000 7fa7cbfff640 1 -- 172.21.15.188:0/1787479574 >> v2:172.21.15.111:3300/0 conn(0x7fa7cc06e5e0 msgr2=0x7fa7cc0b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:03.483 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.480+0000 7fa7cbfff640 1 --2- 172.21.15.188:0/1787479574 >> v2:172.21.15.111:3300/0 conn(0x7fa7cc06e5e0 0x7fa7cc0b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.483 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.480+0000 7fa7cbfff640 1 -- 172.21.15.188:0/1787479574 >> v1:172.21.15.188:6789/0 conn(0x7fa7cc0b2200 legacy=0x7fa7cc0b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.483 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.480+0000 7fa7cb7fe640 1 -- 172.21.15.188:0/1787479574 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896888706 0 0) 0x7fa7cc0ad7f0 con 0x7fa7cc0b2200 2024-09-13T17:57:03.483 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.480+0000 7fa7cbfff640 1 -- 172.21.15.188:0/1787479574 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc0ae090 con 0x7fa7cc0ad360 2024-09-13T17:57:03.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.481+0000 7fa7cbfff640 1 --2- 172.21.15.188:0/1787479574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7cc0ad360 0x7fa7cc0b2cf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009870 tx=0x7fa7b802e670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58d65d7c5fcf1d36 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:03.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.481+0000 7fa7cb7fe640 1 -- 172.21.15.188:0/1787479574 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8033710 con 0x7fa7cc0ad360 2024-09-13T17:57:03.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.481+0000 7fa7cb7fe640 1 -- 172.21.15.188:0/1787479574 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b8031050 con 0x7fa7cc0ad360 2024-09-13T17:57:03.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.481+0000 7fa7cb7fe640 1 -- 172.21.15.188:0/1787479574 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8033a90 con 0x7fa7cc0ad360 2024-09-13T17:57:03.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.482+0000 7fa7d2e44640 1 -- 172.21.15.188:0/1787479574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7cc0ad360 msgr2=0x7fa7cc0b2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.482+0000 7fa7d2e44640 1 --2- 172.21.15.188:0/1787479574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7cc0ad360 0x7fa7cc0b2cf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009870 tx=0x7fa7b802e670 comp rx=0 tx=0).stop 2024-09-13T17:57:03.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.482+0000 7fa7d2e44640 1 -- 172.21.15.188:0/1787479574 shutdown_connections 2024-09-13T17:57:03.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.482+0000 7fa7d2e44640 1 --2- 172.21.15.188:0/1787479574 >> v2:172.21.15.111:3300/0 conn(0x7fa7cc06e5e0 0x7fa7cc0b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.482+0000 7fa7d2e44640 1 --2- 172.21.15.188:0/1787479574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7cc0ad360 0x7fa7cc0b2cf0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.482+0000 7fa7d2e44640 1 -- 172.21.15.188:0/1787479574 >> 172.21.15.188:0/1787479574 conn(0x7fa7cc0b4720 msgr2=0x7fa7cc0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:03.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.482+0000 7fa7d2e44640 1 -- 172.21.15.188:0/1787479574 shutdown_connections 2024-09-13T17:57:03.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.482+0000 7fa7d2e44640 1 -- 172.21.15.188:0/1787479574 wait complete. 2024-09-13T17:57:03.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.483+0000 7fa7d2e44640 1 Processor -- start 2024-09-13T17:57:03.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.483+0000 7fa7d2e44640 1 -- start start 2024-09-13T17:57:03.488 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.484+0000 7fa7d2e44640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7cc06e5e0 0x7fa7cc158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.488 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.484+0000 7fa7d2e44640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7cc0b2200 0x7fa7cc1517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.488 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.484+0000 7fa7d2e44640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7cc151ed0 0x7fa7cc1e5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.488 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.484+0000 7fa7d2e44640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc1e4be0 con 0x7fa7cc151ed0 2024-09-13T17:57:03.488 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.484+0000 7fa7d2e44640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc1e3fd0 con 0x7fa7cc0b2200 2024-09-13T17:57:03.488 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.484+0000 7fa7d2e44640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc1e42b0 con 0x7fa7cc06e5e0 2024-09-13T17:57:03.488 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.484+0000 7fa7d0be0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7cc06e5e0 0x7fa7cc158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.488 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.484+0000 7fa7cbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7cc0b2200 0x7fa7cc1517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.488 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.484+0000 7fa7cbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7cc0b2200 0x7fa7cc1517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:41194/0 (socket says 172.21.15.188:41194) 2024-09-13T17:57:03.488 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.484+0000 7fa7d0be0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7cc06e5e0 0x7fa7cc158570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:52126/0 (socket says 172.21.15.188:52126) 2024-09-13T17:57:03.488 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.484+0000 7fa7cbfff640 1 -- 172.21.15.188:0/3954141110 learned_addr learned my addr 172.21.15.188:0/3954141110 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-09-13T17:57:03.488 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.484+0000 7fa7d13e1640 1 --2- 172.21.15.188:0/3954141110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7cc151ed0 0x7fa7cc1e5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.488 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.485+0000 7fa7cbfff640 1 -- 172.21.15.188:0/3954141110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7cc06e5e0 msgr2=0x7fa7cc158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.489 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.485+0000 7fa7cbfff640 1 --2- 172.21.15.188:0/3954141110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7cc06e5e0 0x7fa7cc158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.489 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.485+0000 7fa7cbfff640 1 -- 172.21.15.188:0/3954141110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7cc151ed0 msgr2=0x7fa7cc1e5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.489 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.485+0000 7fa7cbfff640 1 --2- 172.21.15.188:0/3954141110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7cc151ed0 0x7fa7cc1e5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.489 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.485+0000 7fa7cbfff640 1 -- 172.21.15.188:0/3954141110 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b80094d0 con 0x7fa7cc0b2200 2024-09-13T17:57:03.489 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.485+0000 7fa7d0be0640 1 --2- 172.21.15.188:0/3954141110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7cc06e5e0 0x7fa7cc158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:03.489 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.485+0000 7fa7cbfff640 1 --2- 172.21.15.188:0/3954141110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7cc0b2200 0x7fa7cc1517c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8000c00 tx=0x7fa7b80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:03.489 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.485+0000 7fa7c97fa640 1 -- 172.21.15.188:0/3954141110 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b803e810 con 0x7fa7cc0b2200 2024-09-13T17:57:03.489 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.485+0000 7fa7c97fa640 1 -- 172.21.15.188:0/3954141110 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b802ee90 con 0x7fa7cc0b2200 2024-09-13T17:57:03.489 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.485+0000 7fa7d2e44640 1 -- 172.21.15.188:0/3954141110 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7cc1e4590 con 0x7fa7cc0b2200 2024-09-13T17:57:03.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.485+0000 7fa7c97fa640 1 -- 172.21.15.188:0/3954141110 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b803ebb0 con 0x7fa7cc0b2200 2024-09-13T17:57:03.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.485+0000 7fa7d2e44640 1 -- 172.21.15.188:0/3954141110 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7cc1e55b0 con 0x7fa7cc0b2200 2024-09-13T17:57:03.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.485+0000 7fa7c97fa640 1 -- 172.21.15.188:0/3954141110 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fa7b803c040 con 0x7fa7cc0b2200 2024-09-13T17:57:03.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.485+0000 7fa7c97fa640 1 -- 172.21.15.188:0/3954141110 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fa7b8035890 con 0x7fa7cc0b2200 2024-09-13T17:57:03.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.486+0000 7fa7d2e44640 1 -- 172.21.15.188:0/3954141110 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7cc06a500 con 0x7fa7cc0b2200 2024-09-13T17:57:03.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.488+0000 7fa7c97fa640 1 -- 172.21.15.188:0/3954141110 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fa7b8068e30 con 0x7fa7cc0b2200 2024-09-13T17:57:03.555 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.554+0000 7fa7d2e44640 1 -- 172.21.15.188:0/3954141110 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9bc539c8-40b1-4070-9967-891d80f266a4", "id": 3} v 0) v1 -- 0x7fa7cc0cb570 con 0x7fa7cc0b2200 2024-09-13T17:57:03.563 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.562+0000 7fa7c97fa640 1 -- 172.21.15.188:0/3954141110 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9bc539c8-40b1-4070-9967-891d80f266a4", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa7b8057070 con 0x7fa7cc0b2200 2024-09-13T17:57:03.564 INFO:teuthology.orchestra.run.smithi188.stdout:3 2024-09-13T17:57:03.564 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.562+0000 7fa7d2e44640 1 -- 172.21.15.188:0/3954141110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7cc0b2200 msgr2=0x7fa7cc1517c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.565 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.562+0000 7fa7d2e44640 1 --2- 172.21.15.188:0/3954141110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7cc0b2200 0x7fa7cc1517c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8000c00 tx=0x7fa7b80046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.565 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.563+0000 7fa7d2e44640 1 -- 172.21.15.188:0/3954141110 shutdown_connections 2024-09-13T17:57:03.565 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.563+0000 7fa7d2e44640 1 --2- 172.21.15.188:0/3954141110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7cc151ed0 0x7fa7cc1e5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.565 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.563+0000 7fa7d2e44640 1 --2- 172.21.15.188:0/3954141110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7cc0b2200 0x7fa7cc1517c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.565 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.563+0000 7fa7d2e44640 1 --2- 172.21.15.188:0/3954141110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7cc06e5e0 0x7fa7cc158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.565 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.563+0000 7fa7d2e44640 1 -- 172.21.15.188:0/3954141110 >> 172.21.15.188:0/3954141110 conn(0x7fa7cc0b4720 msgr2=0x7fa7cc15b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:03.565 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.563+0000 7fa7d2e44640 1 -- 172.21.15.188:0/3954141110 shutdown_connections 2024-09-13T17:57:03.565 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.563+0000 7fa7d2e44640 1 -- 172.21.15.188:0/3954141110 wait complete. 2024-09-13T17:57:03.574 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new a7264062-0f38-44a0-acc9-796dfac73619 4 2024-09-13T17:57:03.672 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.670+0000 7fb5d3bed640 1 Processor -- start 2024-09-13T17:57:03.672 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.670+0000 7fb5d3bed640 1 -- start start 2024-09-13T17:57:03.674 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.671+0000 7fb5d3bed640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb5cc159200 0x7fb5cc0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.674 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.671+0000 7fb5d3bed640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb5cc0ade90 0x7fb5cc0b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.674 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.671+0000 7fb5d3bed640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5cc15cf50 con 0x7fb5cc0b33d0 2024-09-13T17:57:03.674 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.671+0000 7fb5d3bed640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5cc0ce930 con 0x7fb5cc0ade90 2024-09-13T17:57:03.674 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.671+0000 7fb5d3bed640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5cc1595e0 con 0x7fb5cc159200 2024-09-13T17:57:03.674 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.671+0000 7fb5d1989640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb5cc159200 0x7fb5cc0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.674 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.671+0000 7fb5d1989640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb5cc159200 0x7fb5cc0cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:41208/0 (socket says 172.21.15.188:41208) 2024-09-13T17:57:03.675 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.671+0000 7fb5d1989640 1 -- 172.21.15.188:0/4020141877 learned_addr learned my addr 172.21.15.188:0/4020141877 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-09-13T17:57:03.675 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.671+0000 7fb5d1188640 1 --2- 172.21.15.188:0/4020141877 >> v2:172.21.15.111:3300/0 conn(0x7fb5cc0ade90 0x7fb5cc0b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.675 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.671+0000 7fb5d1989640 1 -- 172.21.15.188:0/4020141877 >> v2:172.21.15.111:3300/0 conn(0x7fb5cc0ade90 msgr2=0x7fb5cc0b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:03.675 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.671+0000 7fb5d1989640 1 --2- 172.21.15.188:0/4020141877 >> v2:172.21.15.111:3300/0 conn(0x7fb5cc0ade90 0x7fb5cc0b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.675 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.671+0000 7fb5d1989640 1 -- 172.21.15.188:0/4020141877 >> v1:172.21.15.188:6789/0 conn(0x7fb5cc0b33d0 legacy=0x7fb5cc15d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.675 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.671+0000 7fb5d1989640 1 -- 172.21.15.188:0/4020141877 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc0cddf0 con 0x7fb5cc159200 2024-09-13T17:57:03.675 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.672+0000 7fb5d1989640 1 --2- 172.21.15.188:0/4020141877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5cc159200 0x7fb5cc0cd8b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc066170 tx=0x7fb5c002c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bae9635b756204 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:03.675 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.672+0000 7fb5d0987640 1 -- 172.21.15.188:0/4020141877 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0032640 con 0x7fb5cc159200 2024-09-13T17:57:03.675 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.672+0000 7fb5d0987640 1 -- 172.21.15.188:0/4020141877 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c00327a0 con 0x7fb5cc159200 2024-09-13T17:57:03.675 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.672+0000 7fb5d0987640 1 -- 172.21.15.188:0/4020141877 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0032b30 con 0x7fb5cc159200 2024-09-13T17:57:03.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.673+0000 7fb5d3bed640 1 -- 172.21.15.188:0/4020141877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5cc159200 msgr2=0x7fb5cc0cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.673+0000 7fb5d3bed640 1 --2- 172.21.15.188:0/4020141877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5cc159200 0x7fb5cc0cd8b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc066170 tx=0x7fb5c002c940 comp rx=0 tx=0).stop 2024-09-13T17:57:03.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.673+0000 7fb5d3bed640 1 -- 172.21.15.188:0/4020141877 shutdown_connections 2024-09-13T17:57:03.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.673+0000 7fb5d3bed640 1 --2- 172.21.15.188:0/4020141877 >> v2:172.21.15.111:3300/0 conn(0x7fb5cc0ade90 0x7fb5cc0b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.673+0000 7fb5d3bed640 1 --2- 172.21.15.188:0/4020141877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5cc159200 0x7fb5cc0cd8b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.673+0000 7fb5d3bed640 1 -- 172.21.15.188:0/4020141877 >> 172.21.15.188:0/4020141877 conn(0x7fb5cc069ed0 msgr2=0x7fb5cc06a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:03.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.673+0000 7fb5d3bed640 1 -- 172.21.15.188:0/4020141877 shutdown_connections 2024-09-13T17:57:03.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.673+0000 7fb5d3bed640 1 -- 172.21.15.188:0/4020141877 wait complete. 2024-09-13T17:57:03.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.674+0000 7fb5d3bed640 1 Processor -- start 2024-09-13T17:57:03.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.674+0000 7fb5d3bed640 1 -- start start 2024-09-13T17:57:03.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d3bed640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5cc0ade90 0x7fb5cc14e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d3bed640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5cc0b33d0 0x7fb5cc14d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d3bed640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5cc159200 0x7fb5cc15bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d3bed640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc14da60 con 0x7fb5cc159200 2024-09-13T17:57:03.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d3bed640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc15c8d0 con 0x7fb5cc0ade90 2024-09-13T17:57:03.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d3bed640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc150590 con 0x7fb5cc0b33d0 2024-09-13T17:57:03.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d1989640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5cc0ade90 0x7fb5cc14e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.680 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d1188640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5cc0b33d0 0x7fb5cc14d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.680 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d1989640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5cc0ade90 0x7fb5cc14e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:41224/0 (socket says 172.21.15.188:41224) 2024-09-13T17:57:03.680 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d1989640 1 -- 172.21.15.188:0/4238090495 learned_addr learned my addr 172.21.15.188:0/4238090495 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-09-13T17:57:03.680 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d1188640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5cc0b33d0 0x7fb5cc14d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:52138/0 (socket says 172.21.15.188:52138) 2024-09-13T17:57:03.680 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d218a640 1 --2- 172.21.15.188:0/4238090495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5cc159200 0x7fb5cc15bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.680 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d1989640 1 -- 172.21.15.188:0/4238090495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5cc0b33d0 msgr2=0x7fb5cc14d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.680 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d1989640 1 --2- 172.21.15.188:0/4238090495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5cc0b33d0 0x7fb5cc14d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.680 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d1989640 1 -- 172.21.15.188:0/4238090495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5cc159200 msgr2=0x7fb5cc15bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.680 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d1989640 1 --2- 172.21.15.188:0/4238090495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5cc159200 0x7fb5cc15bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.680 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d1989640 1 -- 172.21.15.188:0/4238090495 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c000b770 con 0x7fb5cc0ade90 2024-09-13T17:57:03.680 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.675+0000 7fb5d1188640 1 --2- 172.21.15.188:0/4238090495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5cc0b33d0 0x7fb5cc14d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:03.680 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.676+0000 7fb5d1989640 1 --2- 172.21.15.188:0/4238090495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5cc0ade90 0x7fb5cc14e070 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb5c002c980 tx=0x7fb5c00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:03.680 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.676+0000 7fb5be7fc640 1 -- 172.21.15.188:0/4238090495 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c003bbe0 con 0x7fb5cc0ade90 2024-09-13T17:57:03.681 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.676+0000 7fb5be7fc640 1 -- 172.21.15.188:0/4238090495 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c003bd40 con 0x7fb5cc0ade90 2024-09-13T17:57:03.681 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.676+0000 7fb5d3bed640 1 -- 172.21.15.188:0/4238090495 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5cc150870 con 0x7fb5cc0ade90 2024-09-13T17:57:03.681 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.676+0000 7fb5be7fc640 1 -- 172.21.15.188:0/4238090495 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0043630 con 0x7fb5cc0ade90 2024-09-13T17:57:03.681 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.676+0000 7fb5d3bed640 1 -- 172.21.15.188:0/4238090495 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5cc150e30 con 0x7fb5cc0ade90 2024-09-13T17:57:03.681 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.676+0000 7fb5be7fc640 1 -- 172.21.15.188:0/4238090495 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fb5c0043a90 con 0x7fb5cc0ade90 2024-09-13T17:57:03.681 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.676+0000 7fb5be7fc640 1 -- 172.21.15.188:0/4238090495 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fb5c004c8c0 con 0x7fb5cc0ade90 2024-09-13T17:57:03.681 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.677+0000 7fb5d3bed640 1 -- 172.21.15.188:0/4238090495 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5cc0b2e90 con 0x7fb5cc0ade90 2024-09-13T17:57:03.682 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.679+0000 7fb5be7fc640 1 -- 172.21.15.188:0/4238090495 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fb5c0048070 con 0x7fb5cc0ade90 2024-09-13T17:57:03.748 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.746+0000 7fb5d3bed640 1 -- 172.21.15.188:0/4238090495 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a7264062-0f38-44a0-acc9-796dfac73619", "id": 4} v 0) v1 -- 0x7fb5cc06f000 con 0x7fb5cc0ade90 2024-09-13T17:57:03.754 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.752+0000 7fb5be7fc640 1 -- 172.21.15.188:0/4238090495 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a7264062-0f38-44a0-acc9-796dfac73619", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb5c0036020 con 0x7fb5cc0ade90 2024-09-13T17:57:03.754 INFO:teuthology.orchestra.run.smithi188.stdout:4 2024-09-13T17:57:03.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.753+0000 7fb5d3bed640 1 -- 172.21.15.188:0/4238090495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5cc0ade90 msgr2=0x7fb5cc14e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.753+0000 7fb5d3bed640 1 --2- 172.21.15.188:0/4238090495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5cc0ade90 0x7fb5cc14e070 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb5c002c980 tx=0x7fb5c00046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.753+0000 7fb5d3bed640 1 -- 172.21.15.188:0/4238090495 shutdown_connections 2024-09-13T17:57:03.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.753+0000 7fb5d3bed640 1 --2- 172.21.15.188:0/4238090495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5cc159200 0x7fb5cc15bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.756 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.753+0000 7fb5d3bed640 1 --2- 172.21.15.188:0/4238090495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5cc0b33d0 0x7fb5cc14d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.756 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.753+0000 7fb5d3bed640 1 --2- 172.21.15.188:0/4238090495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5cc0ade90 0x7fb5cc14e070 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.756 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.753+0000 7fb5d3bed640 1 -- 172.21.15.188:0/4238090495 >> 172.21.15.188:0/4238090495 conn(0x7fb5cc069ed0 msgr2=0x7fb5cc0b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:03.756 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.753+0000 7fb5d3bed640 1 -- 172.21.15.188:0/4238090495 shutdown_connections 2024-09-13T17:57:03.756 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.753+0000 7fb5d3bed640 1 -- 172.21.15.188:0/4238090495 wait complete. 2024-09-13T17:57:03.765 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 98e5b234-4ed0-47c0-a6cc-1d63574e67be 5 2024-09-13T17:57:03.863 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.861+0000 7fba6ba9d640 1 Processor -- start 2024-09-13T17:57:03.863 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.861+0000 7fba6ba9d640 1 -- start start 2024-09-13T17:57:03.864 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.862+0000 7fba6ba9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fba641594b0 0x7fba640cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.864 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.862+0000 7fba6ba9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba640ce4d0 0x7fba640ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.864 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.862+0000 7fba6ba9d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba64159890 con 0x7fba640cbfe0 2024-09-13T17:57:03.865 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.862+0000 7fba6ba9d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba640cf0b0 con 0x7fba641594b0 2024-09-13T17:57:03.865 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.862+0000 7fba6ba9d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba640cc450 con 0x7fba640ce4d0 2024-09-13T17:57:03.865 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.862+0000 7fba6a03a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba640ce4d0 0x7fba640ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.865 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.862+0000 7fba6a03a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba640ce4d0 0x7fba640ca210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:52148/0 (socket says 172.21.15.188:52148) 2024-09-13T17:57:03.865 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.862+0000 7fba6a03a640 1 -- 172.21.15.188:0/147776536 learned_addr learned my addr 172.21.15.188:0/147776536 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-09-13T17:57:03.865 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.862+0000 7fba69839640 1 --2- 172.21.15.188:0/147776536 >> v2:172.21.15.111:3300/0 conn(0x7fba641594b0 0x7fba640cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.865 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.863+0000 7fba6a03a640 1 -- 172.21.15.188:0/147776536 >> v2:172.21.15.111:3300/0 conn(0x7fba641594b0 msgr2=0x7fba640cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:03.865 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.863+0000 7fba6a03a640 1 --2- 172.21.15.188:0/147776536 >> v2:172.21.15.111:3300/0 conn(0x7fba641594b0 0x7fba640cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.865 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.863+0000 7fba6a03a640 1 -- 172.21.15.188:0/147776536 >> v1:172.21.15.111:6789/0 conn(0x7fba640cbfe0 legacy=0x7fba640cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.865 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.863+0000 7fba6a03a640 1 -- 172.21.15.188:0/147776536 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba640c9d80 con 0x7fba640ce4d0 2024-09-13T17:57:03.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.863+0000 7fba6a03a640 1 --2- 172.21.15.188:0/147776536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba640ce4d0 0x7fba640ca210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009e80 tx=0x7fba5c02a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20aabbfecf8e37df server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:03.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.863+0000 7fba68837640 1 -- 172.21.15.188:0/147776536 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c031600 con 0x7fba640ce4d0 2024-09-13T17:57:03.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.863+0000 7fba68837640 1 -- 172.21.15.188:0/147776536 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c02d050 con 0x7fba640ce4d0 2024-09-13T17:57:03.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.863+0000 7fba68837640 1 -- 172.21.15.188:0/147776536 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c031980 con 0x7fba640ce4d0 2024-09-13T17:57:03.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.864+0000 7fba6ba9d640 1 -- 172.21.15.188:0/147776536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba640ce4d0 msgr2=0x7fba640ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.864+0000 7fba6ba9d640 1 --2- 172.21.15.188:0/147776536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba640ce4d0 0x7fba640ca210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009e80 tx=0x7fba5c02a750 comp rx=0 tx=0).stop 2024-09-13T17:57:03.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.864+0000 7fba6ba9d640 1 -- 172.21.15.188:0/147776536 shutdown_connections 2024-09-13T17:57:03.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.864+0000 7fba6ba9d640 1 --2- 172.21.15.188:0/147776536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba640ce4d0 0x7fba640ca210 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.864+0000 7fba6ba9d640 1 --2- 172.21.15.188:0/147776536 >> v2:172.21.15.111:3300/0 conn(0x7fba641594b0 0x7fba640cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.864+0000 7fba6ba9d640 1 -- 172.21.15.188:0/147776536 >> 172.21.15.188:0/147776536 conn(0x7fba640989d0 msgr2=0x7fba640b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:03.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.864+0000 7fba6ba9d640 1 -- 172.21.15.188:0/147776536 shutdown_connections 2024-09-13T17:57:03.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.864+0000 7fba6ba9d640 1 -- 172.21.15.188:0/147776536 wait complete. 2024-09-13T17:57:03.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.865+0000 7fba6ba9d640 1 Processor -- start 2024-09-13T17:57:03.868 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.865+0000 7fba6ba9d640 1 -- start start 2024-09-13T17:57:03.870 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.866+0000 7fba6ba9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba640cbfe0 0x7fba64154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.870 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.866+0000 7fba6ba9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba640ce4d0 0x7fba640b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.870 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.866+0000 7fba6ba9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba641594b0 0x7fba6406dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.870 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.866+0000 7fba6ba9d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba640b37e0 con 0x7fba641594b0 2024-09-13T17:57:03.870 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.866+0000 7fba6ba9d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba640b2ca0 con 0x7fba640ce4d0 2024-09-13T17:57:03.870 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.866+0000 7fba6ba9d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba640b2f40 con 0x7fba640cbfe0 2024-09-13T17:57:03.870 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.866+0000 7fba69839640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba640cbfe0 0x7fba64154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.870 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.866+0000 7fba69839640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba640cbfe0 0x7fba64154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:52150/0 (socket says 172.21.15.188:52150) 2024-09-13T17:57:03.870 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.866+0000 7fba69038640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba640ce4d0 0x7fba640b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.870 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.866+0000 7fba69839640 1 -- 172.21.15.188:0/2736488472 learned_addr learned my addr 172.21.15.188:0/2736488472 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-09-13T17:57:03.871 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.866+0000 7fba6a03a640 1 --2- 172.21.15.188:0/2736488472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba641594b0 0x7fba6406dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.871 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.866+0000 7fba69839640 1 -- 172.21.15.188:0/2736488472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba640ce4d0 msgr2=0x7fba640b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.871 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.866+0000 7fba69839640 1 --2- 172.21.15.188:0/2736488472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba640ce4d0 0x7fba640b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.871 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.866+0000 7fba69839640 1 -- 172.21.15.188:0/2736488472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba641594b0 msgr2=0x7fba6406dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.871 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.866+0000 7fba69839640 1 --2- 172.21.15.188:0/2736488472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba641594b0 0x7fba6406dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.871 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.866+0000 7fba69839640 1 -- 172.21.15.188:0/2736488472 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c009b30 con 0x7fba640cbfe0 2024-09-13T17:57:03.871 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.866+0000 7fba69038640 1 --2- 172.21.15.188:0/2736488472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba640ce4d0 0x7fba640b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:03.871 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.867+0000 7fba69839640 1 --2- 172.21.15.188:0/2736488472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba640cbfe0 0x7fba64154240 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fba6000e870 tx=0x7fba6000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:03.871 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.867+0000 7fba5a7fc640 1 -- 172.21.15.188:0/2736488472 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba600094f0 con 0x7fba640cbfe0 2024-09-13T17:57:03.871 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.867+0000 7fba5a7fc640 1 -- 172.21.15.188:0/2736488472 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6000c940 con 0x7fba640cbfe0 2024-09-13T17:57:03.871 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.867+0000 7fba5a7fc640 1 -- 172.21.15.188:0/2736488472 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6000ccf0 con 0x7fba640cbfe0 2024-09-13T17:57:03.871 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.867+0000 7fba6ba9d640 1 -- 172.21.15.188:0/2736488472 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba640b3220 con 0x7fba640cbfe0 2024-09-13T17:57:03.871 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.867+0000 7fba6ba9d640 1 -- 172.21.15.188:0/2736488472 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6415b080 con 0x7fba640cbfe0 2024-09-13T17:57:03.871 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.867+0000 7fba5a7fc640 1 -- 172.21.15.188:0/2736488472 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fba60005c90 con 0x7fba640cbfe0 2024-09-13T17:57:03.872 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.867+0000 7fba5a7fc640 1 -- 172.21.15.188:0/2736488472 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fba60006830 con 0x7fba640cbfe0 2024-09-13T17:57:03.872 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.868+0000 7fba6ba9d640 1 -- 172.21.15.188:0/2736488472 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba640b2530 con 0x7fba640cbfe0 2024-09-13T17:57:03.872 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.870+0000 7fba5a7fc640 1 -- 172.21.15.188:0/2736488472 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fba60021060 con 0x7fba640cbfe0 2024-09-13T17:57:03.938 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.936+0000 7fba6ba9d640 1 -- 172.21.15.188:0/2736488472 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "98e5b234-4ed0-47c0-a6cc-1d63574e67be", "id": 5} v 0) v1 -- 0x7fba64158990 con 0x7fba640cbfe0 2024-09-13T17:57:03.944 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.942+0000 7fba5a7fc640 1 -- 172.21.15.188:0/2736488472 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "98e5b234-4ed0-47c0-a6cc-1d63574e67be", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fba60017070 con 0x7fba640cbfe0 2024-09-13T17:57:03.944 INFO:teuthology.orchestra.run.smithi188.stdout:5 2024-09-13T17:57:03.945 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.943+0000 7fba6ba9d640 1 -- 172.21.15.188:0/2736488472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba640cbfe0 msgr2=0x7fba64154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.945 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.943+0000 7fba6ba9d640 1 --2- 172.21.15.188:0/2736488472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba640cbfe0 0x7fba64154240 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fba6000e870 tx=0x7fba6000ed30 comp rx=0 tx=0).stop 2024-09-13T17:57:03.945 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.943+0000 7fba6ba9d640 1 -- 172.21.15.188:0/2736488472 shutdown_connections 2024-09-13T17:57:03.945 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.943+0000 7fba6ba9d640 1 --2- 172.21.15.188:0/2736488472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba641594b0 0x7fba6406dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.945 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.943+0000 7fba6ba9d640 1 --2- 172.21.15.188:0/2736488472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba640ce4d0 0x7fba640b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.946 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.943+0000 7fba6ba9d640 1 --2- 172.21.15.188:0/2736488472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba640cbfe0 0x7fba64154240 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.946 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.943+0000 7fba6ba9d640 1 -- 172.21.15.188:0/2736488472 >> 172.21.15.188:0/2736488472 conn(0x7fba640989d0 msgr2=0x7fba640ae160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:03.946 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.943+0000 7fba6ba9d640 1 -- 172.21.15.188:0/2736488472 shutdown_connections 2024-09-13T17:57:03.946 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:03.943+0000 7fba6ba9d640 1 -- 172.21.15.188:0/2736488472 wait complete. 2024-09-13T17:57:03.955 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new efddf674-a908-4929-a8b3-0f0de44358ba 6 2024-09-13T17:57:04.053 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.051+0000 7fdbae77d640 1 Processor -- start 2024-09-13T17:57:04.053 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.051+0000 7fdbae77d640 1 -- start start 2024-09-13T17:57:04.054 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.052+0000 7fdbae77d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdba80cdea0 0x7fdba80b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.055 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.052+0000 7fdbae77d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdba80ae490 0x7fdba806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.055 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.052+0000 7fdbae77d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba80ccd80 con 0x7fdba815ca40 2024-09-13T17:57:04.055 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.052+0000 7fdbae77d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba80cd060 con 0x7fdba80cdea0 2024-09-13T17:57:04.055 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.052+0000 7fdbae77d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80cd340 con 0x7fdba80ae490 2024-09-13T17:57:04.055 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.052+0000 7fdbacd1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdba80ae490 0x7fdba806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.055 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.052+0000 7fdbacd1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdba80ae490 0x7fdba806d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:52164/0 (socket says 172.21.15.188:52164) 2024-09-13T17:57:04.055 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.052+0000 7fdbacd1a640 1 -- 172.21.15.188:0/3047466628 learned_addr learned my addr 172.21.15.188:0/3047466628 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-09-13T17:57:04.055 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.052+0000 7fdba77fe640 1 --2- 172.21.15.188:0/3047466628 >> v2:172.21.15.188:3300/0 conn(0x7fdba80cdea0 0x7fdba80b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.055 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.052+0000 7fdba77fe640 1 -- 172.21.15.188:0/3047466628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdba80ae490 msgr2=0x7fdba806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:04.055 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.052+0000 7fdba77fe640 1 --2- 172.21.15.188:0/3047466628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdba80ae490 0x7fdba806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.055 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.052+0000 7fdba77fe640 1 -- 172.21.15.188:0/3047466628 >> v1:172.21.15.188:6789/0 conn(0x7fdba815ca40 legacy=0x7fdba80cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.056 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.052+0000 7fdba77fe640 1 -- 172.21.15.188:0/3047466628 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba80cc550 con 0x7fdba80cdea0 2024-09-13T17:57:04.056 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.052+0000 7fdbacd1a640 1 --2- 172.21.15.188:0/3047466628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdba80ae490 0x7fdba806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:04.056 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.053+0000 7fdba77fe640 1 --2- 172.21.15.188:0/3047466628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdba80cdea0 0x7fdba80b1880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdb900097a0 tx=0x7fdb9002a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=449d8327efa6ae8b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.056 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.053+0000 7fdba6ffd640 1 -- 172.21.15.188:0/3047466628 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9002adb0 con 0x7fdba80cdea0 2024-09-13T17:57:04.056 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.053+0000 7fdba6ffd640 1 -- 172.21.15.188:0/3047466628 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb90032440 con 0x7fdba80cdea0 2024-09-13T17:57:04.056 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.053+0000 7fdba6ffd640 1 -- 172.21.15.188:0/3047466628 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb900327d0 con 0x7fdba80cdea0 2024-09-13T17:57:04.059 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.054+0000 7fdbae77d640 1 -- 172.21.15.188:0/3047466628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdba80cdea0 msgr2=0x7fdba80b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.054+0000 7fdbae77d640 1 --2- 172.21.15.188:0/3047466628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdba80cdea0 0x7fdba80b1880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdb900097a0 tx=0x7fdb9002a060 comp rx=0 tx=0).stop 2024-09-13T17:57:04.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.054+0000 7fdbae77d640 1 -- 172.21.15.188:0/3047466628 shutdown_connections 2024-09-13T17:57:04.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.054+0000 7fdbae77d640 1 --2- 172.21.15.188:0/3047466628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdba80cdea0 0x7fdba80b1880 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.054+0000 7fdbae77d640 1 --2- 172.21.15.188:0/3047466628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdba80ae490 0x7fdba806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.054+0000 7fdbae77d640 1 -- 172.21.15.188:0/3047466628 >> 172.21.15.188:0/3047466628 conn(0x7fdba80b0d60 msgr2=0x7fdba80b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:04.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.054+0000 7fdbae77d640 1 -- 172.21.15.188:0/3047466628 shutdown_connections 2024-09-13T17:57:04.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.054+0000 7fdbae77d640 1 -- 172.21.15.188:0/3047466628 wait complete. 2024-09-13T17:57:04.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.055+0000 7fdbae77d640 1 Processor -- start 2024-09-13T17:57:04.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.055+0000 7fdbae77d640 1 -- start start 2024-09-13T17:57:04.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdbae77d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdba80ae490 0x7fdba80c3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdbae77d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdba80cdea0 0x7fdba8153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdbae77d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdba815ca40 0x7fdba80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdbae77d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80b5030 con 0x7fdba80cdea0 2024-09-13T17:57:04.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdbae77d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80b21e0 con 0x7fdba80ae490 2024-09-13T17:57:04.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdbae77d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80b24c0 con 0x7fdba815ca40 2024-09-13T17:57:04.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdba7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdba80ae490 0x7fdba80c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdbacd1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdba815ca40 0x7fdba80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdba7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdba80ae490 0x7fdba80c3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:41256/0 (socket says 172.21.15.188:41256) 2024-09-13T17:57:04.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdbacd1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdba815ca40 0x7fdba80b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:52180/0 (socket says 172.21.15.188:52180) 2024-09-13T17:57:04.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdba7fff640 1 -- 172.21.15.188:0/3806411007 learned_addr learned my addr 172.21.15.188:0/3806411007 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-09-13T17:57:04.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdba77fe640 1 --2- 172.21.15.188:0/3806411007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdba80cdea0 0x7fdba8153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdbacd1a640 1 -- 172.21.15.188:0/3806411007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdba80ae490 msgr2=0x7fdba80c3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdbacd1a640 1 --2- 172.21.15.188:0/3806411007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdba80ae490 0x7fdba80c3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdbacd1a640 1 -- 172.21.15.188:0/3806411007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdba80cdea0 msgr2=0x7fdba8153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.062 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdbacd1a640 1 --2- 172.21.15.188:0/3806411007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdba80cdea0 0x7fdba8153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.062 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdbacd1a640 1 -- 172.21.15.188:0/3806411007 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba0009b30 con 0x7fdba815ca40 2024-09-13T17:57:04.062 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdba7fff640 1 --2- 172.21.15.188:0/3806411007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdba80ae490 0x7fdba80c3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:04.062 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdba77fe640 1 --2- 172.21.15.188:0/3806411007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdba80cdea0 0x7fdba8153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:04.062 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.056+0000 7fdbacd1a640 1 --2- 172.21.15.188:0/3806411007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdba815ca40 0x7fdba80b4670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdba000efc0 tx=0x7fdba000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.062 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.057+0000 7fdba4ff9640 1 -- 172.21.15.188:0/3806411007 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0018070 con 0x7fdba815ca40 2024-09-13T17:57:04.062 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.057+0000 7fdba4ff9640 1 -- 172.21.15.188:0/3806411007 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba000f040 con 0x7fdba815ca40 2024-09-13T17:57:04.062 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.057+0000 7fdba4ff9640 1 -- 172.21.15.188:0/3806411007 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba000cd40 con 0x7fdba815ca40 2024-09-13T17:57:04.062 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.057+0000 7fdbae77d640 1 -- 172.21.15.188:0/3806411007 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb90009400 con 0x7fdba815ca40 2024-09-13T17:57:04.062 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.057+0000 7fdbae77d640 1 -- 172.21.15.188:0/3806411007 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba80b2a80 con 0x7fdba815ca40 2024-09-13T17:57:04.062 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.057+0000 7fdba4ff9640 1 -- 172.21.15.188:0/3806411007 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fdba0003c90 con 0x7fdba815ca40 2024-09-13T17:57:04.062 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.057+0000 7fdba4ff9640 1 -- 172.21.15.188:0/3806411007 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fdba0014a70 con 0x7fdba815ca40 2024-09-13T17:57:04.062 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.058+0000 7fdbae77d640 1 -- 172.21.15.188:0/3806411007 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba806d1b0 con 0x7fdba815ca40 2024-09-13T17:57:04.063 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.060+0000 7fdba4ff9640 1 -- 172.21.15.188:0/3806411007 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fdba0004840 con 0x7fdba815ca40 2024-09-13T17:57:04.130 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.128+0000 7fdbae77d640 1 -- 172.21.15.188:0/3806411007 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "efddf674-a908-4929-a8b3-0f0de44358ba", "id": 6} v 0) v1 -- 0x7fdba806acc0 con 0x7fdba815ca40 2024-09-13T17:57:04.138 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.136+0000 7fdba4ff9640 1 -- 172.21.15.188:0/3806411007 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "efddf674-a908-4929-a8b3-0f0de44358ba", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdba002b0b0 con 0x7fdba815ca40 2024-09-13T17:57:04.138 INFO:teuthology.orchestra.run.smithi188.stdout:6 2024-09-13T17:57:04.139 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.137+0000 7fdbae77d640 1 -- 172.21.15.188:0/3806411007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdba815ca40 msgr2=0x7fdba80b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.139 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.137+0000 7fdbae77d640 1 --2- 172.21.15.188:0/3806411007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdba815ca40 0x7fdba80b4670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdba000efc0 tx=0x7fdba000c490 comp rx=0 tx=0).stop 2024-09-13T17:57:04.140 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.137+0000 7fdbae77d640 1 -- 172.21.15.188:0/3806411007 shutdown_connections 2024-09-13T17:57:04.140 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.137+0000 7fdbae77d640 1 --2- 172.21.15.188:0/3806411007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdba815ca40 0x7fdba80b4670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.140 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.137+0000 7fdbae77d640 1 --2- 172.21.15.188:0/3806411007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdba80cdea0 0x7fdba8153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.140 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.137+0000 7fdbae77d640 1 --2- 172.21.15.188:0/3806411007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdba80ae490 0x7fdba80c3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.140 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.137+0000 7fdbae77d640 1 -- 172.21.15.188:0/3806411007 >> 172.21.15.188:0/3806411007 conn(0x7fdba80b0d60 msgr2=0x7fdba80ad650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:04.140 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.137+0000 7fdbae77d640 1 -- 172.21.15.188:0/3806411007 shutdown_connections 2024-09-13T17:57:04.140 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.137+0000 7fdbae77d640 1 -- 172.21.15.188:0/3806411007 wait complete. 2024-09-13T17:57:04.149 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 82233e9f-d2fb-438b-8800-785cb1f3c44a 7 2024-09-13T17:57:04.250 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.244+0000 7fa3f2d2c640 1 Processor -- start 2024-09-13T17:57:04.250 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.244+0000 7fa3f2d2c640 1 -- start start 2024-09-13T17:57:04.250 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.245+0000 7fa3f2d2c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa3ec1652d0 0x7fa3ec1676c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.250 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.245+0000 7fa3f2d2c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ec06b6c0 con 0x7fa3ec0d22d0 2024-09-13T17:57:04.251 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.245+0000 7fa3f2d2c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ec0ce500 con 0x7fa3ec15fbc0 2024-09-13T17:57:04.251 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.245+0000 7fa3f2d2c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ec0cdc60 con 0x7fa3ec1652d0 2024-09-13T17:57:04.251 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.245+0000 7fa3f12c9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa3ec1652d0 0x7fa3ec1676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.251 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.245+0000 7fa3f12c9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa3ec1652d0 0x7fa3ec1676c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:41260/0 (socket says 172.21.15.188:41260) 2024-09-13T17:57:04.251 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.245+0000 7fa3f12c9640 1 -- 172.21.15.188:0/2110250 learned_addr learned my addr 172.21.15.188:0/2110250 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-09-13T17:57:04.251 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.246+0000 7fa3f12c9640 1 -- 172.21.15.188:0/2110250 >> v1:172.21.15.188:6789/0 conn(0x7fa3ec15fbc0 legacy=0x7fa3ec0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.251 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.246+0000 7fa3f12c9640 1 -- 172.21.15.188:0/2110250 >> v1:172.21.15.111:6789/0 conn(0x7fa3ec0d22d0 legacy=0x7fa3ec0d2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.251 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.246+0000 7fa3f12c9640 1 -- 172.21.15.188:0/2110250 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec0cdf40 con 0x7fa3ec1652d0 2024-09-13T17:57:04.251 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.246+0000 7fa3f12c9640 1 --2- 172.21.15.188:0/2110250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3ec1652d0 0x7fa3ec1676c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009870 tx=0x7fa3d402a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6d67a419a92aac3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.251 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.246+0000 7fa3eb7fe640 1 -- 172.21.15.188:0/2110250 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4032640 con 0x7fa3ec1652d0 2024-09-13T17:57:04.251 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.247+0000 7fa3eb7fe640 1 -- 172.21.15.188:0/2110250 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d40327a0 con 0x7fa3ec1652d0 2024-09-13T17:57:04.251 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.247+0000 7fa3eb7fe640 1 -- 172.21.15.188:0/2110250 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4032b30 con 0x7fa3ec1652d0 2024-09-13T17:57:04.251 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.247+0000 7fa3f2d2c640 1 -- 172.21.15.188:0/2110250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3ec1652d0 msgr2=0x7fa3ec1676c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.252 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.247+0000 7fa3f2d2c640 1 --2- 172.21.15.188:0/2110250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3ec1652d0 0x7fa3ec1676c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009870 tx=0x7fa3d402a300 comp rx=0 tx=0).stop 2024-09-13T17:57:04.252 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.247+0000 7fa3f2d2c640 1 -- 172.21.15.188:0/2110250 shutdown_connections 2024-09-13T17:57:04.252 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.247+0000 7fa3f2d2c640 1 --2- 172.21.15.188:0/2110250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3ec1652d0 0x7fa3ec1676c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.252 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.247+0000 7fa3f2d2c640 1 -- 172.21.15.188:0/2110250 >> 172.21.15.188:0/2110250 conn(0x7fa3ec09b830 msgr2=0x7fa3ec089f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:04.252 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.247+0000 7fa3f2d2c640 1 -- 172.21.15.188:0/2110250 shutdown_connections 2024-09-13T17:57:04.252 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.247+0000 7fa3f2d2c640 1 -- 172.21.15.188:0/2110250 wait complete. 2024-09-13T17:57:04.252 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.248+0000 7fa3f2d2c640 1 Processor -- start 2024-09-13T17:57:04.252 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.248+0000 7fa3f2d2c640 1 -- start start 2024-09-13T17:57:04.254 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.249+0000 7fa3f2d2c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3ec0d22d0 0x7fa3ec1535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.254 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.249+0000 7fa3f2d2c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3ec15fbc0 0x7fa3ec0b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.254 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.249+0000 7fa3f2d2c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3ec1652d0 0x7fa3ec0b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.254 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.249+0000 7fa3f2d2c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec15aee0 con 0x7fa3ec0d22d0 2024-09-13T17:57:04.254 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.249+0000 7fa3f2d2c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec15b1c0 con 0x7fa3ec1652d0 2024-09-13T17:57:04.254 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.249+0000 7fa3f2d2c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec15b4a0 con 0x7fa3ec15fbc0 2024-09-13T17:57:04.255 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.249+0000 7fa3f12c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3ec1652d0 0x7fa3ec0b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.255 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.249+0000 7fa3ebfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3ec15fbc0 0x7fa3ec0b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.255 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.249+0000 7fa3f12c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3ec1652d0 0x7fa3ec0b52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:41262/0 (socket says 172.21.15.188:41262) 2024-09-13T17:57:04.255 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.249+0000 7fa3f12c9640 1 -- 172.21.15.188:0/2857686927 learned_addr learned my addr 172.21.15.188:0/2857686927 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-09-13T17:57:04.255 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.249+0000 7fa3f0ac8640 1 --2- 172.21.15.188:0/2857686927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3ec0d22d0 0x7fa3ec1535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.255 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.250+0000 7fa3f12c9640 1 -- 172.21.15.188:0/2857686927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3ec15fbc0 msgr2=0x7fa3ec0b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.255 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.250+0000 7fa3f12c9640 1 --2- 172.21.15.188:0/2857686927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3ec15fbc0 0x7fa3ec0b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.255 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.250+0000 7fa3f12c9640 1 -- 172.21.15.188:0/2857686927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3ec0d22d0 msgr2=0x7fa3ec1535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.255 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.250+0000 7fa3f12c9640 1 --2- 172.21.15.188:0/2857686927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3ec0d22d0 0x7fa3ec1535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.255 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.250+0000 7fa3f12c9640 1 -- 172.21.15.188:0/2857686927 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d40094d0 con 0x7fa3ec1652d0 2024-09-13T17:57:04.255 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.250+0000 7fa3f12c9640 1 --2- 172.21.15.188:0/2857686927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3ec1652d0 0x7fa3ec0b52f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009870 tx=0x7fa3d40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.255 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.250+0000 7fa3e97fa640 1 -- 172.21.15.188:0/2857686927 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4030b80 con 0x7fa3ec1652d0 2024-09-13T17:57:04.255 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.250+0000 7fa3e97fa640 1 -- 172.21.15.188:0/2857686927 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d4030d00 con 0x7fa3ec1652d0 2024-09-13T17:57:04.255 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.250+0000 7fa3f2d2c640 1 -- 172.21.15.188:0/2857686927 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ec15b780 con 0x7fa3ec1652d0 2024-09-13T17:57:04.256 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.250+0000 7fa3e97fa640 1 -- 172.21.15.188:0/2857686927 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d403d660 con 0x7fa3ec1652d0 2024-09-13T17:57:04.256 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.250+0000 7fa3f2d2c640 1 -- 172.21.15.188:0/2857686927 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ec0b3870 con 0x7fa3ec1652d0 2024-09-13T17:57:04.256 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.250+0000 7fa3e97fa640 1 -- 172.21.15.188:0/2857686927 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fa3d4039040 con 0x7fa3ec1652d0 2024-09-13T17:57:04.256 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.250+0000 7fa3e97fa640 1 -- 172.21.15.188:0/2857686927 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fa3d4046c40 con 0x7fa3ec1652d0 2024-09-13T17:57:04.256 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.251+0000 7fa3f2d2c640 1 -- 172.21.15.188:0/2857686927 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ec15bca0 con 0x7fa3ec1652d0 2024-09-13T17:57:04.256 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.255+0000 7fa3e97fa640 1 -- 172.21.15.188:0/2857686927 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fa3ec15bca0 con 0x7fa3ec1652d0 2024-09-13T17:57:04.324 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.322+0000 7fa3f2d2c640 1 -- 172.21.15.188:0/2857686927 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "82233e9f-d2fb-438b-8800-785cb1f3c44a", "id": 7} v 0) v1 -- 0x7fa3ec0b3a00 con 0x7fa3ec1652d0 2024-09-13T17:57:04.332 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.330+0000 7fa3e97fa640 1 -- 172.21.15.188:0/2857686927 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "82233e9f-d2fb-438b-8800-785cb1f3c44a", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa3d404b060 con 0x7fa3ec1652d0 2024-09-13T17:57:04.332 INFO:teuthology.orchestra.run.smithi188.stdout:7 2024-09-13T17:57:04.333 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.331+0000 7fa3f2d2c640 1 -- 172.21.15.188:0/2857686927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3ec1652d0 msgr2=0x7fa3ec0b52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.333 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.331+0000 7fa3f2d2c640 1 --2- 172.21.15.188:0/2857686927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3ec1652d0 0x7fa3ec0b52f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009870 tx=0x7fa3d40046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.334 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.331+0000 7fa3f2d2c640 1 -- 172.21.15.188:0/2857686927 shutdown_connections 2024-09-13T17:57:04.334 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.331+0000 7fa3f2d2c640 1 --2- 172.21.15.188:0/2857686927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3ec1652d0 0x7fa3ec0b52f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.334 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.331+0000 7fa3f2d2c640 1 --2- 172.21.15.188:0/2857686927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3ec15fbc0 0x7fa3ec0b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.334 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.331+0000 7fa3f2d2c640 1 --2- 172.21.15.188:0/2857686927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3ec0d22d0 0x7fa3ec1535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.334 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.331+0000 7fa3f2d2c640 1 -- 172.21.15.188:0/2857686927 >> 172.21.15.188:0/2857686927 conn(0x7fa3ec09b830 msgr2=0x7fa3ec15e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:04.334 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.331+0000 7fa3f2d2c640 1 -- 172.21.15.188:0/2857686927 shutdown_connections 2024-09-13T17:57:04.334 INFO:teuthology.orchestra.run.smithi188.stderr:2024-09-13T17:57:04.332+0000 7fa3f2d2c640 1 -- 172.21.15.188:0/2857686927 wait complete. 2024-09-13T17:57:04.343 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T17:57:04.344 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T17:57:04.346 INFO:tasks.ceph.osd.0:Started 2024-09-13T17:57:04.346 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T17:57:04.346 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T17:57:04.347 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:04.345+0000 7f6a9ba15040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:57:04.348 INFO:tasks.ceph.osd.1:Started 2024-09-13T17:57:04.348 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T17:57:04.348 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T17:57:04.350 INFO:tasks.ceph.osd.2:Started 2024-09-13T17:57:04.350 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T17:57:04.350 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T17:57:04.352 INFO:tasks.ceph.osd.3:Started 2024-09-13T17:57:04.352 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-13T17:57:04.352 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-13T17:57:04.354 INFO:tasks.ceph.osd.4:Started 2024-09-13T17:57:04.354 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T17:57:04.355 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T17:57:04.356 INFO:tasks.ceph.osd.5:Started 2024-09-13T17:57:04.357 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-13T17:57:04.357 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-13T17:57:04.359 INFO:tasks.ceph.osd.6:Started 2024-09-13T17:57:04.359 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-13T17:57:04.359 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-13T17:57:04.361 INFO:tasks.ceph.osd.7:Started 2024-09-13T17:57:04.361 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:04.432 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:04.430+0000 7f6a9ba15040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:57:04.437 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:04.435+0000 7f8156532040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:57:04.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.475+0000 7f4f44048640 1 Processor -- start 2024-09-13T17:57:04.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.475+0000 7f4f44048640 1 -- start start 2024-09-13T17:57:04.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.476+0000 7f4f44048640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4f3c15a0d0 0x7f4f3c0b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.476+0000 7f4f44048640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4f3c0b3630 0x7f4f3c0ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.476+0000 7f4f44048640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f3c0989d0 con 0x7f4f3c15a4b0 2024-09-13T17:57:04.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.476+0000 7f4f44048640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f3c098ee0 con 0x7f4f3c15a0d0 2024-09-13T17:57:04.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.476+0000 7f4f44048640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f3c071690 con 0x7f4f3c0b3630 2024-09-13T17:57:04.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.476+0000 7f4f425e5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4f3c0b3630 0x7f4f3c0ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.476+0000 7f4f425e5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4f3c0b3630 0x7f4f3c0ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:58712/0 (socket says 172.21.15.111:58712) 2024-09-13T17:57:04.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.476+0000 7f4f425e5640 1 -- 172.21.15.111:0/2822381947 learned_addr learned my addr 172.21.15.111:0/2822381947 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:04.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.476+0000 7f4f41de4640 1 --2- 172.21.15.111:0/2822381947 >> v2:172.21.15.111:3300/0 conn(0x7f4f3c15a0d0 0x7f4f3c0b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.477+0000 7f4f425e5640 1 -- 172.21.15.111:0/2822381947 >> v2:172.21.15.111:3300/0 conn(0x7f4f3c15a0d0 msgr2=0x7f4f3c0b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:04.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.477+0000 7f4f425e5640 1 --2- 172.21.15.111:0/2822381947 >> v2:172.21.15.111:3300/0 conn(0x7f4f3c15a0d0 0x7f4f3c0b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.477+0000 7f4f425e5640 1 -- 172.21.15.111:0/2822381947 >> v1:172.21.15.188:6789/0 conn(0x7f4f3c15a4b0 legacy=0x7f4f3c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.477+0000 7f4f425e5640 1 -- 172.21.15.111:0/2822381947 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3c0b24a0 con 0x7f4f3c0b3630 2024-09-13T17:57:04.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.477+0000 7f4f425e5640 1 --2- 172.21.15.111:0/2822381947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f3c0b3630 0x7f4f3c0ad860 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0aee90 tx=0x7f4f2c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=533a7e6c7d9b6cd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.478+0000 7f4f40de2640 1 -- 172.21.15.111:0/2822381947 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c032440 con 0x7f4f3c0b3630 2024-09-13T17:57:04.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.478+0000 7f4f40de2640 1 -- 172.21.15.111:0/2822381947 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2c0325a0 con 0x7f4f3c0b3630 2024-09-13T17:57:04.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.478+0000 7f4f40de2640 1 -- 172.21.15.111:0/2822381947 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c032930 con 0x7f4f3c0b3630 2024-09-13T17:57:04.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.478+0000 7f4f44048640 1 -- 172.21.15.111:0/2822381947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f3c0b3630 msgr2=0x7f4f3c0ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.478+0000 7f4f44048640 1 --2- 172.21.15.111:0/2822381947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f3c0b3630 0x7f4f3c0ad860 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0aee90 tx=0x7f4f2c02a270 comp rx=0 tx=0).stop 2024-09-13T17:57:04.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.478+0000 7f4f44048640 1 -- 172.21.15.111:0/2822381947 shutdown_connections 2024-09-13T17:57:04.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.478+0000 7f4f44048640 1 --2- 172.21.15.111:0/2822381947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f3c0b3630 0x7f4f3c0ad860 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.478+0000 7f4f44048640 1 --2- 172.21.15.111:0/2822381947 >> v2:172.21.15.111:3300/0 conn(0x7f4f3c15a0d0 0x7f4f3c0b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.478+0000 7f4f44048640 1 -- 172.21.15.111:0/2822381947 >> 172.21.15.111:0/2822381947 conn(0x7f4f3c0b49b0 msgr2=0x7f4f3c0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:04.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.478+0000 7f4f44048640 1 -- 172.21.15.111:0/2822381947 shutdown_connections 2024-09-13T17:57:04.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.479+0000 7f4f44048640 1 -- 172.21.15.111:0/2822381947 wait complete. 2024-09-13T17:57:04.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.480+0000 7f4f44048640 1 Processor -- start 2024-09-13T17:57:04.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.480+0000 7f4f44048640 1 -- start start 2024-09-13T17:57:04.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f44048640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f3c0b3630 0x7f4f3c14d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f44048640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f3c15a0d0 0x7f4f3c146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f44048640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f3c15a4b0 0x7f4f3c149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f44048640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c14fea0 con 0x7f4f3c15a4b0 2024-09-13T17:57:04.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f44048640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c146660 con 0x7f4f3c0b3630 2024-09-13T17:57:04.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f44048640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c153330 con 0x7f4f3c15a0d0 2024-09-13T17:57:04.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f425e5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f3c15a4b0 0x7f4f3c149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f425e5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f3c15a4b0 0x7f4f3c149850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54284/0 (socket says 172.21.15.111:54284) 2024-09-13T17:57:04.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f425e5640 1 -- 172.21.15.111:0/3002093894 learned_addr learned my addr 172.21.15.111:0/3002093894 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:04.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f41de4640 1 --2- 172.21.15.111:0/3002093894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f3c0b3630 0x7f4f3c14d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f415e3640 1 --2- 172.21.15.111:0/3002093894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f3c15a0d0 0x7f4f3c146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f425e5640 1 -- 172.21.15.111:0/3002093894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f3c15a0d0 msgr2=0x7f4f3c146120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f425e5640 1 --2- 172.21.15.111:0/3002093894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f3c15a0d0 0x7f4f3c146120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f425e5640 1 -- 172.21.15.111:0/3002093894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f3c0b3630 msgr2=0x7f4f3c14d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f425e5640 1 --2- 172.21.15.111:0/3002093894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f3c0b3630 0x7f4f3c14d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f425e5640 1 -- 172.21.15.111:0/3002093894 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2c0094d0 con 0x7f4f3c15a4b0 2024-09-13T17:57:04.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f41de4640 1 --2- 172.21.15.111:0/3002093894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f3c0b3630 0x7f4f3c14d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:04.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f425e5640 1 --2- 172.21.15.111:0/3002093894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f3c15a4b0 0x7f4f3c149850 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c02a2b0 tx=0x7f4f2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f3a7fc640 1 -- 172.21.15.111:0/3002093894 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c03d830 con 0x7f4f3c15a4b0 2024-09-13T17:57:04.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f3a7fc640 1 -- 172.21.15.111:0/3002093894 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2c03d9b0 con 0x7f4f3c15a4b0 2024-09-13T17:57:04.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f3a7fc640 1 -- 172.21.15.111:0/3002093894 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c03dd60 con 0x7f4f3c15a4b0 2024-09-13T17:57:04.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.481+0000 7f4f44048640 1 -- 172.21.15.111:0/3002093894 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3c153610 con 0x7f4f3c15a4b0 2024-09-13T17:57:04.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.482+0000 7f4f44048640 1 -- 172.21.15.111:0/3002093894 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3c14f7e0 con 0x7f4f3c15a4b0 2024-09-13T17:57:04.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.482+0000 7f4f3a7fc640 1 -- 172.21.15.111:0/3002093894 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f4f2c039040 con 0x7f4f3c15a4b0 2024-09-13T17:57:04.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.482+0000 7f4f3a7fc640 1 -- 172.21.15.111:0/3002093894 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4f2c03bac0 con 0x7f4f3c15a4b0 2024-09-13T17:57:04.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.482+0000 7f4f44048640 1 -- 172.21.15.111:0/3002093894 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f3c14f640 con 0x7f4f3c15a4b0 2024-09-13T17:57:04.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.485+0000 7f4f3a7fc640 1 -- 172.21.15.111:0/3002093894 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f4f3c14f640 con 0x7f4f3c15a4b0 2024-09-13T17:57:04.491 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:04.490+0000 7f8156532040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:57:04.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.542+0000 7f4f44048640 1 -- 172.21.15.111:0/3002093894 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f3c14ffe0 con 0x7f4f3c15a4b0 2024-09-13T17:57:04.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.542+0000 7f4f3a7fc640 1 -- 172.21.15.111:0/3002093894 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4f3c14ffe0 con 0x7f4f3c15a4b0 2024-09-13T17:57:04.544 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:04.545 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":10,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:04.324664+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:04.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.543+0000 7f4f44048640 1 -- 172.21.15.111:0/3002093894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f3c15a4b0 msgr2=0x7f4f3c149850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.543+0000 7f4f44048640 1 --2- 172.21.15.111:0/3002093894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f3c15a4b0 0x7f4f3c149850 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c02a2b0 tx=0x7f4f2c0046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.543+0000 7f4f44048640 1 -- 172.21.15.111:0/3002093894 shutdown_connections 2024-09-13T17:57:04.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.543+0000 7f4f44048640 1 --2- 172.21.15.111:0/3002093894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f3c15a4b0 0x7f4f3c149850 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.543+0000 7f4f44048640 1 --2- 172.21.15.111:0/3002093894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f3c15a0d0 0x7f4f3c146120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.543+0000 7f4f44048640 1 --2- 172.21.15.111:0/3002093894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f3c0b3630 0x7f4f3c14d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.543+0000 7f4f44048640 1 -- 172.21.15.111:0/3002093894 >> 172.21.15.111:0/3002093894 conn(0x7f4f3c0b49b0 msgr2=0x7f4f3c0b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:04.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.544+0000 7f4f44048640 1 -- 172.21.15.111:0/3002093894 shutdown_connections 2024-09-13T17:57:04.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:04.544+0000 7f4f44048640 1 -- 172.21.15.111:0/3002093894 wait complete. 2024-09-13T17:57:04.555 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T17:57:04.556 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T17:57:04.806 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:04.805+0000 7f6a9ba15040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T17:57:04.854 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:04.852+0000 7f8156532040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T17:57:04.890 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:04.888+0000 7f6a9ba15040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:57:04.937 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:04.936+0000 7f8156532040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:57:04.952 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-09-13T17:57:04.950+0000 7f1e854dc640 -1 Falling back to public interface 2024-09-13T17:57:04.959 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-09-13T17:57:04.957+0000 7f6d9f20f640 -1 Falling back to public interface 2024-09-13T17:57:04.961 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-09-13T17:57:04.959+0000 7f0730db2640 -1 Falling back to public interface 2024-09-13T17:57:04.969 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-09-13T17:57:04.967+0000 7f80eca0f640 -1 Falling back to public interface 2024-09-13T17:57:04.970 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-09-13T17:57:04.968+0000 7f905efdb640 -1 Falling back to public interface 2024-09-13T17:57:04.974 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-09-13T17:57:04.972+0000 7f194f729640 -1 Falling back to public interface 2024-09-13T17:57:04.980 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-09-13T17:57:04.978+0000 7f8e52adc640 -1 Falling back to public interface 2024-09-13T17:57:04.984 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-09-13T17:57:04.982+0000 7fb4cca87640 -1 Falling back to public interface 2024-09-13T17:57:04.998 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:04.996+0000 7f6a9ba15040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:57:05.044 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:05.042+0000 7f6a9ba15040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:57:05.045 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:05.043+0000 7f8156532040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:57:05.090 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:05.088+0000 7f8156532040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:57:05.152 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:05.150+0000 7f6a9ba15040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T17:57:05.199 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:05.198+0000 7f8156532040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T17:57:05.204 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:05.202+0000 7f6a9ba15040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:57:05.249 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:05.247+0000 7f6a9ba15040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:57:05.253 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:05.251+0000 7f8156532040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:57:05.299 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:05.298+0000 7f8156532040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:57:05.360 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:05.359+0000 7f6a9ba15040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:57:05.361 DEBUG:teuthology.orchestra.run.smithi111:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:05.408 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:05.406+0000 7f8156532040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:57:05.412 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:05.410+0000 7f6a9ba15040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:57:05.451 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.449+0000 7f92ac31f640 1 Processor -- start 2024-09-13T17:57:05.451 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.449+0000 7f92ac31f640 1 -- start start 2024-09-13T17:57:05.452 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.450+0000 7f92ac31f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f92a40af5b0 0x7f92a40afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:05.452 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.450+0000 7f92ac31f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f92a40ce8c0 0x7f92a40ced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:05.452 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.450+0000 7f92ac31f640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a415de50 con 0x7f92a415da70 2024-09-13T17:57:05.452 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.450+0000 7f92ac31f640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a40cf2b0 con 0x7f92a40af5b0 2024-09-13T17:57:05.453 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.450+0000 7f92ac31f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a4099dd0 con 0x7f92a40ce8c0 2024-09-13T17:57:05.453 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.450+0000 7f92a98ba640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f92a40af5b0 0x7f92a40afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:05.453 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.450+0000 7f92aa0bb640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f92a415da70 0x7f92a40996c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49312/0 (socket says 172.21.15.111:49312) 2024-09-13T17:57:05.453 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.450+0000 7f92aa0bb640 1 -- 172.21.15.111:0/2272260202 learned_addr learned my addr 172.21.15.111:0/2272260202 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:05.453 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.450+0000 7f92aa8bc640 1 --2- 172.21.15.111:0/2272260202 >> v2:172.21.15.188:3300/0 conn(0x7f92a40ce8c0 0x7f92a40ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:05.453 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.450+0000 7f92a98ba640 1 -- 172.21.15.111:0/2272260202 >> v2:172.21.15.188:3300/0 conn(0x7f92a40ce8c0 msgr2=0x7f92a40ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:05.453 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.450+0000 7f92a98ba640 1 --2- 172.21.15.111:0/2272260202 >> v2:172.21.15.188:3300/0 conn(0x7f92a40ce8c0 0x7f92a40ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:05.453 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.450+0000 7f92a98ba640 1 -- 172.21.15.111:0/2272260202 >> v1:172.21.15.111:6789/0 conn(0x7f92a415da70 legacy=0x7f92a40996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:05.453 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.450+0000 7f92a90b9640 1 -- 172.21.15.111:0/2272260202 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040820077 0 0) 0x7f92a415de50 con 0x7f92a415da70 2024-09-13T17:57:05.453 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.450+0000 7f92a98ba640 1 -- 172.21.15.111:0/2272260202 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a40cdf50 con 0x7f92a40af5b0 2024-09-13T17:57:05.453 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.450+0000 7f92aa8bc640 1 --2- 172.21.15.111:0/2272260202 >> v2:172.21.15.188:3300/0 conn(0x7f92a40ce8c0 0x7f92a40ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:05.455 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.451+0000 7f92a98ba640 1 --2- 172.21.15.111:0/2272260202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92a40af5b0 0x7f92a40afa90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f92a40b2e30 tx=0x7f929402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15a53b9dcf7d969 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:05.455 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.451+0000 7f92a90b9640 1 -- 172.21.15.111:0/2272260202 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294032440 con 0x7f92a40af5b0 2024-09-13T17:57:05.455 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.451+0000 7f92a90b9640 1 -- 172.21.15.111:0/2272260202 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92940325a0 con 0x7f92a40af5b0 2024-09-13T17:57:05.455 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.451+0000 7f92a90b9640 1 -- 172.21.15.111:0/2272260202 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294032930 con 0x7f92a40af5b0 2024-09-13T17:57:05.455 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.452+0000 7f92ac31f640 1 -- 172.21.15.111:0/2272260202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92a40af5b0 msgr2=0x7f92a40afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:05.455 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.452+0000 7f92ac31f640 1 --2- 172.21.15.111:0/2272260202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92a40af5b0 0x7f92a40afa90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f92a40b2e30 tx=0x7f929402a1a0 comp rx=0 tx=0).stop 2024-09-13T17:57:05.455 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.452+0000 7f92ac31f640 1 -- 172.21.15.111:0/2272260202 shutdown_connections 2024-09-13T17:57:05.455 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.452+0000 7f92ac31f640 1 --2- 172.21.15.111:0/2272260202 >> v2:172.21.15.188:3300/0 conn(0x7f92a40ce8c0 0x7f92a40ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:05.456 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.452+0000 7f92ac31f640 1 --2- 172.21.15.111:0/2272260202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92a40af5b0 0x7f92a40afa90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:05.456 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.452+0000 7f92ac31f640 1 -- 172.21.15.111:0/2272260202 >> 172.21.15.111:0/2272260202 conn(0x7f92a40d02e0 msgr2=0x7f92a40d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:05.456 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.452+0000 7f92ac31f640 1 -- 172.21.15.111:0/2272260202 shutdown_connections 2024-09-13T17:57:05.456 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.452+0000 7f92ac31f640 1 -- 172.21.15.111:0/2272260202 wait complete. 2024-09-13T17:57:05.456 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.453+0000 7f92ac31f640 1 Processor -- start 2024-09-13T17:57:05.456 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.453+0000 7f92ac31f640 1 -- start start 2024-09-13T17:57:05.458 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f92ac31f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92a40af5b0 0x7f92a4155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:05.458 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f92ac31f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a40ce8c0 0x7f92a41e4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:05.458 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f92ac31f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92a415da70 0x7f92a41e5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:05.458 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f92ac31f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a41e3e20 con 0x7f92a415da70 2024-09-13T17:57:05.458 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f92ac31f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40ae7d0 con 0x7f92a40ce8c0 2024-09-13T17:57:05.458 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f92ac31f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a41e4070 con 0x7f92a40af5b0 2024-09-13T17:57:05.458 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f92aa8bc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92a415da70 0x7f92a41e5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:05.459 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f92aa8bc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92a415da70 0x7f92a41e5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54302/0 (socket says 172.21.15.111:54302) 2024-09-13T17:57:05.459 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f92aa8bc640 1 -- 172.21.15.111:0/3021239 learned_addr learned my addr 172.21.15.111:0/3021239 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:05.459 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f92a98ba640 1 --2- 172.21.15.111:0/3021239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a40ce8c0 0x7f92a41e4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:05.459 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f92aa0bb640 1 --2- 172.21.15.111:0/3021239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92a40af5b0 0x7f92a4155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:05.459 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f92aa8bc640 1 -- 172.21.15.111:0/3021239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92a40af5b0 msgr2=0x7f92a4155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:05.459 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f92aa8bc640 1 --2- 172.21.15.111:0/3021239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92a40af5b0 0x7f92a4155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:05.459 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f92aa8bc640 1 -- 172.21.15.111:0/3021239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a40ce8c0 msgr2=0x7f92a41e4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:05.459 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f92aa8bc640 1 --2- 172.21.15.111:0/3021239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a40ce8c0 0x7f92a41e4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:05.459 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f92aa8bc640 1 -- 172.21.15.111:0/3021239 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9294009400 con 0x7f92a415da70 2024-09-13T17:57:05.460 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f92aa8bc640 1 --2- 172.21.15.111:0/3021239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92a415da70 0x7f92a41e5480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f929c007df0 tx=0x7f929c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:05.460 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.454+0000 7f929affd640 1 -- 172.21.15.111:0/3021239 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c003990 con 0x7f92a415da70 2024-09-13T17:57:05.460 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.455+0000 7f929affd640 1 -- 172.21.15.111:0/3021239 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929c00f040 con 0x7f92a415da70 2024-09-13T17:57:05.460 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.455+0000 7f92ac31f640 1 -- 172.21.15.111:0/3021239 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a41e4350 con 0x7f92a415da70 2024-09-13T17:57:05.460 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.455+0000 7f929affd640 1 -- 172.21.15.111:0/3021239 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c003d30 con 0x7f92a415da70 2024-09-13T17:57:05.460 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.455+0000 7f92ac31f640 1 -- 172.21.15.111:0/3021239 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a41562f0 con 0x7f92a415da70 2024-09-13T17:57:05.460 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.455+0000 7f929affd640 1 -- 172.21.15.111:0/3021239 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f929c004330 con 0x7f92a415da70 2024-09-13T17:57:05.460 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.455+0000 7f929affd640 1 -- 172.21.15.111:0/3021239 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f929c015b10 con 0x7f92a415da70 2024-09-13T17:57:05.460 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.455+0000 7f92ac31f640 1 -- 172.21.15.111:0/3021239 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a41e4870 con 0x7f92a415da70 2024-09-13T17:57:05.461 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:05.455+0000 7f8156532040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:57:05.461 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.457+0000 7f929affd640 1 -- 172.21.15.111:0/3021239 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f92a41e4870 con 0x7f92a415da70 2024-09-13T17:57:05.467 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:05.465+0000 7f6a9ba15040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:57:05.509 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:05.507+0000 7f8156532040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:57:05.516 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.515+0000 7f92ac31f640 1 -- 172.21.15.111:0/3021239 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92a40ced70 con 0x7f92a415da70 2024-09-13T17:57:05.517 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.515+0000 7f929affd640 1 -- 172.21.15.111:0/3021239 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f929c01a070 con 0x7f92a415da70 2024-09-13T17:57:05.517 INFO:teuthology.misc.health.smithi111.stdout: 2024-09-13T17:57:05.517 INFO:teuthology.misc.health.smithi111.stdout:{"epoch":10,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:04.324664+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:05.518 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.516+0000 7f92ac31f640 1 -- 172.21.15.111:0/3021239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92a415da70 msgr2=0x7f92a41e5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:05.518 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.516+0000 7f92ac31f640 1 --2- 172.21.15.111:0/3021239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92a415da70 0x7f92a41e5480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f929c007df0 tx=0x7f929c00d4a0 comp rx=0 tx=0).stop 2024-09-13T17:57:05.518 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.516+0000 7f92ac31f640 1 -- 172.21.15.111:0/3021239 shutdown_connections 2024-09-13T17:57:05.518 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.516+0000 7f92ac31f640 1 --2- 172.21.15.111:0/3021239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92a415da70 0x7f92a41e5480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:05.519 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.516+0000 7f92ac31f640 1 --2- 172.21.15.111:0/3021239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a40ce8c0 0x7f92a41e4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:05.519 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.516+0000 7f92ac31f640 1 --2- 172.21.15.111:0/3021239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92a40af5b0 0x7f92a4155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:05.519 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.516+0000 7f92ac31f640 1 -- 172.21.15.111:0/3021239 >> 172.21.15.111:0/3021239 conn(0x7f92a40d02e0 msgr2=0x7f92a415de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:05.519 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.516+0000 7f92ac31f640 1 -- 172.21.15.111:0/3021239 shutdown_connections 2024-09-13T17:57:05.519 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:05.516+0000 7f92ac31f640 1 -- 172.21.15.111:0/3021239 wait complete. 2024-09-13T17:57:05.527 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T17:57:05.537 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:05.535+0000 7f6a9ba15040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:57:05.576 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:05.575+0000 7f8156532040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:57:05.582 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:05.580+0000 7f6a9ba15040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:57:05.620 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:05.618+0000 7f8156532040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:57:05.625 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:05.623+0000 7f6a9ba15040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:57:05.663 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:05.661+0000 7f8156532040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:57:06.258 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:06.256+0000 7f6a9ba15040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:57:06.301 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-09-13T17:57:06.299+0000 7f8156532040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:57:07.271 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:07.270+0000 7f6a3bfdb640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T17:57:07.272 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-09-13T17:57:07.270+0000 7f6a3bfdb640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T17:57:08.434 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-09-13T17:57:08.432+0000 7f1e854dc640 -1 osd.6 0 log_to_monitors true 2024-09-13T17:57:08.476 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-09-13T17:57:08.474+0000 7fb4cca87640 -1 osd.7 0 log_to_monitors true 2024-09-13T17:57:08.482 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-09-13T17:57:08.481+0000 7f0730db2640 -1 osd.2 0 log_to_monitors true 2024-09-13T17:57:08.500 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-09-13T17:57:08.499+0000 7f6d9f20f640 -1 osd.5 0 log_to_monitors true 2024-09-13T17:57:08.505 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-09-13T17:57:08.504+0000 7f8e52adc640 -1 osd.4 0 log_to_monitors true 2024-09-13T17:57:08.540 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-09-13T17:57:08.538+0000 7f194f729640 -1 osd.3 0 log_to_monitors true 2024-09-13T17:57:08.555 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-09-13T17:57:08.553+0000 7f905efdb640 -1 osd.0 0 log_to_monitors true 2024-09-13T17:57:08.565 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-09-13T17:57:08.563+0000 7f80eca0f640 -1 osd.1 0 log_to_monitors true 2024-09-13T17:57:12.316 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-09-13T17:57:12.314+0000 7f6d9aaeb640 -1 osd.5 0 waiting for initial osdmap 2024-09-13T17:57:12.317 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-09-13T17:57:12.313+0000 7f072cd3f640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T17:57:12.325 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-09-13T17:57:12.323+0000 7f6d9290c640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:57:12.334 DEBUG:teuthology.orchestra.run.smithi111:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:12.338 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-09-13T17:57:12.336+0000 7f0724b60640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:57:12.365 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-09-13T17:57:12.363+0000 7f194b6b6640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T17:57:12.368 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-09-13T17:57:12.366+0000 7fb4c8a14640 -1 osd.7 0 waiting for initial osdmap 2024-09-13T17:57:12.370 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-09-13T17:57:12.368+0000 7f80e841a640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T17:57:12.370 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-09-13T17:57:12.369+0000 7f905af68640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T17:57:12.379 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-09-13T17:57:12.378+0000 7f19434d7640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:57:12.383 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-09-13T17:57:12.382+0000 7fb4c0835640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:57:12.384 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-09-13T17:57:12.383+0000 7f9052d89640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:57:12.385 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-09-13T17:57:12.383+0000 7f80e023b640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:57:12.395 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-09-13T17:57:12.394+0000 7f8e4ea69640 -1 osd.4 0 waiting for initial osdmap 2024-09-13T17:57:12.403 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-09-13T17:57:12.402+0000 7f1e81469640 -1 osd.6 0 waiting for initial osdmap 2024-09-13T17:57:12.411 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-09-13T17:57:12.410+0000 7f8e4688a640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:57:12.415 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-09-13T17:57:12.414+0000 7f1e7928a640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:57:12.437 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.435+0000 7f04c3a73640 1 Processor -- start 2024-09-13T17:57:12.437 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.435+0000 7f04c3a73640 1 -- start start 2024-09-13T17:57:12.439 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.436+0000 7f04c3a73640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f04bc15c690 0x7f04bc0cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:12.439 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.436+0000 7f04c3a73640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04bc0cde10 con 0x7f04bc0d2b40 2024-09-13T17:57:12.439 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.436+0000 7f04c3a73640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04bc0ce0f0 con 0x7f04bc0cd7d0 2024-09-13T17:57:12.439 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.436+0000 7f04c3a73640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04bc0ce3d0 con 0x7f04bc15c690 2024-09-13T17:57:12.439 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.436+0000 7f04c2010640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f04bc0d2b40 0x7f04bc0d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38978/0 (socket says 172.21.15.111:38978) 2024-09-13T17:57:12.439 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.436+0000 7f04c2010640 1 -- 172.21.15.111:0/526279823 learned_addr learned my addr 172.21.15.111:0/526279823 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:12.439 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.437+0000 7f04c180f640 1 --2- 172.21.15.111:0/526279823 >> v2:172.21.15.188:3300/0 conn(0x7f04bc15c690 0x7f04bc0cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:12.439 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.437+0000 7f04c080d640 1 -- 172.21.15.111:0/526279823 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1978146144 0 0) 0x7f04bc0cde10 con 0x7f04bc0d2b40 2024-09-13T17:57:12.439 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.437+0000 7f04c080d640 1 -- 172.21.15.111:0/526279823 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04a4003520 con 0x7f04bc0d2b40 2024-09-13T17:57:12.439 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.437+0000 7f04c180f640 1 -- 172.21.15.111:0/526279823 >> v1:172.21.15.188:6789/0 conn(0x7f04bc0cd7d0 legacy=0x7f04bc0d2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:12.440 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.437+0000 7f04c180f640 1 -- 172.21.15.111:0/526279823 >> v1:172.21.15.111:6789/0 conn(0x7f04bc0d2b40 legacy=0x7f04bc0d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:12.440 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.437+0000 7f04c080d640 1 -- 172.21.15.111:0/526279823 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2613013724 0 0) 0x7f04a4003520 con 0x7f04bc0d2b40 2024-09-13T17:57:12.440 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.437+0000 7f04c180f640 1 -- 172.21.15.111:0/526279823 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc0ce6b0 con 0x7f04bc15c690 2024-09-13T17:57:12.441 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.438+0000 7f04c180f640 1 --2- 172.21.15.111:0/526279823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04bc15c690 0x7f04bc0cd290 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f04b40097a0 tx=0x7f04b402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c54104f44a57a55a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:12.441 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.438+0000 7f04c080d640 1 -- 172.21.15.111:0/526279823 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac001670 con 0x7f04bc15c690 2024-09-13T17:57:12.441 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.438+0000 7f04c080d640 1 -- 172.21.15.111:0/526279823 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b402ac50 con 0x7f04bc15c690 2024-09-13T17:57:12.441 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.438+0000 7f04c080d640 1 -- 172.21.15.111:0/526279823 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4032440 con 0x7f04bc15c690 2024-09-13T17:57:12.441 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.438+0000 7f04c3a73640 1 -- 172.21.15.111:0/526279823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04bc15c690 msgr2=0x7f04bc0cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:12.441 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.438+0000 7f04c3a73640 1 --2- 172.21.15.111:0/526279823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04bc15c690 0x7f04bc0cd290 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f04b40097a0 tx=0x7f04b402a1c0 comp rx=0 tx=0).stop 2024-09-13T17:57:12.441 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.438+0000 7f04c3a73640 1 -- 172.21.15.111:0/526279823 shutdown_connections 2024-09-13T17:57:12.442 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.438+0000 7f04c3a73640 1 --2- 172.21.15.111:0/526279823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04bc15c690 0x7f04bc0cd290 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:12.442 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.438+0000 7f04c3a73640 1 -- 172.21.15.111:0/526279823 >> 172.21.15.111:0/526279823 conn(0x7f04bc09b810 msgr2=0x7f04bc089f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:12.442 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.438+0000 7f04c3a73640 1 -- 172.21.15.111:0/526279823 shutdown_connections 2024-09-13T17:57:12.442 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.438+0000 7f04c3a73640 1 -- 172.21.15.111:0/526279823 wait complete. 2024-09-13T17:57:12.442 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.439+0000 7f04c3a73640 1 Processor -- start 2024-09-13T17:57:12.442 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.439+0000 7f04c3a73640 1 -- start start 2024-09-13T17:57:12.444 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.440+0000 7f04c3a73640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04bc0cd7d0 0x7f04bc0b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:12.444 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.440+0000 7f04c3a73640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04bc0d2b40 0x7f04bc15eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:12.445 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.440+0000 7f04c3a73640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04bc15c690 0x7f04bc15f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:12.445 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.440+0000 7f04c3a73640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc153cd0 con 0x7f04bc0cd7d0 2024-09-13T17:57:12.445 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.440+0000 7f04c3a73640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc15dc00 con 0x7f04bc0d2b40 2024-09-13T17:57:12.445 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.440+0000 7f04c3a73640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc15dea0 con 0x7f04bc15c690 2024-09-13T17:57:12.445 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.440+0000 7f04c180f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04bc0cd7d0 0x7f04bc0b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:12.445 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.440+0000 7f04c180f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04bc0cd7d0 0x7f04bc0b7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49466/0 (socket says 172.21.15.111:49466) 2024-09-13T17:57:12.445 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.440+0000 7f04c180f640 1 -- 172.21.15.111:0/167080726 learned_addr learned my addr 172.21.15.111:0/167080726 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:12.445 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.440+0000 7f04c100e640 1 --2- 172.21.15.111:0/167080726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04bc0d2b40 0x7f04bc15eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:12.445 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.440+0000 7f04c2010640 1 --2- 172.21.15.111:0/167080726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04bc15c690 0x7f04bc15f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:12.445 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.441+0000 7f04c180f640 1 -- 172.21.15.111:0/167080726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04bc15c690 msgr2=0x7f04bc15f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:12.445 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.441+0000 7f04c180f640 1 --2- 172.21.15.111:0/167080726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04bc15c690 0x7f04bc15f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:12.445 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.441+0000 7f04c180f640 1 -- 172.21.15.111:0/167080726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04bc0d2b40 msgr2=0x7f04bc15eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:12.445 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.441+0000 7f04c180f640 1 --2- 172.21.15.111:0/167080726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04bc0d2b40 0x7f04bc15eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:12.446 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.441+0000 7f04c180f640 1 -- 172.21.15.111:0/167080726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b4009400 con 0x7f04bc0cd7d0 2024-09-13T17:57:12.446 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.441+0000 7f04c180f640 1 --2- 172.21.15.111:0/167080726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04bc0cd7d0 0x7f04bc0b7b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f04b4035040 tx=0x7f04b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:12.446 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.441+0000 7f04ba7fc640 1 -- 172.21.15.111:0/167080726 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b403c680 con 0x7f04bc0cd7d0 2024-09-13T17:57:12.446 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.441+0000 7f04ba7fc640 1 -- 172.21.15.111:0/167080726 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b403c7e0 con 0x7f04bc0cd7d0 2024-09-13T17:57:12.446 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.441+0000 7f04c3a73640 1 -- 172.21.15.111:0/167080726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04bc15e180 con 0x7f04bc0cd7d0 2024-09-13T17:57:12.446 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.441+0000 7f04ba7fc640 1 -- 172.21.15.111:0/167080726 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b403cb70 con 0x7f04bc0cd7d0 2024-09-13T17:57:12.446 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.441+0000 7f04c3a73640 1 -- 172.21.15.111:0/167080726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04bc155cf0 con 0x7f04bc0cd7d0 2024-09-13T17:57:12.446 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.442+0000 7f04c3a73640 1 -- 172.21.15.111:0/167080726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04bc0d2960 con 0x7f04bc0cd7d0 2024-09-13T17:57:12.446 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.442+0000 7f04ba7fc640 1 -- 172.21.15.111:0/167080726 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7f04b4039040 con 0x7f04bc0cd7d0 2024-09-13T17:57:12.447 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.443+0000 7f04ba7fc640 1 --2- 172.21.15.111:0/167080726 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f04a806ab70 0x7f04a806d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:12.447 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.443+0000 7f04ba7fc640 1 -- 172.21.15.111:0/167080726 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f04b40a8bf0 con 0x7f04bc0cd7d0 2024-09-13T17:57:12.447 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.443+0000 7f04c100e640 1 --2- 172.21.15.111:0/167080726 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f04a806ab70 0x7f04a806d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:12.447 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.443+0000 7f04c100e640 1 --2- 172.21.15.111:0/167080726 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f04a806ab70 0x7f04a806d030 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0b7ae0 tx=0x7f04b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:12.449 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.447+0000 7f04ba7fc640 1 -- 172.21.15.111:0/167080726 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f04b4074e10 con 0x7f04bc0cd7d0 2024-09-13T17:57:12.613 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.611+0000 7f04c3a73640 1 -- 172.21.15.111:0/167080726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04bc08af30 con 0x7f04bc0cd7d0 2024-09-13T17:57:12.613 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.611+0000 7f04ba7fc640 1 -- 172.21.15.111:0/167080726 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f04b4037070 con 0x7f04bc0cd7d0 2024-09-13T17:57:12.614 INFO:teuthology.misc.health.smithi111.stdout: 2024-09-13T17:57:12.614 INFO:teuthology.misc.health.smithi111.stdout:{"epoch":12,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:10.292326+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:12.615 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.613+0000 7f04c3a73640 1 -- 172.21.15.111:0/167080726 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f04a806ab70 msgr2=0x7f04a806d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:12.615 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.613+0000 7f04c3a73640 1 --2- 172.21.15.111:0/167080726 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f04a806ab70 0x7f04a806d030 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0b7ae0 tx=0x7f04b0009040 comp rx=0 tx=0).stop 2024-09-13T17:57:12.615 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.613+0000 7f04c3a73640 1 -- 172.21.15.111:0/167080726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04bc0cd7d0 msgr2=0x7f04bc0b7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:12.615 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.613+0000 7f04c3a73640 1 --2- 172.21.15.111:0/167080726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04bc0cd7d0 0x7f04bc0b7b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f04b4035040 tx=0x7f04b40046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:12.616 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.613+0000 7f04c3a73640 1 -- 172.21.15.111:0/167080726 shutdown_connections 2024-09-13T17:57:12.616 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.613+0000 7f04c3a73640 1 --2- 172.21.15.111:0/167080726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04bc15c690 0x7f04bc15f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:12.616 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.613+0000 7f04c3a73640 1 --2- 172.21.15.111:0/167080726 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f04a806ab70 0x7f04a806d030 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:12.616 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.613+0000 7f04c3a73640 1 --2- 172.21.15.111:0/167080726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04bc0d2b40 0x7f04bc15eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:12.616 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.613+0000 7f04c3a73640 1 --2- 172.21.15.111:0/167080726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04bc0cd7d0 0x7f04bc0b7b50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:12.616 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.613+0000 7f04c3a73640 1 -- 172.21.15.111:0/167080726 >> 172.21.15.111:0/167080726 conn(0x7f04bc09b810 msgr2=0x7f04bc0d0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:12.616 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.613+0000 7f04c3a73640 1 -- 172.21.15.111:0/167080726 shutdown_connections 2024-09-13T17:57:12.616 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:12.613+0000 7f04c3a73640 1 -- 172.21.15.111:0/167080726 wait complete. 2024-09-13T17:57:12.624 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T17:57:19.432 DEBUG:teuthology.orchestra.run.smithi111:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:19.532 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.530+0000 7ff520215640 1 Processor -- start 2024-09-13T17:57:19.532 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.530+0000 7ff520215640 1 -- start start 2024-09-13T17:57:19.536 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.531+0000 7ff520215640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff51815c8d0 0x7ff51806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.536 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.531+0000 7ff520215640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5180cea30 con 0x7ff5180989d0 2024-09-13T17:57:19.536 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.531+0000 7ff520215640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5180d0dd0 con 0x7ff5180af260 2024-09-13T17:57:19.536 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.531+0000 7ff520215640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5180cb040 con 0x7ff51815c8d0 2024-09-13T17:57:19.536 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.531+0000 7ff51e7b2640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff5180989d0 0x7ff5180ceec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38990/0 (socket says 172.21.15.111:38990) 2024-09-13T17:57:19.537 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.531+0000 7ff51e7b2640 1 -- 172.21.15.111:0/4017528861 learned_addr learned my addr 172.21.15.111:0/4017528861 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:19.537 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.531+0000 7ff51dfb1640 1 --2- 172.21.15.111:0/4017528861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff51815c8d0 0x7ff51806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.537 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.531+0000 7ff51cfaf640 1 -- 172.21.15.111:0/4017528861 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221186147 0 0) 0x7ff5180cea30 con 0x7ff5180989d0 2024-09-13T17:57:19.537 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.531+0000 7ff51cfaf640 1 -- 172.21.15.111:0/4017528861 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff504003520 con 0x7ff5180989d0 2024-09-13T17:57:19.537 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51cfaf640 1 -- 172.21.15.111:0/4017528861 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514432546 0 0) 0x7ff5180d0dd0 con 0x7ff5180af260 2024-09-13T17:57:19.537 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51cfaf640 1 -- 172.21.15.111:0/4017528861 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5180cea30 con 0x7ff5180af260 2024-09-13T17:57:19.537 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51cfaf640 1 -- 172.21.15.111:0/4017528861 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4071344850 0 0) 0x7ff504003520 con 0x7ff5180989d0 2024-09-13T17:57:19.537 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51cfaf640 1 -- 172.21.15.111:0/4017528861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff51815c8d0 msgr2=0x7ff51806bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:19.537 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51cfaf640 1 --2- 172.21.15.111:0/4017528861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff51815c8d0 0x7ff51806bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.537 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51cfaf640 1 -- 172.21.15.111:0/4017528861 >> v1:172.21.15.188:6789/0 conn(0x7ff5180af260 legacy=0x7ff5180ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:19.537 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51cfaf640 1 -- 172.21.15.111:0/4017528861 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5180d1050 con 0x7ff5180989d0 2024-09-13T17:57:19.537 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51cfaf640 1 -- 172.21.15.111:0/4017528861 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7ff510001670 con 0x7ff5180989d0 2024-09-13T17:57:19.538 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51cfaf640 1 -- 172.21.15.111:0/4017528861 >> v1:172.21.15.111:6789/0 conn(0x7ff5180989d0 legacy=0x7ff5180ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:19.538 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51cfaf640 1 --2- 172.21.15.111:0/4017528861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff504003b30 0x7ff504003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.538 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51cfaf640 1 --2- 172.21.15.111:0/4017528861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff51815c8d0 0x7ff5040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.538 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51cfaf640 1 --2- 172.21.15.111:0/4017528861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff504004d00 0x7ff504005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.538 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51cfaf640 1 -- 172.21.15.111:0/4017528861 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5180cb040 con 0x7ff504004d00 2024-09-13T17:57:19.538 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51cfaf640 1 -- 172.21.15.111:0/4017528861 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50400ba30 con 0x7ff504003b30 2024-09-13T17:57:19.538 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51cfaf640 1 -- 172.21.15.111:0/4017528861 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50400bbf0 con 0x7ff51815c8d0 2024-09-13T17:57:19.538 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51e7b2640 1 --2- 172.21.15.111:0/4017528861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff504004d00 0x7ff504005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.538 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51dfb1640 1 --2- 172.21.15.111:0/4017528861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff504003b30 0x7ff504003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.538 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51d7b0640 1 --2- 172.21.15.111:0/4017528861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff51815c8d0 0x7ff5040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.538 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51e7b2640 1 -- 172.21.15.111:0/4017528861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff51815c8d0 msgr2=0x7ff5040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:19.539 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51e7b2640 1 --2- 172.21.15.111:0/4017528861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff51815c8d0 0x7ff5040047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.539 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51e7b2640 1 -- 172.21.15.111:0/4017528861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff504003b30 msgr2=0x7ff504003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:19.539 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51e7b2640 1 --2- 172.21.15.111:0/4017528861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff504003b30 0x7ff504003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.539 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.532+0000 7ff51e7b2640 1 -- 172.21.15.111:0/4017528861 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff504003520 con 0x7ff504004d00 2024-09-13T17:57:19.539 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.533+0000 7ff51e7b2640 1 --2- 172.21.15.111:0/4017528861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff504004d00 0x7ff504005180 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff51000c160 tx=0x7ff5040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5f0392484ed3d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:19.539 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.533+0000 7ff51cfaf640 1 -- 172.21.15.111:0/4017528861 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510005ad0 con 0x7ff504004d00 2024-09-13T17:57:19.539 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.533+0000 7ff51cfaf640 1 -- 172.21.15.111:0/4017528861 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51002d070 con 0x7ff504004d00 2024-09-13T17:57:19.539 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.533+0000 7ff51cfaf640 1 -- 172.21.15.111:0/4017528861 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510005e90 con 0x7ff504004d00 2024-09-13T17:57:19.539 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.534+0000 7ff520215640 1 -- 172.21.15.111:0/4017528861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff504004d00 msgr2=0x7ff504005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:19.539 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.534+0000 7ff520215640 1 --2- 172.21.15.111:0/4017528861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff504004d00 0x7ff504005180 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff51000c160 tx=0x7ff5040057f0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.539 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.534+0000 7ff51e7b2640 1 -- 172.21.15.111:0/4017528861 reap_dead start 2024-09-13T17:57:19.539 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.534+0000 7ff520215640 1 -- 172.21.15.111:0/4017528861 shutdown_connections 2024-09-13T17:57:19.539 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.534+0000 7ff520215640 1 -- 172.21.15.111:0/4017528861 >> 172.21.15.111:0/4017528861 conn(0x7ff5180d02e0 msgr2=0x7ff5180d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:19.540 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.534+0000 7ff520215640 1 -- 172.21.15.111:0/4017528861 shutdown_connections 2024-09-13T17:57:19.540 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.534+0000 7ff520215640 1 -- 172.21.15.111:0/4017528861 wait complete. 2024-09-13T17:57:19.542 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.535+0000 7ff520215640 1 Processor -- start 2024-09-13T17:57:19.543 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.535+0000 7ff520215640 1 -- start start 2024-09-13T17:57:19.543 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff520215640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff504004d00 0x7ff518153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.543 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff520215640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5180b5da0 0x7ff51815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.543 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff520215640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5181f1ed0 0x7ff5180ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.543 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff520215640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51815f470 con 0x7ff5180b5da0 2024-09-13T17:57:19.543 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff520215640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5180b5340 con 0x7ff504004d00 2024-09-13T17:57:19.543 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff520215640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51815f6c0 con 0x7ff5181f1ed0 2024-09-13T17:57:19.543 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff51d7b0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5180b5da0 0x7ff51815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.543 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff51d7b0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5180b5da0 0x7ff51815ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49518/0 (socket says 172.21.15.111:49518) 2024-09-13T17:57:19.543 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff51d7b0640 1 -- 172.21.15.111:0/2479296951 learned_addr learned my addr 172.21.15.111:0/2479296951 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:19.543 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff51dfb1640 1 --2- 172.21.15.111:0/2479296951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff504004d00 0x7ff518153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.543 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff51e7b2640 1 --2- 172.21.15.111:0/2479296951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5181f1ed0 0x7ff5180ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.544 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff51d7b0640 1 -- 172.21.15.111:0/2479296951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5181f1ed0 msgr2=0x7ff5180ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:19.544 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff51d7b0640 1 --2- 172.21.15.111:0/2479296951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5181f1ed0 0x7ff5180ae3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.544 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff51d7b0640 1 -- 172.21.15.111:0/2479296951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff504004d00 msgr2=0x7ff518153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:19.544 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff51d7b0640 1 --2- 172.21.15.111:0/2479296951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff504004d00 0x7ff518153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.544 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff51d7b0640 1 -- 172.21.15.111:0/2479296951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51815f9a0 con 0x7ff5180b5da0 2024-09-13T17:57:19.544 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff51d7b0640 1 --2- 172.21.15.111:0/2479296951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5180b5da0 0x7ff51815ccb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff50800f770 tx=0x7ff50800fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:19.545 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.536+0000 7ff50e7fc640 1 -- 172.21.15.111:0/2479296951 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5080050d0 con 0x7ff5180b5da0 2024-09-13T17:57:19.545 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.537+0000 7ff50e7fc640 1 -- 172.21.15.111:0/2479296951 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff508005270 con 0x7ff5180b5da0 2024-09-13T17:57:19.545 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.537+0000 7ff520215640 1 -- 172.21.15.111:0/2479296951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5180b55a0 con 0x7ff5180b5da0 2024-09-13T17:57:19.545 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.537+0000 7ff50e7fc640 1 -- 172.21.15.111:0/2479296951 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50800d730 con 0x7ff5180b5da0 2024-09-13T17:57:19.545 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.537+0000 7ff520215640 1 -- 172.21.15.111:0/2479296951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5181f2380 con 0x7ff5180b5da0 2024-09-13T17:57:19.545 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.537+0000 7ff520215640 1 -- 172.21.15.111:0/2479296951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5180b5ac0 con 0x7ff5180b5da0 2024-09-13T17:57:19.545 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.538+0000 7ff50e7fc640 1 -- 172.21.15.111:0/2479296951 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff50800daa0 con 0x7ff5180b5da0 2024-09-13T17:57:19.545 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.538+0000 7ff50e7fc640 1 --2- 172.21.15.111:0/2479296951 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff4fc06ac00 0x7ff4fc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.545 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.538+0000 7ff50e7fc640 1 -- 172.21.15.111:0/2479296951 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff508089110 con 0x7ff5180b5da0 2024-09-13T17:57:19.545 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.538+0000 7ff51dfb1640 1 --2- 172.21.15.111:0/2479296951 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff4fc06ac00 0x7ff4fc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.545 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.539+0000 7ff51dfb1640 1 --2- 172.21.15.111:0/2479296951 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff4fc06ac00 0x7ff4fc06d0c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff514014510 tx=0x7ff51400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:19.546 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.543+0000 7ff50e7fc640 1 -- 172.21.15.111:0/2479296951 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff5180b5ac0 con 0x7ff5180b5da0 2024-09-13T17:57:19.705 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.704+0000 7ff520215640 1 -- 172.21.15.111:0/2479296951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5180b5ac0 con 0x7ff5180b5da0 2024-09-13T17:57:19.706 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.704+0000 7ff50e7fc640 1 -- 172.21.15.111:0/2479296951 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12910 (secure 0 0 0) 0x7ff5180b5ac0 con 0x7ff5180b5da0 2024-09-13T17:57:19.706 INFO:teuthology.misc.health.smithi111.stdout: 2024-09-13T17:57:19.707 INFO:teuthology.misc.health.smithi111.stdout:{"epoch":16,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:16.526252+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:19.708 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.706+0000 7ff520215640 1 -- 172.21.15.111:0/2479296951 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff4fc06ac00 msgr2=0x7ff4fc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:19.709 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.706+0000 7ff520215640 1 --2- 172.21.15.111:0/2479296951 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff4fc06ac00 0x7ff4fc06d0c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff514014510 tx=0x7ff51400b040 comp rx=0 tx=0).stop 2024-09-13T17:57:19.709 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.706+0000 7ff520215640 1 -- 172.21.15.111:0/2479296951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5180b5da0 msgr2=0x7ff51815ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:19.709 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.706+0000 7ff520215640 1 --2- 172.21.15.111:0/2479296951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5180b5da0 0x7ff51815ccb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff50800f770 tx=0x7ff50800fc30 comp rx=0 tx=0).stop 2024-09-13T17:57:19.709 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.706+0000 7ff520215640 1 -- 172.21.15.111:0/2479296951 shutdown_connections 2024-09-13T17:57:19.709 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.706+0000 7ff520215640 1 --2- 172.21.15.111:0/2479296951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5181f1ed0 0x7ff5180ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.709 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.706+0000 7ff520215640 1 --2- 172.21.15.111:0/2479296951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5180b5da0 0x7ff51815ccb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.709 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.706+0000 7ff520215640 1 --2- 172.21.15.111:0/2479296951 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff4fc06ac00 0x7ff4fc06d0c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.709 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.706+0000 7ff520215640 1 --2- 172.21.15.111:0/2479296951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff504004d00 0x7ff518153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.709 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.706+0000 7ff520215640 1 -- 172.21.15.111:0/2479296951 >> 172.21.15.111:0/2479296951 conn(0x7ff5180d02e0 msgr2=0x7ff51806ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:19.709 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.707+0000 7ff520215640 1 -- 172.21.15.111:0/2479296951 shutdown_connections 2024-09-13T17:57:19.709 INFO:teuthology.misc.health.smithi111.stderr:2024-09-13T17:57:19.707+0000 7ff520215640 1 -- 172.21.15.111:0/2479296951 wait complete. 2024-09-13T17:57:19.718 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-13T17:57:19.718 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T17:57:19.718 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:57:19.718 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:57:19.720 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:57:19.720 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T17:57:19.721 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T17:57:19.722 INFO:tasks.ceph.mds.c:Started 2024-09-13T17:57:19.722 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T17:57:19.723 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T17:57:19.726 INFO:tasks.ceph.mds.b:Started 2024-09-13T17:57:19.726 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T17:57:19.726 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T17:57:19.728 INFO:tasks.ceph.mds.d:Started 2024-09-13T17:57:19.728 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-13T17:57:19.728 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:19.794 INFO:tasks.ceph.mds.a.smithi111.stdout:starting mds.a at 2024-09-13T17:57:19.795 INFO:tasks.ceph.mds.c.smithi111.stdout:starting mds.c at 2024-09-13T17:57:19.795 INFO:tasks.ceph.mds.a.smithi111.stderr:2024-09-13T17:57:19.793+0000 7fb97cbf4640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-13T17:57:19.810 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-09-13T17:57:19.810 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-09-13T17:57:19.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.827+0000 7f7266976640 1 Processor -- start 2024-09-13T17:57:19.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.827+0000 7f7266976640 1 -- start start 2024-09-13T17:57:19.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.828+0000 7f7266976640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f726015c6e0 0x7f72600b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.828+0000 7f7266976640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f72600d07b0 0x7f72600b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.828+0000 7f7266976640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72600d1840 con 0x7f72600ccb20 2024-09-13T17:57:19.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.828+0000 7f7266976640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726006bd20 con 0x7f726015c6e0 2024-09-13T17:57:19.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.828+0000 7f7266976640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72600d0c30 con 0x7f72600d07b0 2024-09-13T17:57:19.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.828+0000 7f725ffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f726015c6e0 0x7f72600b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.828+0000 7f7264f13640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f72600ccb20 0x7f7260160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38992/0 (socket says 172.21.15.111:38992) 2024-09-13T17:57:19.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.828+0000 7f7264f13640 1 -- 172.21.15.111:0/4214414990 learned_addr learned my addr 172.21.15.111:0/4214414990 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:19.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.829+0000 7f725effd640 1 -- 172.21.15.111:0/4214414990 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722406882 0 0) 0x7f72600d1840 con 0x7f72600ccb20 2024-09-13T17:57:19.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.829+0000 7f725f7fe640 1 --2- 172.21.15.111:0/4214414990 >> v2:172.21.15.188:3300/0 conn(0x7f72600d07b0 0x7f72600b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.829+0000 7f725effd640 1 -- 172.21.15.111:0/4214414990 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7248003520 con 0x7f72600ccb20 2024-09-13T17:57:19.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.829+0000 7f725ffff640 1 -- 172.21.15.111:0/4214414990 >> v2:172.21.15.188:3300/0 conn(0x7f72600d07b0 msgr2=0x7f72600b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:19.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.829+0000 7f725ffff640 1 --2- 172.21.15.111:0/4214414990 >> v2:172.21.15.188:3300/0 conn(0x7f72600d07b0 0x7f72600b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.829+0000 7f725ffff640 1 -- 172.21.15.111:0/4214414990 >> v1:172.21.15.111:6789/0 conn(0x7f72600ccb20 legacy=0x7f7260160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:19.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.829+0000 7f725ffff640 1 -- 172.21.15.111:0/4214414990 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72600d0f10 con 0x7f726015c6e0 2024-09-13T17:57:19.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.829+0000 7f725f7fe640 1 --2- 172.21.15.111:0/4214414990 >> v2:172.21.15.188:3300/0 conn(0x7f72600d07b0 0x7f72600b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:19.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.829+0000 7f725ffff640 1 --2- 172.21.15.111:0/4214414990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f726015c6e0 0x7f72600b3f50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f72600b1d80 tx=0x7f725002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36f5f131bdb9ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:19.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.830+0000 7f725effd640 1 -- 172.21.15.111:0/4214414990 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725002ae60 con 0x7f726015c6e0 2024-09-13T17:57:19.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.830+0000 7f725effd640 1 -- 172.21.15.111:0/4214414990 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7250032440 con 0x7f726015c6e0 2024-09-13T17:57:19.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.830+0000 7f725effd640 1 -- 172.21.15.111:0/4214414990 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72500327d0 con 0x7f726015c6e0 2024-09-13T17:57:19.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.830+0000 7f7266976640 1 -- 172.21.15.111:0/4214414990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f726015c6e0 msgr2=0x7f72600b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:19.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.830+0000 7f7266976640 1 --2- 172.21.15.111:0/4214414990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f726015c6e0 0x7f72600b3f50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f72600b1d80 tx=0x7f725002a110 comp rx=0 tx=0).stop 2024-09-13T17:57:19.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.830+0000 7f7266976640 1 -- 172.21.15.111:0/4214414990 shutdown_connections 2024-09-13T17:57:19.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.830+0000 7f7266976640 1 --2- 172.21.15.111:0/4214414990 >> v2:172.21.15.188:3300/0 conn(0x7f72600d07b0 0x7f72600b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.830+0000 7f7266976640 1 --2- 172.21.15.111:0/4214414990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f726015c6e0 0x7f72600b3f50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.830+0000 7f7266976640 1 -- 172.21.15.111:0/4214414990 >> 172.21.15.111:0/4214414990 conn(0x7f7260089f60 msgr2=0x7f72600d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:19.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.830+0000 7f7266976640 1 -- 172.21.15.111:0/4214414990 shutdown_connections 2024-09-13T17:57:19.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.830+0000 7f7266976640 1 -- 172.21.15.111:0/4214414990 wait complete. 2024-09-13T17:57:19.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.831+0000 7f7266976640 1 Processor -- start 2024-09-13T17:57:19.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.831+0000 7f7266976640 1 -- start start 2024-09-13T17:57:19.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.832+0000 7f7266976640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72600ccb20 0x7f7260154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.832+0000 7f7266976640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72600d07b0 0x7f7260154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.832+0000 7f7266976640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f726015c6e0 0x7f7260155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.832+0000 7f7266976640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72600c9410 con 0x7f72600d07b0 2024-09-13T17:57:19.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.832+0000 7f7266976640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7260157660 con 0x7f72600ccb20 2024-09-13T17:57:19.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.832+0000 7f7266976640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72601557a0 con 0x7f726015c6e0 2024-09-13T17:57:19.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.832+0000 7f725f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72600d07b0 0x7f7260154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.832+0000 7f725f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72600d07b0 0x7f7260154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49552/0 (socket says 172.21.15.111:49552) 2024-09-13T17:57:19.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.832+0000 7f725f7fe640 1 -- 172.21.15.111:0/2872657010 learned_addr learned my addr 172.21.15.111:0/2872657010 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:19.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.832+0000 7f7264f13640 1 --2- 172.21.15.111:0/2872657010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f726015c6e0 0x7f7260155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.832+0000 7f725ffff640 1 --2- 172.21.15.111:0/2872657010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72600ccb20 0x7f7260154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.833+0000 7f725f7fe640 1 -- 172.21.15.111:0/2872657010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f726015c6e0 msgr2=0x7f7260155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:19.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.833+0000 7f725f7fe640 1 --2- 172.21.15.111:0/2872657010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f726015c6e0 0x7f7260155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.833+0000 7f725f7fe640 1 -- 172.21.15.111:0/2872657010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72600ccb20 msgr2=0x7f7260154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:19.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.833+0000 7f725f7fe640 1 --2- 172.21.15.111:0/2872657010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72600ccb20 0x7f7260154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.833+0000 7f725f7fe640 1 -- 172.21.15.111:0/2872657010 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7250009400 con 0x7f72600d07b0 2024-09-13T17:57:19.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.833+0000 7f725ffff640 1 --2- 172.21.15.111:0/2872657010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72600ccb20 0x7f7260154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:19.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.833+0000 7f725f7fe640 1 --2- 172.21.15.111:0/2872657010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72600d07b0 0x7f7260154260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f725400c7f0 tx=0x7f725400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:19.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.833+0000 7f725cff9640 1 -- 172.21.15.111:0/2872657010 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72540077c0 con 0x7f72600d07b0 2024-09-13T17:57:19.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.833+0000 7f725cff9640 1 -- 172.21.15.111:0/2872657010 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7254007920 con 0x7f72600d07b0 2024-09-13T17:57:19.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.833+0000 7f7266976640 1 -- 172.21.15.111:0/2872657010 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72600cc550 con 0x7f72600d07b0 2024-09-13T17:57:19.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.833+0000 7f725cff9640 1 -- 172.21.15.111:0/2872657010 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254007c90 con 0x7f72600d07b0 2024-09-13T17:57:19.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.833+0000 7f7266976640 1 -- 172.21.15.111:0/2872657010 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72600cafe0 con 0x7f72600d07b0 2024-09-13T17:57:19.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.834+0000 7f7266976640 1 -- 172.21.15.111:0/2872657010 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72600b4c80 con 0x7f72600d07b0 2024-09-13T17:57:19.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.834+0000 7f725cff9640 1 -- 172.21.15.111:0/2872657010 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f72540113f0 con 0x7f72600d07b0 2024-09-13T17:57:19.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.835+0000 7f725cff9640 1 --2- 172.21.15.111:0/2872657010 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f724c06ac00 0x7f724c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.835+0000 7f725ffff640 1 --2- 172.21.15.111:0/2872657010 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f724c06ac00 0x7f724c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.835+0000 7f725cff9640 1 -- 172.21.15.111:0/2872657010 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7254089340 con 0x7f72600d07b0 2024-09-13T17:57:19.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.835+0000 7f725ffff640 1 --2- 172.21.15.111:0/2872657010 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f724c06ac00 0x7f724c06d0c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7250034000 tx=0x7f725002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:19.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:19.839+0000 7f725cff9640 1 -- 172.21.15.111:0/2872657010 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f72540547d0 con 0x7f72600d07b0 2024-09-13T17:57:20.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.001+0000 7f7266976640 1 -- 172.21.15.111:0/2872657010 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f726008c610 con 0x7f72600d07b0 2024-09-13T17:57:20.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.001+0000 7f725cff9640 1 -- 172.21.15.111:0/2872657010 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12910 (secure 0 0 0) 0x7f7254057df0 con 0x7f72600d07b0 2024-09-13T17:57:20.003 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:20.004 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":16,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:16.526252+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:20.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.004+0000 7f7266976640 1 -- 172.21.15.111:0/2872657010 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f724c06ac00 msgr2=0x7f724c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:20.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.004+0000 7f7266976640 1 --2- 172.21.15.111:0/2872657010 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f724c06ac00 0x7f724c06d0c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7250034000 tx=0x7f725002e000 comp rx=0 tx=0).stop 2024-09-13T17:57:20.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.004+0000 7f7266976640 1 -- 172.21.15.111:0/2872657010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72600d07b0 msgr2=0x7f7260154260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:20.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.004+0000 7f7266976640 1 --2- 172.21.15.111:0/2872657010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72600d07b0 0x7f7260154260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f725400c7f0 tx=0x7f725400ccb0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.004+0000 7f7266976640 1 -- 172.21.15.111:0/2872657010 shutdown_connections 2024-09-13T17:57:20.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.004+0000 7f7266976640 1 --2- 172.21.15.111:0/2872657010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f726015c6e0 0x7f7260155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.004+0000 7f7266976640 1 --2- 172.21.15.111:0/2872657010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72600d07b0 0x7f7260154260 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.004+0000 7f7266976640 1 --2- 172.21.15.111:0/2872657010 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f724c06ac00 0x7f724c06d0c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.004+0000 7f7266976640 1 --2- 172.21.15.111:0/2872657010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72600ccb20 0x7f7260154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.004+0000 7f7266976640 1 -- 172.21.15.111:0/2872657010 >> 172.21.15.111:0/2872657010 conn(0x7f7260089f60 msgr2=0x7f726006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:20.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.004+0000 7f7266976640 1 -- 172.21.15.111:0/2872657010 shutdown_connections 2024-09-13T17:57:20.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.004+0000 7f7266976640 1 -- 172.21.15.111:0/2872657010 wait complete. 2024-09-13T17:57:20.017 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:57:20.017 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:57:20.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.115+0000 7fc1b5cb7640 1 Processor -- start 2024-09-13T17:57:20.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.115+0000 7fc1b5cb7640 1 -- start start 2024-09-13T17:57:20.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.116+0000 7fc1b5cb7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc1b00cd500 0x7fc1b00b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:20.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.116+0000 7fc1b5cb7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b00d1f40 con 0x7fc1b00d1650 2024-09-13T17:57:20.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.116+0000 7fc1b5cb7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b00d2190 con 0x7fc1b00b1370 2024-09-13T17:57:20.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.116+0000 7fc1b5cb7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b00ccb20 con 0x7fc1b00cd500 2024-09-13T17:57:20.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.116+0000 7fc1af7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc1b00cd500 0x7fc1b00b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:20.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.116+0000 7fc1affff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc1b00d1650 0x7fc1b00b1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38998/0 (socket says 172.21.15.111:38998) 2024-09-13T17:57:20.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.116+0000 7fc1affff640 1 -- 172.21.15.111:0/1364422978 learned_addr learned my addr 172.21.15.111:0/1364422978 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:20.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.117+0000 7fc1af7fe640 1 -- 172.21.15.111:0/1364422978 >> v1:172.21.15.188:6789/0 conn(0x7fc1b00b1370 legacy=0x7fc1b00b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:20.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.117+0000 7fc1af7fe640 1 -- 172.21.15.111:0/1364422978 >> v1:172.21.15.111:6789/0 conn(0x7fc1b00d1650 legacy=0x7fc1b00b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:20.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.117+0000 7fc1ae7fc640 1 -- 172.21.15.111:0/1364422978 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816682946 0 0) 0x7fc1b00d1f40 con 0x7fc1b00d1650 2024-09-13T17:57:20.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.117+0000 7fc1af7fe640 1 -- 172.21.15.111:0/1364422978 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b00cce00 con 0x7fc1b00cd500 2024-09-13T17:57:20.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.117+0000 7fc1af7fe640 1 --2- 172.21.15.111:0/1364422978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1b00cd500 0x7fc1b00b3850 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009d90 tx=0x7fc1a002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31de4b192adb4ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:20.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.117+0000 7fc1ae7fc640 1 -- 172.21.15.111:0/1364422978 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0007740 con 0x7fc1b00cd500 2024-09-13T17:57:20.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.117+0000 7fc1ae7fc640 1 -- 172.21.15.111:0/1364422978 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a002d050 con 0x7fc1b00cd500 2024-09-13T17:57:20.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.118+0000 7fc1ae7fc640 1 -- 172.21.15.111:0/1364422978 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0007ac0 con 0x7fc1b00cd500 2024-09-13T17:57:20.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.118+0000 7fc1b5cb7640 1 -- 172.21.15.111:0/1364422978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1b00cd500 msgr2=0x7fc1b00b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:20.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.118+0000 7fc1b5cb7640 1 --2- 172.21.15.111:0/1364422978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1b00cd500 0x7fc1b00b3850 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009d90 tx=0x7fc1a002a6a0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.118+0000 7fc1b5cb7640 1 -- 172.21.15.111:0/1364422978 shutdown_connections 2024-09-13T17:57:20.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.118+0000 7fc1b5cb7640 1 --2- 172.21.15.111:0/1364422978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1b00cd500 0x7fc1b00b3850 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.118+0000 7fc1b5cb7640 1 -- 172.21.15.111:0/1364422978 >> 172.21.15.111:0/1364422978 conn(0x7fc1b0089f60 msgr2=0x7fc1b00d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:20.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.118+0000 7fc1b5cb7640 1 -- 172.21.15.111:0/1364422978 shutdown_connections 2024-09-13T17:57:20.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.118+0000 7fc1b5cb7640 1 -- 172.21.15.111:0/1364422978 wait complete. 2024-09-13T17:57:20.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.119+0000 7fc1b5cb7640 1 Processor -- start 2024-09-13T17:57:20.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.119+0000 7fc1b5cb7640 1 -- start start 2024-09-13T17:57:20.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.120+0000 7fc1b5cb7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1b00cd500 0x7fc1b0145fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:20.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.120+0000 7fc1b5cb7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1b00d1650 0x7fc1b01f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:20.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.120+0000 7fc1b5cb7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1b00c7ba0 0x7fc1b00d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:20.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.120+0000 7fc1b5cb7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b00c5f90 con 0x7fc1b00cd500 2024-09-13T17:57:20.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.120+0000 7fc1b5cb7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b01464f0 con 0x7fc1b00c7ba0 2024-09-13T17:57:20.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.120+0000 7fc1b5cb7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b00c8160 con 0x7fc1b00d1650 2024-09-13T17:57:20.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.120+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1b00cd500 0x7fc1b0145fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:20.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.120+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1b00cd500 0x7fc1b0145fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49576/0 (socket says 172.21.15.111:49576) 2024-09-13T17:57:20.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.120+0000 7fc1af7fe640 1 -- 172.21.15.111:0/266223823 learned_addr learned my addr 172.21.15.111:0/266223823 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:20.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.120+0000 7fc1af7fe640 1 -- 172.21.15.111:0/266223823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1b00d1650 msgr2=0x7fc1b01f1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:20.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.120+0000 7fc1af7fe640 1 --2- 172.21.15.111:0/266223823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1b00d1650 0x7fc1b01f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.120+0000 7fc1af7fe640 1 -- 172.21.15.111:0/266223823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1b00c7ba0 msgr2=0x7fc1b00d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:20.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.120+0000 7fc1af7fe640 1 --2- 172.21.15.111:0/266223823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1b00c7ba0 0x7fc1b00d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.120+0000 7fc1af7fe640 1 -- 172.21.15.111:0/266223823 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a0009a70 con 0x7fc1b00cd500 2024-09-13T17:57:20.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.121+0000 7fc1af7fe640 1 --2- 172.21.15.111:0/266223823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1b00cd500 0x7fc1b0145fb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009d90 tx=0x7fc1a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:20.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.121+0000 7fc197fff640 1 -- 172.21.15.111:0/266223823 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0031990 con 0x7fc1b00cd500 2024-09-13T17:57:20.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.121+0000 7fc197fff640 1 -- 172.21.15.111:0/266223823 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a0031af0 con 0x7fc1b00cd500 2024-09-13T17:57:20.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.121+0000 7fc1b5cb7640 1 -- 172.21.15.111:0/266223823 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b00c8440 con 0x7fc1b00cd500 2024-09-13T17:57:20.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.121+0000 7fc197fff640 1 -- 172.21.15.111:0/266223823 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0031e60 con 0x7fc1b00cd500 2024-09-13T17:57:20.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.121+0000 7fc1b5cb7640 1 -- 172.21.15.111:0/266223823 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b00c8a00 con 0x7fc1b00cd500 2024-09-13T17:57:20.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.122+0000 7fc1b5cb7640 1 -- 172.21.15.111:0/266223823 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b00b3850 con 0x7fc1b00cd500 2024-09-13T17:57:20.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.122+0000 7fc197fff640 1 -- 172.21.15.111:0/266223823 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc1a003c420 con 0x7fc1b00cd500 2024-09-13T17:57:20.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.123+0000 7fc197fff640 1 --2- 172.21.15.111:0/266223823 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc19c06ac30 0x7fc19c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:20.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.123+0000 7fc197fff640 1 -- 172.21.15.111:0/266223823 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc1a00aaec0 con 0x7fc1b00cd500 2024-09-13T17:57:20.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.123+0000 7fc1aeffd640 1 --2- 172.21.15.111:0/266223823 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc19c06ac30 0x7fc19c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:20.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.123+0000 7fc1aeffd640 1 --2- 172.21.15.111:0/266223823 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc19c06ac30 0x7fc19c06d0f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00d15f0 tx=0x7fc1a40079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:20.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.127+0000 7fc197fff640 1 -- 172.21.15.111:0/266223823 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc1a0076530 con 0x7fc1b00cd500 2024-09-13T17:57:20.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.291+0000 7fc1b5cb7640 1 -- 172.21.15.111:0/266223823 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc1b00b4ed0 con 0x7fc1b00cd500 2024-09-13T17:57:20.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.292+0000 7fc197fff640 1 -- 172.21.15.111:0/266223823 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc1a0040020 con 0x7fc1b00cd500 2024-09-13T17:57:20.294 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:57:20.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.294+0000 7fc1b5cb7640 1 -- 172.21.15.111:0/266223823 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc19c06ac30 msgr2=0x7fc19c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:20.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.294+0000 7fc1b5cb7640 1 --2- 172.21.15.111:0/266223823 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc19c06ac30 0x7fc19c06d0f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00d15f0 tx=0x7fc1a40079b0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.294+0000 7fc1b5cb7640 1 -- 172.21.15.111:0/266223823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1b00cd500 msgr2=0x7fc1b0145fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:20.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.294+0000 7fc1b5cb7640 1 --2- 172.21.15.111:0/266223823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1b00cd500 0x7fc1b0145fb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009d90 tx=0x7fc1a0004a40 comp rx=0 tx=0).stop 2024-09-13T17:57:20.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.294+0000 7fc1b5cb7640 1 -- 172.21.15.111:0/266223823 shutdown_connections 2024-09-13T17:57:20.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.294+0000 7fc1b5cb7640 1 --2- 172.21.15.111:0/266223823 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc19c06ac30 0x7fc19c06d0f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.294+0000 7fc1b5cb7640 1 --2- 172.21.15.111:0/266223823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1b00c7ba0 0x7fc1b00d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.294+0000 7fc1b5cb7640 1 --2- 172.21.15.111:0/266223823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1b00d1650 0x7fc1b01f1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.294+0000 7fc1b5cb7640 1 --2- 172.21.15.111:0/266223823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1b00cd500 0x7fc1b0145fb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.294+0000 7fc1b5cb7640 1 -- 172.21.15.111:0/266223823 >> 172.21.15.111:0/266223823 conn(0x7fc1b0089f60 msgr2=0x7fc1b015dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:20.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.294+0000 7fc1b5cb7640 1 -- 172.21.15.111:0/266223823 shutdown_connections 2024-09-13T17:57:20.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.294+0000 7fc1b5cb7640 1 -- 172.21.15.111:0/266223823 wait complete. 2024-09-13T17:57:20.306 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:57:20.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.400+0000 7f51f61a0640 1 Processor -- start 2024-09-13T17:57:20.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.401+0000 7f51f61a0640 1 -- start start 2024-09-13T17:57:20.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.401+0000 7f51f61a0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f51f006bf00 0x7f51f013efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:20.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.401+0000 7f51f61a0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51f015ca10 0x7f51f006c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:20.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.401+0000 7f51f61a0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f01593f0 con 0x7f51f0152f40 2024-09-13T17:57:20.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.401+0000 7f51f61a0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f00cae30 con 0x7f51f006bf00 2024-09-13T17:57:20.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.401+0000 7f51f61a0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f013f520 con 0x7f51f015ca10 2024-09-13T17:57:20.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.402+0000 7f51f499d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51f015ca10 0x7f51f006c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:20.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.402+0000 7f51f519e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f51f006bf00 0x7f51f013efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:20.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.402+0000 7f51f499d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51f015ca10 0x7f51f006c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:42980/0 (socket says 172.21.15.111:42980) 2024-09-13T17:57:20.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.402+0000 7f51f499d640 1 -- 172.21.15.111:0/2581266233 learned_addr learned my addr 172.21.15.111:0/2581266233 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:20.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.402+0000 7f51f499d640 1 -- 172.21.15.111:0/2581266233 >> v2:172.21.15.188:3300/0 conn(0x7f51f006bf00 msgr2=0x7f51f013efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:20.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.402+0000 7f51f499d640 1 --2- 172.21.15.111:0/2581266233 >> v2:172.21.15.188:3300/0 conn(0x7f51f006bf00 0x7f51f013efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.402+0000 7f51f499d640 1 -- 172.21.15.111:0/2581266233 >> v1:172.21.15.188:6789/0 conn(0x7f51f0152f40 legacy=0x7f51f01533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:20.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.402+0000 7f51f499d640 1 -- 172.21.15.111:0/2581266233 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f006c820 con 0x7f51f015ca10 2024-09-13T17:57:20.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.402+0000 7f51f519e640 1 --2- 172.21.15.111:0/2581266233 >> v2:172.21.15.188:3300/0 conn(0x7f51f006bf00 0x7f51f013efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:20.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.403+0000 7f51f499d640 1 --2- 172.21.15.111:0/2581266233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51f015ca10 0x7f51f006c2e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009870 tx=0x7f51e402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa9853a4c2d8173a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:20.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.403+0000 7f51effff640 1 -- 172.21.15.111:0/2581266233 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4032440 con 0x7f51f015ca10 2024-09-13T17:57:20.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.403+0000 7f51effff640 1 -- 172.21.15.111:0/2581266233 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e40325a0 con 0x7f51f015ca10 2024-09-13T17:57:20.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.403+0000 7f51effff640 1 -- 172.21.15.111:0/2581266233 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4032930 con 0x7f51f015ca10 2024-09-13T17:57:20.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.403+0000 7f51f61a0640 1 -- 172.21.15.111:0/2581266233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51f015ca10 msgr2=0x7f51f006c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:20.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.403+0000 7f51f61a0640 1 --2- 172.21.15.111:0/2581266233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51f015ca10 0x7f51f006c2e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009870 tx=0x7f51e402a1c0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.404+0000 7f51f61a0640 1 -- 172.21.15.111:0/2581266233 shutdown_connections 2024-09-13T17:57:20.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.404+0000 7f51f61a0640 1 --2- 172.21.15.111:0/2581266233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51f015ca10 0x7f51f006c2e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.404+0000 7f51f61a0640 1 --2- 172.21.15.111:0/2581266233 >> v2:172.21.15.188:3300/0 conn(0x7f51f006bf00 0x7f51f013efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.404+0000 7f51f61a0640 1 -- 172.21.15.111:0/2581266233 >> 172.21.15.111:0/2581266233 conn(0x7f51f006d860 msgr2=0x7f51f0069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:20.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.404+0000 7f51f61a0640 1 -- 172.21.15.111:0/2581266233 shutdown_connections 2024-09-13T17:57:20.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.404+0000 7f51f61a0640 1 -- 172.21.15.111:0/2581266233 wait complete. 2024-09-13T17:57:20.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.405+0000 7f51f61a0640 1 Processor -- start 2024-09-13T17:57:20.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.405+0000 7f51f61a0640 1 -- start start 2024-09-13T17:57:20.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.405+0000 7f51f61a0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51f006bf00 0x7f51f013a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:20.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.405+0000 7f51f61a0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51f0152f40 0x7f51f013b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:20.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.405+0000 7f51f61a0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51f015ca10 0x7f51f013aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.405+0000 7f51f61a0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f013a800 con 0x7f51f015ca10 2024-09-13T17:57:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.405+0000 7f51f61a0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f015ad60 con 0x7f51f006bf00 2024-09-13T17:57:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.405+0000 7f51f61a0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f015b040 con 0x7f51f0152f40 2024-09-13T17:57:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.406+0000 7f51f599f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51f015ca10 0x7f51f013aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.406+0000 7f51f499d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51f0152f40 0x7f51f013b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.406+0000 7f51f599f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51f015ca10 0x7f51f013aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49590/0 (socket says 172.21.15.111:49590) 2024-09-13T17:57:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.406+0000 7f51f599f640 1 -- 172.21.15.111:0/1230997927 learned_addr learned my addr 172.21.15.111:0/1230997927 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.406+0000 7f51f519e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51f006bf00 0x7f51f013a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.406+0000 7f51f599f640 1 -- 172.21.15.111:0/1230997927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51f0152f40 msgr2=0x7f51f013b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.406+0000 7f51f599f640 1 --2- 172.21.15.111:0/1230997927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51f0152f40 0x7f51f013b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.406+0000 7f51f599f640 1 -- 172.21.15.111:0/1230997927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51f006bf00 msgr2=0x7f51f013a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.406+0000 7f51f599f640 1 --2- 172.21.15.111:0/1230997927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51f006bf00 0x7f51f013a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.406+0000 7f51f599f640 1 -- 172.21.15.111:0/1230997927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e8009a70 con 0x7f51f015ca10 2024-09-13T17:57:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.406+0000 7f51f599f640 1 --2- 172.21.15.111:0/1230997927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51f015ca10 0x7f51f013aa30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f51e0001eb0 tx=0x7f51e000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.406+0000 7f51edffb640 1 -- 172.21.15.111:0/1230997927 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e00052d0 con 0x7f51f015ca10 2024-09-13T17:57:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.406+0000 7f51edffb640 1 -- 172.21.15.111:0/1230997927 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e0005430 con 0x7f51f015ca10 2024-09-13T17:57:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.406+0000 7f51f61a0640 1 -- 172.21.15.111:0/1230997927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e40094d0 con 0x7f51f015ca10 2024-09-13T17:57:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.406+0000 7f51edffb640 1 -- 172.21.15.111:0/1230997927 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0003810 con 0x7f51f015ca10 2024-09-13T17:57:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.406+0000 7f51f61a0640 1 -- 172.21.15.111:0/1230997927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f0155890 con 0x7f51f015ca10 2024-09-13T17:57:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.407+0000 7f51f61a0640 1 -- 172.21.15.111:0/1230997927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f00672b0 con 0x7f51f015ca10 2024-09-13T17:57:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.408+0000 7f51edffb640 1 -- 172.21.15.111:0/1230997927 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f51e0003970 con 0x7f51f015ca10 2024-09-13T17:57:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.408+0000 7f51edffb640 1 --2- 172.21.15.111:0/1230997927 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f51dc06ac00 0x7f51dc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.408+0000 7f51edffb640 1 -- 172.21.15.111:0/1230997927 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f51e0088510 con 0x7f51f015ca10 2024-09-13T17:57:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.408+0000 7f51f519e640 1 --2- 172.21.15.111:0/1230997927 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f51dc06ac00 0x7f51dc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:20.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.409+0000 7f51f519e640 1 --2- 172.21.15.111:0/1230997927 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f51dc06ac00 0x7f51dc06d0c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f51e80114e0 tx=0x7f51e800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:20.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.412+0000 7f51edffb640 1 -- 172.21.15.111:0/1230997927 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f51e0053af0 con 0x7f51f015ca10 2024-09-13T17:57:20.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.606+0000 7f51f61a0640 1 -- 172.21.15.111:0/1230997927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51f0155a80 con 0x7f51f015ca10 2024-09-13T17:57:20.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.607+0000 7f51edffb640 1 -- 172.21.15.111:0/1230997927 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1259 (secure 0 0 0) 0x7f51e0017500 con 0x7f51f015ca10 2024-09-13T17:57:20.609 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:20.609 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2565466103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2565466103},{"type":"v1","addr":"172.21.15.111:6837","nonce":2565466103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-09-13T17:57:20.609 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 2 2024-09-13T17:57:20.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.609+0000 7f51f61a0640 1 -- 172.21.15.111:0/1230997927 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f51dc06ac00 msgr2=0x7f51dc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:20.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.609+0000 7f51f61a0640 1 --2- 172.21.15.111:0/1230997927 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f51dc06ac00 0x7f51dc06d0c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f51e80114e0 tx=0x7f51e800b040 comp rx=0 tx=0).stop 2024-09-13T17:57:20.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.609+0000 7f51f61a0640 1 -- 172.21.15.111:0/1230997927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51f015ca10 msgr2=0x7f51f013aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:20.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.609+0000 7f51f61a0640 1 --2- 172.21.15.111:0/1230997927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51f015ca10 0x7f51f013aa30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f51e0001eb0 tx=0x7f51e000c890 comp rx=0 tx=0).stop 2024-09-13T17:57:20.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.609+0000 7f51f61a0640 1 -- 172.21.15.111:0/1230997927 shutdown_connections 2024-09-13T17:57:20.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.609+0000 7f51f61a0640 1 --2- 172.21.15.111:0/1230997927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51f015ca10 0x7f51f013aa30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.609+0000 7f51f61a0640 1 --2- 172.21.15.111:0/1230997927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51f0152f40 0x7f51f013b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.609+0000 7f51f61a0640 1 --2- 172.21.15.111:0/1230997927 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f51dc06ac00 0x7f51dc06d0c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.609+0000 7f51f61a0640 1 --2- 172.21.15.111:0/1230997927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51f006bf00 0x7f51f013a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:20.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.609+0000 7f51f61a0640 1 -- 172.21.15.111:0/1230997927 >> 172.21.15.111:0/1230997927 conn(0x7f51f006d860 msgr2=0x7f51f00706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:20.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.609+0000 7f51f61a0640 1 -- 172.21.15.111:0/1230997927 shutdown_connections 2024-09-13T17:57:20.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:20.609+0000 7f51f61a0640 1 -- 172.21.15.111:0/1230997927 wait complete. 2024-09-13T17:57:22.622 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:57:22.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.729+0000 7fcacc73e640 1 Processor -- start 2024-09-13T17:57:22.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.730+0000 7fcacc73e640 1 -- start start 2024-09-13T17:57:22.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcacc73e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac415c280 0x7fcac406aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:22.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcacc73e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac4066090 con 0x7fcac40ac1f0 2024-09-13T17:57:22.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcacc73e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac40ac6f0 con 0x7fcac415c660 2024-09-13T17:57:22.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcacc73e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac40ac9d0 con 0x7fcac415c280 2024-09-13T17:57:22.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcacacdb640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fcac40ac1f0 0x7fcac40b1f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58022/0 (socket says 172.21.15.111:58022) 2024-09-13T17:57:22.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcacacdb640 1 -- 172.21.15.111:0/3838821011 learned_addr learned my addr 172.21.15.111:0/3838821011 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:22.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcaca4da640 1 --2- 172.21.15.111:0/3838821011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac415c280 0x7fcac406aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:22.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcac94d8640 1 -- 172.21.15.111:0/3838821011 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 194511461 0 0) 0x7fcac4066090 con 0x7fcac40ac1f0 2024-09-13T17:57:22.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcac94d8640 1 -- 172.21.15.111:0/3838821011 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcab0003520 con 0x7fcac40ac1f0 2024-09-13T17:57:22.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcac94d8640 1 -- 172.21.15.111:0/3838821011 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042054423 0 0) 0x7fcac40ac6f0 con 0x7fcac415c660 2024-09-13T17:57:22.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcac94d8640 1 -- 172.21.15.111:0/3838821011 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcac4066090 con 0x7fcac415c660 2024-09-13T17:57:22.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcac94d8640 1 -- 172.21.15.111:0/3838821011 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 470451447 0 0) 0x7fcab0003520 con 0x7fcac40ac1f0 2024-09-13T17:57:22.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcac94d8640 1 -- 172.21.15.111:0/3838821011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac415c280 msgr2=0x7fcac406aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:22.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcac94d8640 1 --2- 172.21.15.111:0/3838821011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac415c280 0x7fcac406aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcac94d8640 1 -- 172.21.15.111:0/3838821011 >> v1:172.21.15.188:6789/0 conn(0x7fcac415c660 legacy=0x7fcac40abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:22.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcac94d8640 1 -- 172.21.15.111:0/3838821011 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac40accb0 con 0x7fcac40ac1f0 2024-09-13T17:57:22.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcac94d8640 1 -- 172.21.15.111:0/3838821011 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7fcabc0019f0 con 0x7fcac40ac1f0 2024-09-13T17:57:22.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcac94d8640 1 -- 172.21.15.111:0/3838821011 >> v1:172.21.15.111:6789/0 conn(0x7fcac40ac1f0 legacy=0x7fcac40b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:22.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcac94d8640 1 --2- 172.21.15.111:0/3838821011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcab0003b30 0x7fcab0003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:22.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcac94d8640 1 --2- 172.21.15.111:0/3838821011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab00047c0 0x7fcab0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:22.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcac94d8640 1 --2- 172.21.15.111:0/3838821011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac415c280 0x7fcab0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:22.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcac94d8640 1 -- 172.21.15.111:0/3838821011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac40ac9d0 con 0x7fcab00047c0 2024-09-13T17:57:22.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcac94d8640 1 -- 172.21.15.111:0/3838821011 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab000ba30 con 0x7fcab0003b30 2024-09-13T17:57:22.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.731+0000 7fcac94d8640 1 -- 172.21.15.111:0/3838821011 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab000bbf0 con 0x7fcac415c280 2024-09-13T17:57:22.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.732+0000 7fcac9cd9640 1 --2- 172.21.15.111:0/3838821011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab00047c0 0x7fcab0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:22.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.732+0000 7fcaca4da640 1 --2- 172.21.15.111:0/3838821011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcab0003b30 0x7fcab0003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:22.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.732+0000 7fcacacdb640 1 --2- 172.21.15.111:0/3838821011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac415c280 0x7fcab0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:22.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.732+0000 7fcac9cd9640 1 -- 172.21.15.111:0/3838821011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac415c280 msgr2=0x7fcab0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:22.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.732+0000 7fcac9cd9640 1 --2- 172.21.15.111:0/3838821011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac415c280 0x7fcab0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.732+0000 7fcac9cd9640 1 -- 172.21.15.111:0/3838821011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcab0003b30 msgr2=0x7fcab0003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:22.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.732+0000 7fcac9cd9640 1 --2- 172.21.15.111:0/3838821011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcab0003b30 0x7fcab0003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.732+0000 7fcac9cd9640 1 -- 172.21.15.111:0/3838821011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab0003520 con 0x7fcab00047c0 2024-09-13T17:57:22.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.732+0000 7fcac9cd9640 1 --2- 172.21.15.111:0/3838821011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab00047c0 0x7fcab0004c40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcab40018a0 tx=0x7fcab00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6770f98ab65bb3e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:22.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.733+0000 7fcac94d8640 1 -- 172.21.15.111:0/3838821011 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab40052d0 con 0x7fcab00047c0 2024-09-13T17:57:22.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.733+0000 7fcac94d8640 1 -- 172.21.15.111:0/3838821011 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab4005430 con 0x7fcab00047c0 2024-09-13T17:57:22.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.733+0000 7fcac94d8640 1 -- 172.21.15.111:0/3838821011 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab402ac60 con 0x7fcab00047c0 2024-09-13T17:57:22.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.733+0000 7fcacc73e640 1 -- 172.21.15.111:0/3838821011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab00047c0 msgr2=0x7fcab0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:22.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.733+0000 7fcacc73e640 1 --2- 172.21.15.111:0/3838821011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab00047c0 0x7fcab0004c40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcab40018a0 tx=0x7fcab00057f0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.733+0000 7fcacacdb640 1 -- 172.21.15.111:0/3838821011 reap_dead start 2024-09-13T17:57:22.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.733+0000 7fcacc73e640 1 -- 172.21.15.111:0/3838821011 shutdown_connections 2024-09-13T17:57:22.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.733+0000 7fcacc73e640 1 -- 172.21.15.111:0/3838821011 >> 172.21.15.111:0/3838821011 conn(0x7fcac40709f0 msgr2=0x7fcac4070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:22.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.733+0000 7fcacc73e640 1 -- 172.21.15.111:0/3838821011 shutdown_connections 2024-09-13T17:57:22.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.733+0000 7fcacc73e640 1 -- 172.21.15.111:0/3838821011 wait complete. 2024-09-13T17:57:22.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.735+0000 7fcacc73e640 1 Processor -- start 2024-09-13T17:57:22.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.735+0000 7fcacc73e640 1 -- start start 2024-09-13T17:57:22.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.735+0000 7fcacc73e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcab00047c0 0x7fcac41486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:22.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.735+0000 7fcacc73e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcac4151970 0x7fcac40cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:22.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.735+0000 7fcacc73e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac415e580 0x7fcac415ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:22.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.735+0000 7fcacc73e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac415ef40 con 0x7fcac415e580 2024-09-13T17:57:22.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.735+0000 7fcacc73e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac4148bf0 con 0x7fcac4151970 2024-09-13T17:57:22.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.735+0000 7fcacc73e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac40cc840 con 0x7fcab00047c0 2024-09-13T17:57:22.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.735+0000 7fcacacdb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac415e580 0x7fcac415ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:22.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.735+0000 7fcacacdb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac415e580 0x7fcac415ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36326/0 (socket says 172.21.15.111:36326) 2024-09-13T17:57:22.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.735+0000 7fcacacdb640 1 -- 172.21.15.111:0/3906684059 learned_addr learned my addr 172.21.15.111:0/3906684059 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:22.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.736+0000 7fcaca4da640 1 --2- 172.21.15.111:0/3906684059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcab00047c0 0x7fcac41486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:22.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.736+0000 7fcac9cd9640 1 --2- 172.21.15.111:0/3906684059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcac4151970 0x7fcac40cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:22.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.736+0000 7fcacacdb640 1 -- 172.21.15.111:0/3906684059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcab00047c0 msgr2=0x7fcac41486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:22.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.736+0000 7fcacacdb640 1 --2- 172.21.15.111:0/3906684059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcab00047c0 0x7fcac41486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.736+0000 7fcacacdb640 1 -- 172.21.15.111:0/3906684059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcac4151970 msgr2=0x7fcac40cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:22.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.736+0000 7fcacacdb640 1 --2- 172.21.15.111:0/3906684059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcac4151970 0x7fcac40cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.736+0000 7fcacacdb640 1 -- 172.21.15.111:0/3906684059 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac40ccb20 con 0x7fcac415e580 2024-09-13T17:57:22.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.736+0000 7fcacacdb640 1 --2- 172.21.15.111:0/3906684059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac415e580 0x7fcac415ea00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcac40ad3a0 tx=0x7fcabc00a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:22.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.736+0000 7fcabaffd640 1 -- 172.21.15.111:0/3906684059 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc00ac00 con 0x7fcac415e580 2024-09-13T17:57:22.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.736+0000 7fcabaffd640 1 -- 172.21.15.111:0/3906684059 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcabc00ada0 con 0x7fcac415e580 2024-09-13T17:57:22.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.736+0000 7fcacc73e640 1 -- 172.21.15.111:0/3906684059 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac40cce00 con 0x7fcac415e580 2024-09-13T17:57:22.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.736+0000 7fcabaffd640 1 -- 172.21.15.111:0/3906684059 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc013630 con 0x7fcac415e580 2024-09-13T17:57:22.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.736+0000 7fcacc73e640 1 -- 172.21.15.111:0/3906684059 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac40cd3c0 con 0x7fcac415e580 2024-09-13T17:57:22.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.737+0000 7fcacc73e640 1 -- 172.21.15.111:0/3906684059 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac40ac6f0 con 0x7fcac415e580 2024-09-13T17:57:22.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.738+0000 7fcabaffd640 1 -- 172.21.15.111:0/3906684059 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcabc013790 con 0x7fcac415e580 2024-09-13T17:57:22.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.738+0000 7fcabaffd640 1 --2- 172.21.15.111:0/3906684059 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcaa806ac00 0x7fcaa806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:22.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.738+0000 7fcabaffd640 1 -- 172.21.15.111:0/3906684059 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcabc0895a0 con 0x7fcac415e580 2024-09-13T17:57:22.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.738+0000 7fcaca4da640 1 --2- 172.21.15.111:0/3906684059 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcaa806ac00 0x7fcaa806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:22.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.739+0000 7fcaca4da640 1 --2- 172.21.15.111:0/3906684059 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcaa806ac00 0x7fcaa806d0c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009a60 tx=0x7fcac0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:22.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.743+0000 7fcabaffd640 1 -- 172.21.15.111:0/3906684059 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fcabc054ad0 con 0x7fcac415e580 2024-09-13T17:57:22.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.938+0000 7fcacc73e640 1 -- 172.21.15.111:0/3906684059 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcac40b36a0 con 0x7fcac415e580 2024-09-13T17:57:22.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.938+0000 7fcabaffd640 1 -- 172.21.15.111:0/3906684059 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3536 (secure 0 0 0) 0x7fcabc0580f0 con 0x7fcac415e580 2024-09-13T17:57:22.941 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:22.941 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4001406804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":4001406804},{"type":"v1","addr":"172.21.15.188:6835","nonce":4001406804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2565466103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2565466103},{"type":"v1","addr":"172.21.15.111:6837","nonce":2565466103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/1252690799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1252690799},{"type":"v1","addr":"172.21.15.111:6836","nonce":1252690799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1266310843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1266310843},{"type":"v1","addr":"172.21.15.188:6834","nonce":1266310843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-09-13T17:57:22.941 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 3 2024-09-13T17:57:22.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.940+0000 7fcacc73e640 1 -- 172.21.15.111:0/3906684059 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcaa806ac00 msgr2=0x7fcaa806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:22.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.940+0000 7fcacc73e640 1 --2- 172.21.15.111:0/3906684059 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcaa806ac00 0x7fcaa806d0c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009a60 tx=0x7fcac0009190 comp rx=0 tx=0).stop 2024-09-13T17:57:22.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.940+0000 7fcacc73e640 1 -- 172.21.15.111:0/3906684059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac415e580 msgr2=0x7fcac415ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:22.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.940+0000 7fcacc73e640 1 --2- 172.21.15.111:0/3906684059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac415e580 0x7fcac415ea00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcac40ad3a0 tx=0x7fcabc00a600 comp rx=0 tx=0).stop 2024-09-13T17:57:22.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.940+0000 7fcacc73e640 1 -- 172.21.15.111:0/3906684059 shutdown_connections 2024-09-13T17:57:22.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.940+0000 7fcacc73e640 1 --2- 172.21.15.111:0/3906684059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac415e580 0x7fcac415ea00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.940+0000 7fcacc73e640 1 --2- 172.21.15.111:0/3906684059 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcaa806ac00 0x7fcaa806d0c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.940+0000 7fcacc73e640 1 --2- 172.21.15.111:0/3906684059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcac4151970 0x7fcac40cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.940+0000 7fcacc73e640 1 --2- 172.21.15.111:0/3906684059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcab00047c0 0x7fcac41486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.940+0000 7fcacc73e640 1 -- 172.21.15.111:0/3906684059 >> 172.21.15.111:0/3906684059 conn(0x7fcac40709f0 msgr2=0x7fcac40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:22.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.940+0000 7fcacc73e640 1 -- 172.21.15.111:0/3906684059 shutdown_connections 2024-09-13T17:57:22.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:22.940+0000 7fcacc73e640 1 -- 172.21.15.111:0/3906684059 wait complete. 2024-09-13T17:57:22.953 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:23.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.048+0000 7fa538353640 1 Processor -- start 2024-09-13T17:57:23.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.048+0000 7fa538353640 1 -- start start 2024-09-13T17:57:23.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.049+0000 7fa538353640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa53015c6e0 0x7fa5300b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.049+0000 7fa538353640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5300d07b0 0x7fa5300b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.049+0000 7fa538353640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5300d1840 con 0x7fa5300ccb20 2024-09-13T17:57:23.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.049+0000 7fa538353640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa53006bd20 con 0x7fa53015c6e0 2024-09-13T17:57:23.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.049+0000 7fa538353640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5300d0c30 con 0x7fa5300d07b0 2024-09-13T17:57:23.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.049+0000 7fa5368f0640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa5300ccb20 0x7fa530160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58032/0 (socket says 172.21.15.111:58032) 2024-09-13T17:57:23.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.049+0000 7fa5368f0640 1 -- 172.21.15.111:0/854949796 learned_addr learned my addr 172.21.15.111:0/854949796 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:23.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.049+0000 7fa5360ef640 1 --2- 172.21.15.111:0/854949796 >> v2:172.21.15.188:3300/0 conn(0x7fa53015c6e0 0x7fa5300b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.049+0000 7fa5358ee640 1 --2- 172.21.15.111:0/854949796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5300d07b0 0x7fa5300b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.049+0000 7fa5350ed640 1 -- 172.21.15.111:0/854949796 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128371618 0 0) 0x7fa5300d1840 con 0x7fa5300ccb20 2024-09-13T17:57:23.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.049+0000 7fa5350ed640 1 -- 172.21.15.111:0/854949796 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa51c003520 con 0x7fa5300ccb20 2024-09-13T17:57:23.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.050+0000 7fa5350ed640 1 -- 172.21.15.111:0/854949796 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1722309101 0 0) 0x7fa51c003520 con 0x7fa5300ccb20 2024-09-13T17:57:23.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.050+0000 7fa5360ef640 1 -- 172.21.15.111:0/854949796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5300d07b0 msgr2=0x7fa5300b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:23.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.050+0000 7fa5360ef640 1 --2- 172.21.15.111:0/854949796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5300d07b0 0x7fa5300b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.050+0000 7fa5360ef640 1 -- 172.21.15.111:0/854949796 >> v1:172.21.15.111:6789/0 conn(0x7fa5300ccb20 legacy=0x7fa530160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.050+0000 7fa5360ef640 1 -- 172.21.15.111:0/854949796 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5300d0f10 con 0x7fa53015c6e0 2024-09-13T17:57:23.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.050+0000 7fa5358ee640 1 --2- 172.21.15.111:0/854949796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5300d07b0 0x7fa5300b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:23.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.050+0000 7fa5360ef640 1 --2- 172.21.15.111:0/854949796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa53015c6e0 0x7fa5300b3f50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0097a0 tx=0x7fa52c02a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ec2897ed09cebe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:23.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.051+0000 7fa5350ed640 1 -- 172.21.15.111:0/854949796 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5280019f0 con 0x7fa53015c6e0 2024-09-13T17:57:23.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.051+0000 7fa5350ed640 1 -- 172.21.15.111:0/854949796 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c02ab10 con 0x7fa53015c6e0 2024-09-13T17:57:23.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.051+0000 7fa5350ed640 1 -- 172.21.15.111:0/854949796 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c02aea0 con 0x7fa53015c6e0 2024-09-13T17:57:23.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.051+0000 7fa538353640 1 -- 172.21.15.111:0/854949796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa53015c6e0 msgr2=0x7fa5300b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.051+0000 7fa538353640 1 --2- 172.21.15.111:0/854949796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa53015c6e0 0x7fa5300b3f50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0097a0 tx=0x7fa52c02a080 comp rx=0 tx=0).stop 2024-09-13T17:57:23.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.051+0000 7fa538353640 1 -- 172.21.15.111:0/854949796 shutdown_connections 2024-09-13T17:57:23.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.051+0000 7fa538353640 1 --2- 172.21.15.111:0/854949796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5300d07b0 0x7fa5300b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.051+0000 7fa538353640 1 --2- 172.21.15.111:0/854949796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa53015c6e0 0x7fa5300b3f50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.051+0000 7fa538353640 1 -- 172.21.15.111:0/854949796 >> 172.21.15.111:0/854949796 conn(0x7fa530089f60 msgr2=0x7fa5300d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:23.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.051+0000 7fa538353640 1 -- 172.21.15.111:0/854949796 shutdown_connections 2024-09-13T17:57:23.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.051+0000 7fa538353640 1 -- 172.21.15.111:0/854949796 wait complete. 2024-09-13T17:57:23.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.052+0000 7fa538353640 1 Processor -- start 2024-09-13T17:57:23.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.052+0000 7fa538353640 1 -- start start 2024-09-13T17:57:23.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.053+0000 7fa538353640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5300ccb20 0x7fa5301540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.053+0000 7fa538353640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5300d07b0 0x7fa53015f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.053+0000 7fa538353640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa53015c6e0 0x7fa5300b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.053+0000 7fa538353640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa530156110 con 0x7fa5300ccb20 2024-09-13T17:57:23.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.053+0000 7fa538353640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5301545e0 con 0x7fa53015c6e0 2024-09-13T17:57:23.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.053+0000 7fa538353640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa530155500 con 0x7fa5300d07b0 2024-09-13T17:57:23.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.053+0000 7fa5360ef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5300ccb20 0x7fa5301540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.053+0000 7fa5360ef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5300ccb20 0x7fa5301540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36330/0 (socket says 172.21.15.111:36330) 2024-09-13T17:57:23.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.053+0000 7fa5360ef640 1 -- 172.21.15.111:0/3721756887 learned_addr learned my addr 172.21.15.111:0/3721756887 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:23.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.053+0000 7fa5368f0640 1 --2- 172.21.15.111:0/3721756887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa53015c6e0 0x7fa5300b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.053+0000 7fa5358ee640 1 --2- 172.21.15.111:0/3721756887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5300d07b0 0x7fa53015f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.053+0000 7fa5360ef640 1 -- 172.21.15.111:0/3721756887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5300d07b0 msgr2=0x7fa53015f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.054+0000 7fa5360ef640 1 --2- 172.21.15.111:0/3721756887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5300d07b0 0x7fa53015f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.054+0000 7fa5360ef640 1 -- 172.21.15.111:0/3721756887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa53015c6e0 msgr2=0x7fa5300b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.054+0000 7fa5360ef640 1 --2- 172.21.15.111:0/3721756887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa53015c6e0 0x7fa5300b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.054+0000 7fa5360ef640 1 -- 172.21.15.111:0/3721756887 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa520009400 con 0x7fa5300ccb20 2024-09-13T17:57:23.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.054+0000 7fa5368f0640 1 --2- 172.21.15.111:0/3721756887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa53015c6e0 0x7fa5300b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:23.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.054+0000 7fa5360ef640 1 --2- 172.21.15.111:0/3721756887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5300ccb20 0x7fa5301540a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa52c035040 tx=0x7fa52c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:23.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.054+0000 7fa526ffd640 1 -- 172.21.15.111:0/3721756887 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c03b810 con 0x7fa5300ccb20 2024-09-13T17:57:23.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.054+0000 7fa526ffd640 1 -- 172.21.15.111:0/3721756887 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c03b970 con 0x7fa5300ccb20 2024-09-13T17:57:23.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.054+0000 7fa538353640 1 -- 172.21.15.111:0/3721756887 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52c009400 con 0x7fa5300ccb20 2024-09-13T17:57:23.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.054+0000 7fa526ffd640 1 -- 172.21.15.111:0/3721756887 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c03bd20 con 0x7fa5300ccb20 2024-09-13T17:57:23.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.054+0000 7fa538353640 1 -- 172.21.15.111:0/3721756887 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa530155ac0 con 0x7fa5300ccb20 2024-09-13T17:57:23.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.055+0000 7fa538353640 1 -- 172.21.15.111:0/3721756887 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa530066dc0 con 0x7fa5300ccb20 2024-09-13T17:57:23.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.055+0000 7fa526ffd640 1 -- 172.21.15.111:0/3721756887 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa52c039040 con 0x7fa5300ccb20 2024-09-13T17:57:23.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.056+0000 7fa526ffd640 1 --2- 172.21.15.111:0/3721756887 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa51406ac00 0x7fa51406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.056+0000 7fa526ffd640 1 -- 172.21.15.111:0/3721756887 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa52c0a96f0 con 0x7fa5300ccb20 2024-09-13T17:57:23.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.056+0000 7fa5358ee640 1 --2- 172.21.15.111:0/3721756887 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa51406ac00 0x7fa51406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.056+0000 7fa5358ee640 1 --2- 172.21.15.111:0/3721756887 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa51406ac00 0x7fa51406d0c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa53015bf10 tx=0x7fa520008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:23.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.060+0000 7fa526ffd640 1 -- 172.21.15.111:0/3721756887 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fa52c074d40 con 0x7fa5300ccb20 2024-09-13T17:57:23.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.221+0000 7fa538353640 1 -- 172.21.15.111:0/3721756887 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5300b4c80 con 0x7fa5300ccb20 2024-09-13T17:57:23.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.222+0000 7fa526ffd640 1 -- 172.21.15.111:0/3721756887 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12910 (secure 0 0 0) 0x7fa52c037090 con 0x7fa5300ccb20 2024-09-13T17:57:23.223 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:23.224 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":16,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:16.526252+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:23.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.223+0000 7fa538353640 1 -- 172.21.15.111:0/3721756887 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa51406ac00 msgr2=0x7fa51406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.223+0000 7fa538353640 1 --2- 172.21.15.111:0/3721756887 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa51406ac00 0x7fa51406d0c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa53015bf10 tx=0x7fa520008040 comp rx=0 tx=0).stop 2024-09-13T17:57:23.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.223+0000 7fa538353640 1 -- 172.21.15.111:0/3721756887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5300ccb20 msgr2=0x7fa5301540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.223+0000 7fa538353640 1 --2- 172.21.15.111:0/3721756887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5300ccb20 0x7fa5301540a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa52c035040 tx=0x7fa52c0046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.223+0000 7fa538353640 1 -- 172.21.15.111:0/3721756887 shutdown_connections 2024-09-13T17:57:23.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.223+0000 7fa538353640 1 --2- 172.21.15.111:0/3721756887 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa51406ac00 0x7fa51406d0c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.223+0000 7fa538353640 1 --2- 172.21.15.111:0/3721756887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa53015c6e0 0x7fa5300b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.223+0000 7fa538353640 1 --2- 172.21.15.111:0/3721756887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5300d07b0 0x7fa53015f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.223+0000 7fa538353640 1 --2- 172.21.15.111:0/3721756887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5300ccb20 0x7fa5301540a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.223+0000 7fa538353640 1 -- 172.21.15.111:0/3721756887 >> 172.21.15.111:0/3721756887 conn(0x7fa530089f60 msgr2=0x7fa53006c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:23.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.224+0000 7fa538353640 1 -- 172.21.15.111:0/3721756887 shutdown_connections 2024-09-13T17:57:23.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.224+0000 7fa538353640 1 -- 172.21.15.111:0/3721756887 wait complete. 2024-09-13T17:57:23.236 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:57:23.237 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:57:23.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.335+0000 7ff88b05a640 1 Processor -- start 2024-09-13T17:57:23.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.335+0000 7ff88b05a640 1 -- start start 2024-09-13T17:57:23.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.335+0000 7ff88b05a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff88415fbc0 0x7ff8840cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.335+0000 7ff88b05a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff8840cd6c0 0x7ff8840d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.335+0000 7ff88b05a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88406b6c0 con 0x7ff8840d2810 2024-09-13T17:57:23.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.335+0000 7ff88b05a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8840ce500 con 0x7ff88415fbc0 2024-09-13T17:57:23.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.335+0000 7ff88b05a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8840d2c80 con 0x7ff8840cd6c0 2024-09-13T17:57:23.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.336+0000 7ff888df6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff88415fbc0 0x7ff8840cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.336+0000 7ff888df6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff88415fbc0 0x7ff8840cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36340/0 (socket says 172.21.15.111:36340) 2024-09-13T17:57:23.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.336+0000 7ff888df6640 1 -- 172.21.15.111:0/32063406 learned_addr learned my addr 172.21.15.111:0/32063406 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:23.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.336+0000 7ff883fff640 1 --2- 172.21.15.111:0/32063406 >> v2:172.21.15.188:3300/0 conn(0x7ff8840cd6c0 0x7ff8840d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.336+0000 7ff888df6640 1 -- 172.21.15.111:0/32063406 >> v2:172.21.15.188:3300/0 conn(0x7ff8840cd6c0 msgr2=0x7ff8840d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:23.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.336+0000 7ff888df6640 1 --2- 172.21.15.111:0/32063406 >> v2:172.21.15.188:3300/0 conn(0x7ff8840cd6c0 0x7ff8840d22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.336+0000 7ff888df6640 1 -- 172.21.15.111:0/32063406 >> v1:172.21.15.188:6789/0 conn(0x7ff8840d2810 legacy=0x7ff8840cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.336+0000 7ff888df6640 1 -- 172.21.15.111:0/32063406 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8840cdc60 con 0x7ff88415fbc0 2024-09-13T17:57:23.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.336+0000 7ff883fff640 1 --2- 172.21.15.111:0/32063406 >> v2:172.21.15.188:3300/0 conn(0x7ff8840cd6c0 0x7ff8840d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:23.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.337+0000 7ff888df6640 1 --2- 172.21.15.111:0/32063406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff88415fbc0 0x7ff8840cd180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff88408b980 tx=0x7ff87802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373475a1cd5ca2c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:23.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.337+0000 7ff8837fe640 1 -- 172.21.15.111:0/32063406 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878032440 con 0x7ff88415fbc0 2024-09-13T17:57:23.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.337+0000 7ff8837fe640 1 -- 172.21.15.111:0/32063406 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8780325a0 con 0x7ff88415fbc0 2024-09-13T17:57:23.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.337+0000 7ff8837fe640 1 -- 172.21.15.111:0/32063406 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878032930 con 0x7ff88415fbc0 2024-09-13T17:57:23.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.337+0000 7ff88b05a640 1 -- 172.21.15.111:0/32063406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff88415fbc0 msgr2=0x7ff8840cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.337+0000 7ff88b05a640 1 --2- 172.21.15.111:0/32063406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff88415fbc0 0x7ff8840cd180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff88408b980 tx=0x7ff87802a270 comp rx=0 tx=0).stop 2024-09-13T17:57:23.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.337+0000 7ff88b05a640 1 -- 172.21.15.111:0/32063406 shutdown_connections 2024-09-13T17:57:23.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.337+0000 7ff88b05a640 1 --2- 172.21.15.111:0/32063406 >> v2:172.21.15.188:3300/0 conn(0x7ff8840cd6c0 0x7ff8840d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.337+0000 7ff88b05a640 1 --2- 172.21.15.111:0/32063406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff88415fbc0 0x7ff8840cd180 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.337+0000 7ff88b05a640 1 -- 172.21.15.111:0/32063406 >> 172.21.15.111:0/32063406 conn(0x7ff88409b830 msgr2=0x7ff884089f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:23.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.337+0000 7ff88b05a640 1 -- 172.21.15.111:0/32063406 shutdown_connections 2024-09-13T17:57:23.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.337+0000 7ff88b05a640 1 -- 172.21.15.111:0/32063406 wait complete. 2024-09-13T17:57:23.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.339+0000 7ff88b05a640 1 Processor -- start 2024-09-13T17:57:23.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.339+0000 7ff88b05a640 1 -- start start 2024-09-13T17:57:23.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.339+0000 7ff88b05a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8840cd6c0 0x7ff8841563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.339+0000 7ff88b05a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8840d2810 0x7ff8840b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.339+0000 7ff88b05a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff88415fbc0 0x7ff884148a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.339+0000 7ff88b05a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840b3930 con 0x7ff8840cd6c0 2024-09-13T17:57:23.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.339+0000 7ff88b05a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840b3c10 con 0x7ff8840d2810 2024-09-13T17:57:23.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.339+0000 7ff88b05a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840b3ef0 con 0x7ff88415fbc0 2024-09-13T17:57:23.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.339+0000 7ff888df6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8840cd6c0 0x7ff8841563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.340+0000 7ff888df6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8840cd6c0 0x7ff8841563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36354/0 (socket says 172.21.15.111:36354) 2024-09-13T17:57:23.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.340+0000 7ff888df6640 1 -- 172.21.15.111:0/3576090579 learned_addr learned my addr 172.21.15.111:0/3576090579 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:23.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.340+0000 7ff883fff640 1 --2- 172.21.15.111:0/3576090579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8840d2810 0x7ff8840b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.340+0000 7ff8895f7640 1 --2- 172.21.15.111:0/3576090579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff88415fbc0 0x7ff884148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.340+0000 7ff888df6640 1 -- 172.21.15.111:0/3576090579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff88415fbc0 msgr2=0x7ff884148a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.340+0000 7ff888df6640 1 --2- 172.21.15.111:0/3576090579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff88415fbc0 0x7ff884148a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.340+0000 7ff888df6640 1 -- 172.21.15.111:0/3576090579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8840d2810 msgr2=0x7ff8840b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.340+0000 7ff888df6640 1 --2- 172.21.15.111:0/3576090579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8840d2810 0x7ff8840b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.340+0000 7ff888df6640 1 -- 172.21.15.111:0/3576090579 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8780094d0 con 0x7ff8840cd6c0 2024-09-13T17:57:23.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.340+0000 7ff883fff640 1 --2- 172.21.15.111:0/3576090579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8840d2810 0x7ff8840b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:23.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.340+0000 7ff8895f7640 1 --2- 172.21.15.111:0/3576090579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff88415fbc0 0x7ff884148a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:23.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.340+0000 7ff888df6640 1 --2- 172.21.15.111:0/3576090579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8840cd6c0 0x7ff8841563b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff87802a2b0 tx=0x7ff8780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:23.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.340+0000 7ff8817fa640 1 -- 172.21.15.111:0/3576090579 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878032440 con 0x7ff8840cd6c0 2024-09-13T17:57:23.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.340+0000 7ff8817fa640 1 -- 172.21.15.111:0/3576090579 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff878030ba0 con 0x7ff8840cd6c0 2024-09-13T17:57:23.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.340+0000 7ff88b05a640 1 -- 172.21.15.111:0/3576090579 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8840b41d0 con 0x7ff8840cd6c0 2024-09-13T17:57:23.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.340+0000 7ff8817fa640 1 -- 172.21.15.111:0/3576090579 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87803d440 con 0x7ff8840cd6c0 2024-09-13T17:57:23.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.340+0000 7ff88b05a640 1 -- 172.21.15.111:0/3576090579 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8841568f0 con 0x7ff8840cd6c0 2024-09-13T17:57:23.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.341+0000 7ff88b05a640 1 -- 172.21.15.111:0/3576090579 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff884156a80 con 0x7ff8840cd6c0 2024-09-13T17:57:23.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.342+0000 7ff8817fa640 1 -- 172.21.15.111:0/3576090579 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff878039040 con 0x7ff8840cd6c0 2024-09-13T17:57:23.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.342+0000 7ff8817fa640 1 --2- 172.21.15.111:0/3576090579 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff86806ac00 0x7ff86806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.342+0000 7ff8817fa640 1 -- 172.21.15.111:0/3576090579 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff8780a9b30 con 0x7ff8840cd6c0 2024-09-13T17:57:23.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.342+0000 7ff883fff640 1 --2- 172.21.15.111:0/3576090579 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff86806ac00 0x7ff86806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.343+0000 7ff883fff640 1 --2- 172.21.15.111:0/3576090579 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff86806ac00 0x7ff86806d0c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff870009780 tx=0x7ff870008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:23.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.346+0000 7ff8817fa640 1 -- 172.21.15.111:0/3576090579 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff8780750f0 con 0x7ff8840cd6c0 2024-09-13T17:57:23.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.504+0000 7ff88b05a640 1 -- 172.21.15.111:0/3576090579 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff8840af090 con 0x7ff8840cd6c0 2024-09-13T17:57:23.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.505+0000 7ff8817fa640 1 -- 172.21.15.111:0/3576090579 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff8780370a0 con 0x7ff8840cd6c0 2024-09-13T17:57:23.507 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:57:23.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.507+0000 7ff88b05a640 1 -- 172.21.15.111:0/3576090579 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff86806ac00 msgr2=0x7ff86806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.507+0000 7ff88b05a640 1 --2- 172.21.15.111:0/3576090579 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff86806ac00 0x7ff86806d0c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff870009780 tx=0x7ff870008040 comp rx=0 tx=0).stop 2024-09-13T17:57:23.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.507+0000 7ff88b05a640 1 -- 172.21.15.111:0/3576090579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8840cd6c0 msgr2=0x7ff8841563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.507+0000 7ff88b05a640 1 --2- 172.21.15.111:0/3576090579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8840cd6c0 0x7ff8841563b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff87802a2b0 tx=0x7ff8780046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.507+0000 7ff88b05a640 1 -- 172.21.15.111:0/3576090579 shutdown_connections 2024-09-13T17:57:23.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.507+0000 7ff88b05a640 1 --2- 172.21.15.111:0/3576090579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff88415fbc0 0x7ff884148a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.507+0000 7ff88b05a640 1 --2- 172.21.15.111:0/3576090579 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff86806ac00 0x7ff86806d0c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.507+0000 7ff88b05a640 1 --2- 172.21.15.111:0/3576090579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8840d2810 0x7ff8840b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.507+0000 7ff88b05a640 1 --2- 172.21.15.111:0/3576090579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8840cd6c0 0x7ff8841563b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.507+0000 7ff88b05a640 1 -- 172.21.15.111:0/3576090579 >> 172.21.15.111:0/3576090579 conn(0x7ff88409b830 msgr2=0x7ff8840ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:23.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.507+0000 7ff88b05a640 1 -- 172.21.15.111:0/3576090579 shutdown_connections 2024-09-13T17:57:23.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.507+0000 7ff88b05a640 1 -- 172.21.15.111:0/3576090579 wait complete. 2024-09-13T17:57:23.519 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T17:57:23.616 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.614+0000 7f279bcb1640 1 Processor -- start 2024-09-13T17:57:23.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.615+0000 7f279bcb1640 1 -- start start 2024-09-13T17:57:23.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.615+0000 7f279bcb1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f279415a3a0 0x7f27940cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.615+0000 7f279bcb1640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279415d480 con 0x7f279415d910 2024-09-13T17:57:23.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.615+0000 7f279bcb1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27940ac1f0 con 0x7f2794070ad0 2024-09-13T17:57:23.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.615+0000 7f279bcb1640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2794071280 con 0x7f279415a3a0 2024-09-13T17:57:23.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.616+0000 7f2799a4d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f279415a3a0 0x7f27940cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.616+0000 7f279924c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f279415d910 0x7f27940703c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58048/0 (socket says 172.21.15.111:58048) 2024-09-13T17:57:23.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.616+0000 7f2799a4d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f279415a3a0 0x7f27940cd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36358/0 (socket says 172.21.15.111:36358) 2024-09-13T17:57:23.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.616+0000 7f2799a4d640 1 -- 172.21.15.111:0/2469367451 learned_addr learned my addr 172.21.15.111:0/2469367451 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:23.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.616+0000 7f2799a4d640 1 -- 172.21.15.111:0/2469367451 >> v1:172.21.15.188:6789/0 conn(0x7f2794070ad0 legacy=0x7f2794160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:23.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.616+0000 7f2799a4d640 1 -- 172.21.15.111:0/2469367451 >> v1:172.21.15.111:6789/0 conn(0x7f279415d910 legacy=0x7f27940703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.616+0000 7f2799a4d640 1 -- 172.21.15.111:0/2469367451 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2794070fa0 con 0x7f279415a3a0 2024-09-13T17:57:23.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.616+0000 7f2799a4d640 1 --2- 172.21.15.111:0/2469367451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f279415a3a0 0x7f27940cd650 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2790009e50 tx=0x7f279002a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969ac7e23d091500 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:23.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.617+0000 7f2798a4b640 1 -- 172.21.15.111:0/2469367451 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2790031600 con 0x7f279415a3a0 2024-09-13T17:57:23.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.617+0000 7f2798a4b640 1 -- 172.21.15.111:0/2469367451 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279002d050 con 0x7f279415a3a0 2024-09-13T17:57:23.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.617+0000 7f2798a4b640 1 -- 172.21.15.111:0/2469367451 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2790031980 con 0x7f279415a3a0 2024-09-13T17:57:23.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.617+0000 7f279bcb1640 1 -- 172.21.15.111:0/2469367451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f279415a3a0 msgr2=0x7f27940cd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.617+0000 7f279bcb1640 1 --2- 172.21.15.111:0/2469367451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f279415a3a0 0x7f27940cd650 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2790009e50 tx=0x7f279002a760 comp rx=0 tx=0).stop 2024-09-13T17:57:23.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.617+0000 7f279bcb1640 1 -- 172.21.15.111:0/2469367451 shutdown_connections 2024-09-13T17:57:23.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.617+0000 7f279bcb1640 1 --2- 172.21.15.111:0/2469367451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f279415a3a0 0x7f27940cd650 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.617+0000 7f279bcb1640 1 -- 172.21.15.111:0/2469367451 >> 172.21.15.111:0/2469367451 conn(0x7f2794069ed0 msgr2=0x7f279406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:23.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.617+0000 7f279bcb1640 1 -- 172.21.15.111:0/2469367451 shutdown_connections 2024-09-13T17:57:23.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.617+0000 7f279bcb1640 1 -- 172.21.15.111:0/2469367451 wait complete. 2024-09-13T17:57:23.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.618+0000 7f279bcb1640 1 Processor -- start 2024-09-13T17:57:23.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.618+0000 7f279bcb1640 1 -- start start 2024-09-13T17:57:23.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.619+0000 7f279bcb1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2794070ad0 0x7f27941e6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.619+0000 7f279bcb1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f279415a3a0 0x7f27941e56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.619+0000 7f279bcb1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279415d910 0x7f279414f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.619+0000 7f279bcb1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2794156640 con 0x7f279415a3a0 2024-09-13T17:57:23.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.619+0000 7f279bcb1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27941e5c30 con 0x7f2794070ad0 2024-09-13T17:57:23.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.619+0000 7f279bcb1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27941e6b10 con 0x7f279415d910 2024-09-13T17:57:23.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.619+0000 7f279924c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f279415a3a0 0x7f27941e56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.619+0000 7f279924c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f279415a3a0 0x7f27941e56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36368/0 (socket says 172.21.15.111:36368) 2024-09-13T17:57:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.619+0000 7f279924c640 1 -- 172.21.15.111:0/2403301310 learned_addr learned my addr 172.21.15.111:0/2403301310 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.620+0000 7f2799a4d640 1 --2- 172.21.15.111:0/2403301310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2794070ad0 0x7f27941e6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.620+0000 7f279a24e640 1 --2- 172.21.15.111:0/2403301310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279415d910 0x7f279414f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.620+0000 7f279924c640 1 -- 172.21.15.111:0/2403301310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279415d910 msgr2=0x7f279414f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.620+0000 7f279924c640 1 --2- 172.21.15.111:0/2403301310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279415d910 0x7f279414f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.620+0000 7f279924c640 1 -- 172.21.15.111:0/2403301310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2794070ad0 msgr2=0x7f27941e6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.620+0000 7f279924c640 1 --2- 172.21.15.111:0/2403301310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2794070ad0 0x7f27941e6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.620+0000 7f279924c640 1 -- 172.21.15.111:0/2403301310 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2790009b30 con 0x7f279415a3a0 2024-09-13T17:57:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.620+0000 7f2799a4d640 1 --2- 172.21.15.111:0/2403301310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2794070ad0 0x7f27941e6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.620+0000 7f279a24e640 1 --2- 172.21.15.111:0/2403301310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279415d910 0x7f279414f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.620+0000 7f279924c640 1 --2- 172.21.15.111:0/2403301310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f279415a3a0 0x7f27941e56f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f278400c3e0 tx=0x7f278400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.620+0000 7f278a7fc640 1 -- 172.21.15.111:0/2403301310 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27840074e0 con 0x7f279415a3a0 2024-09-13T17:57:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.620+0000 7f278a7fc640 1 -- 172.21.15.111:0/2403301310 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2784007640 con 0x7f279415a3a0 2024-09-13T17:57:23.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.620+0000 7f279bcb1640 1 -- 172.21.15.111:0/2403301310 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2794156820 con 0x7f279415a3a0 2024-09-13T17:57:23.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.620+0000 7f278a7fc640 1 -- 172.21.15.111:0/2403301310 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27840049e0 con 0x7f279415a3a0 2024-09-13T17:57:23.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.620+0000 7f279bcb1640 1 -- 172.21.15.111:0/2403301310 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2794156de0 con 0x7f279415a3a0 2024-09-13T17:57:23.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.621+0000 7f279bcb1640 1 -- 172.21.15.111:0/2403301310 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27940cd650 con 0x7f279415a3a0 2024-09-13T17:57:23.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.621+0000 7f278a7fc640 1 -- 172.21.15.111:0/2403301310 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2784004b40 con 0x7f279415a3a0 2024-09-13T17:57:23.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.622+0000 7f278a7fc640 1 --2- 172.21.15.111:0/2403301310 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f277806ac00 0x7f277806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.622+0000 7f278a7fc640 1 -- 172.21.15.111:0/2403301310 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2784088a30 con 0x7f279415a3a0 2024-09-13T17:57:23.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.622+0000 7f2799a4d640 1 --2- 172.21.15.111:0/2403301310 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f277806ac00 0x7f277806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.622+0000 7f2799a4d640 1 --2- 172.21.15.111:0/2403301310 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f277806ac00 0x7f277806d0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2790033000 tx=0x7f2790009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:23.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.626+0000 7f278a7fc640 1 -- 172.21.15.111:0/2403301310 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f2784053f60 con 0x7f279415a3a0 2024-09-13T17:57:23.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.787+0000 7f279bcb1640 1 -- 172.21.15.111:0/2403301310 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f279406ad70 con 0x7f279415a3a0 2024-09-13T17:57:23.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.788+0000 7f278a7fc640 1 -- 172.21.15.111:0/2403301310 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2784057580 con 0x7f279415a3a0 2024-09-13T17:57:23.790 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:23.790 INFO:teuthology.orchestra.run.smithi111.stdout:[ 2024-09-13T17:57:23.790 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:57:23.790 INFO:teuthology.orchestra.run.smithi111.stdout: "poolnum": 1, 2024-09-13T17:57:23.790 INFO:teuthology.orchestra.run.smithi111.stdout: "poolname": ".mgr" 2024-09-13T17:57:23.790 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:57:23.791 INFO:teuthology.orchestra.run.smithi111.stdout:] 2024-09-13T17:57:23.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.790+0000 7f279bcb1640 1 -- 172.21.15.111:0/2403301310 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f277806ac00 msgr2=0x7f277806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.790+0000 7f279bcb1640 1 --2- 172.21.15.111:0/2403301310 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f277806ac00 0x7f277806d0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2790033000 tx=0x7f2790009000 comp rx=0 tx=0).stop 2024-09-13T17:57:23.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.790+0000 7f279bcb1640 1 -- 172.21.15.111:0/2403301310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f279415a3a0 msgr2=0x7f27941e56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.790+0000 7f279bcb1640 1 --2- 172.21.15.111:0/2403301310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f279415a3a0 0x7f27941e56f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f278400c3e0 tx=0x7f278400c8a0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.790+0000 7f279bcb1640 1 -- 172.21.15.111:0/2403301310 shutdown_connections 2024-09-13T17:57:23.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.790+0000 7f279bcb1640 1 --2- 172.21.15.111:0/2403301310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279415d910 0x7f279414f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.790+0000 7f279bcb1640 1 --2- 172.21.15.111:0/2403301310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f279415a3a0 0x7f27941e56f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.790+0000 7f279bcb1640 1 --2- 172.21.15.111:0/2403301310 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f277806ac00 0x7f277806d0c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.790+0000 7f279bcb1640 1 --2- 172.21.15.111:0/2403301310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2794070ad0 0x7f27941e6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.790+0000 7f279bcb1640 1 -- 172.21.15.111:0/2403301310 >> 172.21.15.111:0/2403301310 conn(0x7f2794069ed0 msgr2=0x7f279406cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:23.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.790+0000 7f279bcb1640 1 -- 172.21.15.111:0/2403301310 shutdown_connections 2024-09-13T17:57:23.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.790+0000 7f279bcb1640 1 -- 172.21.15.111:0/2403301310 wait complete. 2024-09-13T17:57:23.802 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T17:57:23.802 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T17:57:23.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.901+0000 7f1ee665a640 1 Processor -- start 2024-09-13T17:57:23.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.901+0000 7f1ee665a640 1 -- start start 2024-09-13T17:57:23.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1ee665a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ee00b2880 0x7f1ee0163690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1ee665a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee00b2d10 con 0x7f1ee006c8a0 2024-09-13T17:57:23.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1ee665a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee00b2ff0 con 0x7f1ee0158b00 2024-09-13T17:57:23.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1ee665a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee0070310 con 0x7f1ee00b2880 2024-09-13T17:57:23.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1edf7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1ee006c8a0 0x7f1ee00b1160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58050/0 (socket says 172.21.15.111:58050) 2024-09-13T17:57:23.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1edf7fe640 1 -- 172.21.15.111:0/3970524612 learned_addr learned my addr 172.21.15.111:0/3970524612 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:23.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1edeffd640 1 -- 172.21.15.111:0/3970524612 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816280037 0 0) 0x7f1ee00b2d10 con 0x7f1ee006c8a0 2024-09-13T17:57:23.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1ee4bf7640 1 --2- 172.21.15.111:0/3970524612 >> v2:172.21.15.188:3300/0 conn(0x7f1ee00b2880 0x7f1ee0163690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1edeffd640 1 -- 172.21.15.111:0/3970524612 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ec8003520 con 0x7f1ee006c8a0 2024-09-13T17:57:23.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1edeffd640 1 -- 172.21.15.111:0/3970524612 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 160624301 0 0) 0x7f1ec8003520 con 0x7f1ee006c8a0 2024-09-13T17:57:23.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1edeffd640 1 -- 172.21.15.111:0/3970524612 >> v2:172.21.15.188:3300/0 conn(0x7f1ee00b2880 msgr2=0x7f1ee0163690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:23.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1edeffd640 1 --2- 172.21.15.111:0/3970524612 >> v2:172.21.15.188:3300/0 conn(0x7f1ee00b2880 0x7f1ee0163690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1edeffd640 1 -- 172.21.15.111:0/3970524612 >> v1:172.21.15.188:6789/0 conn(0x7f1ee0158b00 legacy=0x7f1ee006c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1edeffd640 1 -- 172.21.15.111:0/3970524612 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee00705f0 con 0x7f1ee006c8a0 2024-09-13T17:57:23.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1edeffd640 1 -- 172.21.15.111:0/3970524612 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f1ed4001670 con 0x7f1ee006c8a0 2024-09-13T17:57:23.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1edeffd640 1 -- 172.21.15.111:0/3970524612 >> v1:172.21.15.111:6789/0 conn(0x7f1ee006c8a0 legacy=0x7f1ee00b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1edeffd640 1 --2- 172.21.15.111:0/3970524612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec8003970 0x7f1ec8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1edeffd640 1 --2- 172.21.15.111:0/3970524612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ec80047c0 0x7f1ec8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1edeffd640 1 --2- 172.21.15.111:0/3970524612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec8005180 0x7f1ec800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.902+0000 7f1edeffd640 1 -- 172.21.15.111:0/3970524612 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee0070310 con 0x7f1ec80047c0 2024-09-13T17:57:23.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.903+0000 7f1edeffd640 1 -- 172.21.15.111:0/3970524612 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec800c020 con 0x7f1ec8003970 2024-09-13T17:57:23.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.903+0000 7f1edeffd640 1 -- 172.21.15.111:0/3970524612 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec800c1e0 con 0x7f1ec8005180 2024-09-13T17:57:23.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.903+0000 7f1edf7fe640 1 --2- 172.21.15.111:0/3970524612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ec80047c0 0x7f1ec8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.903+0000 7f1edffff640 1 --2- 172.21.15.111:0/3970524612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec8003970 0x7f1ec8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.903+0000 7f1ee4bf7640 1 --2- 172.21.15.111:0/3970524612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec8005180 0x7f1ec800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.903+0000 7f1edf7fe640 1 -- 172.21.15.111:0/3970524612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec8005180 msgr2=0x7f1ec800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:23.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.903+0000 7f1edf7fe640 1 --2- 172.21.15.111:0/3970524612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec8005180 0x7f1ec800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.903+0000 7f1edf7fe640 1 -- 172.21.15.111:0/3970524612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec8003970 msgr2=0x7f1ec8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:23.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.903+0000 7f1edf7fe640 1 --2- 172.21.15.111:0/3970524612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec8003970 0x7f1ec8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.903+0000 7f1edf7fe640 1 -- 172.21.15.111:0/3970524612 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec8003520 con 0x7f1ec80047c0 2024-09-13T17:57:23.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.903+0000 7f1ee4bf7640 1 --2- 172.21.15.111:0/3970524612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec8005180 0x7f1ec800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:23.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.903+0000 7f1ee4bf7640 1 -- 172.21.15.111:0/3970524612 reap_dead start 2024-09-13T17:57:23.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.903+0000 7f1edf7fe640 1 --2- 172.21.15.111:0/3970524612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ec80047c0 0x7f1ec8004c40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400c240 tx=0x7f1ec80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7031098d8979cc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:23.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.904+0000 7f1edeffd640 1 -- 172.21.15.111:0/3970524612 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4005ad0 con 0x7f1ec80047c0 2024-09-13T17:57:23.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.904+0000 7f1edeffd640 1 -- 172.21.15.111:0/3970524612 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed402d070 con 0x7f1ec80047c0 2024-09-13T17:57:23.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.904+0000 7f1edeffd640 1 -- 172.21.15.111:0/3970524612 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4007070 con 0x7f1ec80047c0 2024-09-13T17:57:23.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.904+0000 7f1ee665a640 1 -- 172.21.15.111:0/3970524612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ec80047c0 msgr2=0x7f1ec8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.904+0000 7f1ee665a640 1 --2- 172.21.15.111:0/3970524612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ec80047c0 0x7f1ec8004c40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400c240 tx=0x7f1ec80057f0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.904+0000 7f1ee665a640 1 -- 172.21.15.111:0/3970524612 shutdown_connections 2024-09-13T17:57:23.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.904+0000 7f1ee665a640 1 --2- 172.21.15.111:0/3970524612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ec80047c0 0x7f1ec8004c40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.904+0000 7f1ee665a640 1 -- 172.21.15.111:0/3970524612 >> 172.21.15.111:0/3970524612 conn(0x7f1ee006a0b0 msgr2=0x7f1ee006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:23.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.904+0000 7f1ee665a640 1 -- 172.21.15.111:0/3970524612 shutdown_connections 2024-09-13T17:57:23.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.904+0000 7f1ee665a640 1 -- 172.21.15.111:0/3970524612 wait complete. 2024-09-13T17:57:23.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.905+0000 7f1ee665a640 1 Processor -- start 2024-09-13T17:57:23.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.906+0000 7f1ee665a640 1 -- start start 2024-09-13T17:57:23.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.906+0000 7f1ee665a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec80047c0 0x7f1ee009b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.906+0000 7f1ee665a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ee009bd80 0x7f1ee0098250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.906+0000 7f1ee665a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee00973f0 0x7f1ee0098790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.906+0000 7f1ee665a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee0099c70 con 0x7f1ee00973f0 2024-09-13T17:57:23.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.906+0000 7f1ee665a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee015bac0 con 0x7f1ee009bd80 2024-09-13T17:57:23.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.906+0000 7f1ee665a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee0099ec0 con 0x7f1ec80047c0 2024-09-13T17:57:23.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.906+0000 7f1ee4bf7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee00973f0 0x7f1ee0098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.906+0000 7f1ee4bf7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee00973f0 0x7f1ee0098790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36374/0 (socket says 172.21.15.111:36374) 2024-09-13T17:57:23.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.906+0000 7f1ee4bf7640 1 -- 172.21.15.111:0/3530214313 learned_addr learned my addr 172.21.15.111:0/3530214313 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:23.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.906+0000 7f1edf7fe640 1 --2- 172.21.15.111:0/3530214313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ee009bd80 0x7f1ee0098250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.906+0000 7f1edffff640 1 --2- 172.21.15.111:0/3530214313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec80047c0 0x7f1ee009b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.907+0000 7f1ee4bf7640 1 -- 172.21.15.111:0/3530214313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec80047c0 msgr2=0x7f1ee009b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.907+0000 7f1ee4bf7640 1 --2- 172.21.15.111:0/3530214313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec80047c0 0x7f1ee009b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.907+0000 7f1ee4bf7640 1 -- 172.21.15.111:0/3530214313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ee009bd80 msgr2=0x7f1ee0098250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.907+0000 7f1ee4bf7640 1 --2- 172.21.15.111:0/3530214313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ee009bd80 0x7f1ee0098250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.907+0000 7f1ee4bf7640 1 -- 172.21.15.111:0/3530214313 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee00950d0 con 0x7f1ee00973f0 2024-09-13T17:57:23.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.907+0000 7f1ee4bf7640 1 --2- 172.21.15.111:0/3530214313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee00973f0 0x7f1ee0098790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00cf390 tx=0x7f1ed8010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:23.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.907+0000 7f1edcff9640 1 -- 172.21.15.111:0/3530214313 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed800e050 con 0x7f1ee00973f0 2024-09-13T17:57:23.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.907+0000 7f1edcff9640 1 -- 172.21.15.111:0/3530214313 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed800f040 con 0x7f1ee00973f0 2024-09-13T17:57:23.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.907+0000 7f1edcff9640 1 -- 172.21.15.111:0/3530214313 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8003990 con 0x7f1ee00973f0 2024-09-13T17:57:23.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.907+0000 7f1ee665a640 1 -- 172.21.15.111:0/3530214313 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee0097940 con 0x7f1ee00973f0 2024-09-13T17:57:23.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.907+0000 7f1ee665a640 1 -- 172.21.15.111:0/3530214313 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee00ac050 con 0x7f1ee00973f0 2024-09-13T17:57:23.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.908+0000 7f1ee665a640 1 -- 172.21.15.111:0/3530214313 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee00b26a0 con 0x7f1ee00973f0 2024-09-13T17:57:23.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.908+0000 7f1edcff9640 1 -- 172.21.15.111:0/3530214313 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1ed8015440 con 0x7f1ee00973f0 2024-09-13T17:57:23.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.909+0000 7f1edcff9640 1 --2- 172.21.15.111:0/3530214313 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ecc06ac00 0x7f1ecc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:23.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.909+0000 7f1edffff640 1 --2- 172.21.15.111:0/3530214313 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ecc06ac00 0x7f1ecc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:23.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.909+0000 7f1edcff9640 1 -- 172.21.15.111:0/3530214313 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1ed8089510 con 0x7f1ee00973f0 2024-09-13T17:57:23.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.909+0000 7f1edffff640 1 --2- 172.21.15.111:0/3530214313 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ecc06ac00 0x7f1ecc06d0c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0000c00 tx=0x7f1ed0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:23.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:23.913+0000 7f1edcff9640 1 -- 172.21.15.111:0/3530214313 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f1ed8054b80 con 0x7f1ee00973f0 2024-09-13T17:57:24.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.071+0000 7f1ee665a640 1 -- 172.21.15.111:0/3530214313 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1ee0097ad0 con 0x7f1ee00973f0 2024-09-13T17:57:24.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.354+0000 7f1edcff9640 1 -- 172.21.15.111:0/3530214313 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1ed80581a0 con 0x7f1ee00973f0 2024-09-13T17:57:24.356 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' created 2024-09-13T17:57:24.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.357+0000 7f1ee665a640 1 -- 172.21.15.111:0/3530214313 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ecc06ac00 msgr2=0x7f1ecc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:24.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.357+0000 7f1ee665a640 1 --2- 172.21.15.111:0/3530214313 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ecc06ac00 0x7f1ecc06d0c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0000c00 tx=0x7f1ed0009040 comp rx=0 tx=0).stop 2024-09-13T17:57:24.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.357+0000 7f1ee665a640 1 -- 172.21.15.111:0/3530214313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee00973f0 msgr2=0x7f1ee0098790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:24.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.357+0000 7f1ee665a640 1 --2- 172.21.15.111:0/3530214313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee00973f0 0x7f1ee0098790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00cf390 tx=0x7f1ed8010590 comp rx=0 tx=0).stop 2024-09-13T17:57:24.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.358+0000 7f1ee665a640 1 -- 172.21.15.111:0/3530214313 shutdown_connections 2024-09-13T17:57:24.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.358+0000 7f1ee665a640 1 --2- 172.21.15.111:0/3530214313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee00973f0 0x7f1ee0098790 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:24.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.358+0000 7f1ee665a640 1 --2- 172.21.15.111:0/3530214313 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ecc06ac00 0x7f1ecc06d0c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:24.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.358+0000 7f1ee665a640 1 --2- 172.21.15.111:0/3530214313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ee009bd80 0x7f1ee0098250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:24.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.358+0000 7f1ee665a640 1 --2- 172.21.15.111:0/3530214313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec80047c0 0x7f1ee009b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:24.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.358+0000 7f1ee665a640 1 -- 172.21.15.111:0/3530214313 >> 172.21.15.111:0/3530214313 conn(0x7f1ee006a0b0 msgr2=0x7f1ee00709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:24.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.358+0000 7f1ee665a640 1 -- 172.21.15.111:0/3530214313 shutdown_connections 2024-09-13T17:57:24.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.358+0000 7f1ee665a640 1 -- 172.21.15.111:0/3530214313 wait complete. 2024-09-13T17:57:24.372 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-13T17:57:24.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.474+0000 7fae22a48640 1 Processor -- start 2024-09-13T17:57:24.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.474+0000 7fae22a48640 1 -- start start 2024-09-13T17:57:24.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.475+0000 7fae22a48640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fae1c0ccb40 0x7fae1c164fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:24.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.475+0000 7fae22a48640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1c0cbc50 con 0x7fae1c0adc30 2024-09-13T17:57:24.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.475+0000 7fae22a48640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1c070560 con 0x7fae1c13da60 2024-09-13T17:57:24.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.475+0000 7fae22a48640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1c0ae010 con 0x7fae1c0ccb40 2024-09-13T17:57:24.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.475+0000 7fae1bfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fae1c0adc30 0x7fae1c0cb540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58066/0 (socket says 172.21.15.111:58066) 2024-09-13T17:57:24.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.475+0000 7fae1bfff640 1 -- 172.21.15.111:0/2191657801 learned_addr learned my addr 172.21.15.111:0/2191657801 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:24.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.475+0000 7fae20fe5640 1 --2- 172.21.15.111:0/2191657801 >> v2:172.21.15.188:3300/0 conn(0x7fae1c0ccb40 0x7fae1c164fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:24.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.475+0000 7fae1b7fe640 1 -- 172.21.15.111:0/2191657801 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874059084 0 0) 0x7fae1c0cbc50 con 0x7fae1c0adc30 2024-09-13T17:57:24.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.475+0000 7fae1b7fe640 1 -- 172.21.15.111:0/2191657801 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae08003520 con 0x7fae1c0adc30 2024-09-13T17:57:24.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1b7fe640 1 -- 172.21.15.111:0/2191657801 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 226787026 0 0) 0x7fae08003520 con 0x7fae1c0adc30 2024-09-13T17:57:24.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1b7fe640 1 -- 172.21.15.111:0/2191657801 >> v2:172.21.15.188:3300/0 conn(0x7fae1c0ccb40 msgr2=0x7fae1c164fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:24.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1b7fe640 1 --2- 172.21.15.111:0/2191657801 >> v2:172.21.15.188:3300/0 conn(0x7fae1c0ccb40 0x7fae1c164fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:24.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1b7fe640 1 -- 172.21.15.111:0/2191657801 >> v1:172.21.15.188:6789/0 conn(0x7fae1c13da60 legacy=0x7fae1c0cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:24.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1b7fe640 1 -- 172.21.15.111:0/2191657801 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c070800 con 0x7fae1c0adc30 2024-09-13T17:57:24.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1b7fe640 1 -- 172.21.15.111:0/2191657801 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7fae0c001670 con 0x7fae1c0adc30 2024-09-13T17:57:24.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1b7fe640 1 -- 172.21.15.111:0/2191657801 >> v1:172.21.15.111:6789/0 conn(0x7fae1c0adc30 legacy=0x7fae1c0cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:24.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1b7fe640 1 --2- 172.21.15.111:0/2191657801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae08003970 0x7fae08003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:24.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1b7fe640 1 --2- 172.21.15.111:0/2191657801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae080047c0 0x7fae08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:24.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1b7fe640 1 --2- 172.21.15.111:0/2191657801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae08005180 0x7fae0800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:24.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1b7fe640 1 -- 172.21.15.111:0/2191657801 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c0ae010 con 0x7fae08003970 2024-09-13T17:57:24.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1b7fe640 1 -- 172.21.15.111:0/2191657801 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0800c020 con 0x7fae08005180 2024-09-13T17:57:24.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1b7fe640 1 -- 172.21.15.111:0/2191657801 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0800c1e0 con 0x7fae080047c0 2024-09-13T17:57:24.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1bfff640 1 --2- 172.21.15.111:0/2191657801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae08003970 0x7fae08003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:24.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae20fe5640 1 --2- 172.21.15.111:0/2191657801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae08005180 0x7fae0800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:24.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae13fff640 1 --2- 172.21.15.111:0/2191657801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae080047c0 0x7fae08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:24.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1bfff640 1 -- 172.21.15.111:0/2191657801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae080047c0 msgr2=0x7fae08004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:24.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1bfff640 1 --2- 172.21.15.111:0/2191657801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae080047c0 0x7fae08004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:24.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1bfff640 1 -- 172.21.15.111:0/2191657801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae08005180 msgr2=0x7fae0800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:24.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1bfff640 1 --2- 172.21.15.111:0/2191657801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae08005180 0x7fae0800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:24.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae1bfff640 1 -- 172.21.15.111:0/2191657801 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae08003520 con 0x7fae08003970 2024-09-13T17:57:24.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae20fe5640 1 --2- 172.21.15.111:0/2191657801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae08005180 0x7fae0800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:24.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.476+0000 7fae20fe5640 1 -- 172.21.15.111:0/2191657801 reap_dead start 2024-09-13T17:57:24.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.477+0000 7fae1bfff640 1 --2- 172.21.15.111:0/2191657801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae08003970 0x7fae08003d90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00c190 tx=0x7fae080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c7fda4ce9e8ce4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:24.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.477+0000 7fae1b7fe640 1 -- 172.21.15.111:0/2191657801 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c005ad0 con 0x7fae08003970 2024-09-13T17:57:24.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.477+0000 7fae1b7fe640 1 -- 172.21.15.111:0/2191657801 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0c02d070 con 0x7fae08003970 2024-09-13T17:57:24.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.477+0000 7fae1b7fe640 1 -- 172.21.15.111:0/2191657801 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c005e90 con 0x7fae08003970 2024-09-13T17:57:24.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.477+0000 7fae22a48640 1 -- 172.21.15.111:0/2191657801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae08003970 msgr2=0x7fae08003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:24.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.477+0000 7fae22a48640 1 --2- 172.21.15.111:0/2191657801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae08003970 0x7fae08003d90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00c190 tx=0x7fae080057f0 comp rx=0 tx=0).stop 2024-09-13T17:57:24.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.477+0000 7fae22a48640 1 -- 172.21.15.111:0/2191657801 shutdown_connections 2024-09-13T17:57:24.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.477+0000 7fae22a48640 1 --2- 172.21.15.111:0/2191657801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae08003970 0x7fae08003d90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:24.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.477+0000 7fae22a48640 1 -- 172.21.15.111:0/2191657801 >> 172.21.15.111:0/2191657801 conn(0x7fae1c065ff0 msgr2=0x7fae1c0b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:24.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.478+0000 7fae22a48640 1 -- 172.21.15.111:0/2191657801 shutdown_connections 2024-09-13T17:57:24.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.478+0000 7fae22a48640 1 -- 172.21.15.111:0/2191657801 wait complete. 2024-09-13T17:57:24.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.479+0000 7fae22a48640 1 Processor -- start 2024-09-13T17:57:24.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.479+0000 7fae22a48640 1 -- start start 2024-09-13T17:57:24.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.479+0000 7fae22a48640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae08003970 0x7fae1c0afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:24.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.479+0000 7fae22a48640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c142460 0x7fae1c1ef5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:24.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.479+0000 7fae22a48640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae1c1efb10 0x7fae1c15ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:24.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.479+0000 7fae22a48640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c139670 con 0x7fae1c142460 2024-09-13T17:57:24.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.479+0000 7fae22a48640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c1401e0 con 0x7fae1c1efb10 2024-09-13T17:57:24.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.479+0000 7fae22a48640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c142c20 con 0x7fae08003970 2024-09-13T17:57:24.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.480+0000 7fae13fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c142460 0x7fae1c1ef5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:24.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.480+0000 7fae13fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c142460 0x7fae1c1ef5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36400/0 (socket says 172.21.15.111:36400) 2024-09-13T17:57:24.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.480+0000 7fae13fff640 1 -- 172.21.15.111:0/617162130 learned_addr learned my addr 172.21.15.111:0/617162130 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:24.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.480+0000 7fae1bfff640 1 --2- 172.21.15.111:0/617162130 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae08003970 0x7fae1c0afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:24.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.480+0000 7fae20fe5640 1 --2- 172.21.15.111:0/617162130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae1c1efb10 0x7fae1c15ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:24.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.480+0000 7fae13fff640 1 -- 172.21.15.111:0/617162130 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae08003970 msgr2=0x7fae1c0afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:24.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.480+0000 7fae13fff640 1 --2- 172.21.15.111:0/617162130 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae08003970 0x7fae1c0afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:24.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.480+0000 7fae13fff640 1 -- 172.21.15.111:0/617162130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae1c1efb10 msgr2=0x7fae1c15ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:24.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.480+0000 7fae13fff640 1 --2- 172.21.15.111:0/617162130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae1c1efb10 0x7fae1c15ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:24.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.480+0000 7fae13fff640 1 -- 172.21.15.111:0/617162130 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c13fb90 con 0x7fae1c142460 2024-09-13T17:57:24.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.480+0000 7fae13fff640 1 --2- 172.21.15.111:0/617162130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c142460 0x7fae1c1ef5d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fae0400e4d0 tx=0x7fae0400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:24.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.480+0000 7fae197fa640 1 -- 172.21.15.111:0/617162130 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae040053a0 con 0x7fae1c142460 2024-09-13T17:57:24.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.480+0000 7fae197fa640 1 -- 172.21.15.111:0/617162130 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae040035f0 con 0x7fae1c142460 2024-09-13T17:57:24.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.480+0000 7fae22a48640 1 -- 172.21.15.111:0/617162130 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1c13fe70 con 0x7fae1c142460 2024-09-13T17:57:24.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.480+0000 7fae197fa640 1 -- 172.21.15.111:0/617162130 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04003980 con 0x7fae1c142460 2024-09-13T17:57:24.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.480+0000 7fae22a48640 1 -- 172.21.15.111:0/617162130 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1c142f00 con 0x7fae1c142460 2024-09-13T17:57:24.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.481+0000 7fae22a48640 1 -- 172.21.15.111:0/617162130 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1c1efef0 con 0x7fae1c142460 2024-09-13T17:57:24.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.482+0000 7fae197fa640 1 -- 172.21.15.111:0/617162130 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fae04003ae0 con 0x7fae1c142460 2024-09-13T17:57:24.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.482+0000 7fae197fa640 1 --2- 172.21.15.111:0/617162130 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fadfc06ac00 0x7fadfc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:24.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.482+0000 7fae197fa640 1 -- 172.21.15.111:0/617162130 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fae04088da0 con 0x7fae1c142460 2024-09-13T17:57:24.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.482+0000 7fae1bfff640 1 --2- 172.21.15.111:0/617162130 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fadfc06ac00 0x7fadfc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:24.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.482+0000 7fae1bfff640 1 --2- 172.21.15.111:0/617162130 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fadfc06ac00 0x7fadfc06d0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fae0c007fd0 tx=0x7fae0c02d000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:24.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.486+0000 7fae197fa640 1 -- 172.21.15.111:0/617162130 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fae04054210 con 0x7fae1c142460 2024-09-13T17:57:24.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:24.649+0000 7fae22a48640 1 -- 172.21.15.111:0/617162130 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fae1c140350 con 0x7fae1c142460 2024-09-13T17:57:25.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.362+0000 7fae197fa640 1 -- 172.21.15.111:0/617162130 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7fae04057830 con 0x7fae1c142460 2024-09-13T17:57:25.364 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' created 2024-09-13T17:57:25.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.367+0000 7fae22a48640 1 -- 172.21.15.111:0/617162130 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fadfc06ac00 msgr2=0x7fadfc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:25.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.367+0000 7fae22a48640 1 --2- 172.21.15.111:0/617162130 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fadfc06ac00 0x7fadfc06d0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fae0c007fd0 tx=0x7fae0c02d000 comp rx=0 tx=0).stop 2024-09-13T17:57:25.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.367+0000 7fae22a48640 1 -- 172.21.15.111:0/617162130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c142460 msgr2=0x7fae1c1ef5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:25.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.367+0000 7fae22a48640 1 --2- 172.21.15.111:0/617162130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c142460 0x7fae1c1ef5d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fae0400e4d0 tx=0x7fae0400e990 comp rx=0 tx=0).stop 2024-09-13T17:57:25.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.371+0000 7fae22a48640 1 -- 172.21.15.111:0/617162130 shutdown_connections 2024-09-13T17:57:25.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.371+0000 7fae22a48640 1 --2- 172.21.15.111:0/617162130 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fadfc06ac00 0x7fadfc06d0c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.371+0000 7fae22a48640 1 --2- 172.21.15.111:0/617162130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae1c1efb10 0x7fae1c15ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.371+0000 7fae22a48640 1 --2- 172.21.15.111:0/617162130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1c142460 0x7fae1c1ef5d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.371+0000 7fae22a48640 1 --2- 172.21.15.111:0/617162130 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae08003970 0x7fae1c0afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.371+0000 7fae22a48640 1 -- 172.21.15.111:0/617162130 >> 172.21.15.111:0/617162130 conn(0x7fae1c065ff0 msgr2=0x7fae1c13e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:25.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.372+0000 7fae22a48640 1 -- 172.21.15.111:0/617162130 shutdown_connections 2024-09-13T17:57:25.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.372+0000 7fae22a48640 1 -- 172.21.15.111:0/617162130 wait complete. 2024-09-13T17:57:25.392 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T17:57:25.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.493+0000 7fb9b365a640 1 Processor -- start 2024-09-13T17:57:25.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.493+0000 7fb9b365a640 1 -- start start 2024-09-13T17:57:25.497 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.494+0000 7fb9b365a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb9ac15c500 0x7fb9ac0d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:25.497 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.494+0000 7fb9b365a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb9ac0cdff0 0x7fb9ac0d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:25.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.494+0000 7fb9b365a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ac0cd450 con 0x7fb9ac0d2590 2024-09-13T17:57:25.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.494+0000 7fb9b365a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ac0d29f0 con 0x7fb9ac15c500 2024-09-13T17:57:25.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.494+0000 7fb9b365a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ac0d0cf0 con 0x7fb9ac0cdff0 2024-09-13T17:57:25.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.494+0000 7fb9b13f6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb9ac15c500 0x7fb9ac0d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:25.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.494+0000 7fb9b13f6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb9ac15c500 0x7fb9ac0d2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36416/0 (socket says 172.21.15.111:36416) 2024-09-13T17:57:25.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.494+0000 7fb9b13f6640 1 -- 172.21.15.111:0/2723677825 learned_addr learned my addr 172.21.15.111:0/2723677825 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:25.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.495+0000 7fb9b1bf7640 1 --2- 172.21.15.111:0/2723677825 >> v2:172.21.15.188:3300/0 conn(0x7fb9ac0cdff0 0x7fb9ac0d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:25.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.495+0000 7fb9b13f6640 1 -- 172.21.15.111:0/2723677825 >> v2:172.21.15.188:3300/0 conn(0x7fb9ac0cdff0 msgr2=0x7fb9ac0d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:25.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.495+0000 7fb9b13f6640 1 --2- 172.21.15.111:0/2723677825 >> v2:172.21.15.188:3300/0 conn(0x7fb9ac0cdff0 0x7fb9ac0d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.495+0000 7fb9b13f6640 1 -- 172.21.15.111:0/2723677825 >> v1:172.21.15.188:6789/0 conn(0x7fb9ac0d2590 legacy=0x7fb9ac0cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:25.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.495+0000 7fb9b13f6640 1 -- 172.21.15.111:0/2723677825 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac0d0fd0 con 0x7fb9ac15c500 2024-09-13T17:57:25.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.495+0000 7fb9b13f6640 1 --2- 172.21.15.111:0/2723677825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9ac15c500 0x7fb9ac0d2050 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0cca00 tx=0x7fb99c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a9dd30994a6c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:25.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.496+0000 7fb9abfff640 1 -- 172.21.15.111:0/2723677825 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c032440 con 0x7fb9ac15c500 2024-09-13T17:57:25.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.496+0000 7fb9abfff640 1 -- 172.21.15.111:0/2723677825 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c0325a0 con 0x7fb9ac15c500 2024-09-13T17:57:25.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.496+0000 7fb9abfff640 1 -- 172.21.15.111:0/2723677825 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c032930 con 0x7fb9ac15c500 2024-09-13T17:57:25.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.496+0000 7fb9b365a640 1 -- 172.21.15.111:0/2723677825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9ac15c500 msgr2=0x7fb9ac0d2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:25.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.496+0000 7fb9b365a640 1 --2- 172.21.15.111:0/2723677825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9ac15c500 0x7fb9ac0d2050 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0cca00 tx=0x7fb99c02a270 comp rx=0 tx=0).stop 2024-09-13T17:57:25.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.496+0000 7fb9b365a640 1 -- 172.21.15.111:0/2723677825 shutdown_connections 2024-09-13T17:57:25.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.496+0000 7fb9b365a640 1 --2- 172.21.15.111:0/2723677825 >> v2:172.21.15.188:3300/0 conn(0x7fb9ac0cdff0 0x7fb9ac0d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.496+0000 7fb9b365a640 1 --2- 172.21.15.111:0/2723677825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9ac15c500 0x7fb9ac0d2050 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.496+0000 7fb9b365a640 1 -- 172.21.15.111:0/2723677825 >> 172.21.15.111:0/2723677825 conn(0x7fb9ac089f60 msgr2=0x7fb9ac09b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:25.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.496+0000 7fb9b365a640 1 -- 172.21.15.111:0/2723677825 shutdown_connections 2024-09-13T17:57:25.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.496+0000 7fb9b365a640 1 -- 172.21.15.111:0/2723677825 wait complete. 2024-09-13T17:57:25.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.497+0000 7fb9b365a640 1 Processor -- start 2024-09-13T17:57:25.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.497+0000 7fb9b365a640 1 -- start start 2024-09-13T17:57:25.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.498+0000 7fb9b365a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9ac0cdff0 0x7fb9ac15d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:25.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.498+0000 7fb9b365a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9ac0d2590 0x7fb9ac154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:25.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.498+0000 7fb9b365a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9ac15c500 0x7fb9ac1f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:25.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.498+0000 7fb9b365a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1f2280 con 0x7fb9ac0d2590 2024-09-13T17:57:25.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.498+0000 7fb9b365a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1f1f70 con 0x7fb9ac15c500 2024-09-13T17:57:25.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.498+0000 7fb9b365a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac0ca170 con 0x7fb9ac0cdff0 2024-09-13T17:57:25.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.498+0000 7fb9b0bf5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9ac0d2590 0x7fb9ac154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:25.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.498+0000 7fb9b0bf5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9ac0d2590 0x7fb9ac154210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36428/0 (socket says 172.21.15.111:36428) 2024-09-13T17:57:25.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.498+0000 7fb9b0bf5640 1 -- 172.21.15.111:0/4157781355 learned_addr learned my addr 172.21.15.111:0/4157781355 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:25.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.498+0000 7fb9b13f6640 1 --2- 172.21.15.111:0/4157781355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9ac0cdff0 0x7fb9ac15d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:25.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.498+0000 7fb9b1bf7640 1 --2- 172.21.15.111:0/4157781355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9ac15c500 0x7fb9ac1f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:25.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.499+0000 7fb9b0bf5640 1 -- 172.21.15.111:0/4157781355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9ac0cdff0 msgr2=0x7fb9ac15d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:25.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.499+0000 7fb9b0bf5640 1 --2- 172.21.15.111:0/4157781355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9ac0cdff0 0x7fb9ac15d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.499+0000 7fb9b0bf5640 1 -- 172.21.15.111:0/4157781355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9ac15c500 msgr2=0x7fb9ac1f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:25.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.499+0000 7fb9b0bf5640 1 --2- 172.21.15.111:0/4157781355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9ac15c500 0x7fb9ac1f2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.499+0000 7fb9b0bf5640 1 -- 172.21.15.111:0/4157781355 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c0094d0 con 0x7fb9ac0d2590 2024-09-13T17:57:25.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.499+0000 7fb9b0bf5640 1 --2- 172.21.15.111:0/4157781355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9ac0d2590 0x7fb9ac154210 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000c480 tx=0x7fb9a000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:25.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.499+0000 7fb9a9ffb640 1 -- 172.21.15.111:0/4157781355 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a00052d0 con 0x7fb9ac0d2590 2024-09-13T17:57:25.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.499+0000 7fb9a9ffb640 1 -- 172.21.15.111:0/4157781355 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a0005430 con 0x7fb9ac0d2590 2024-09-13T17:57:25.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.499+0000 7fb9b365a640 1 -- 172.21.15.111:0/4157781355 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ac0ca450 con 0x7fb9ac0d2590 2024-09-13T17:57:25.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.499+0000 7fb9a9ffb640 1 -- 172.21.15.111:0/4157781355 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0012870 con 0x7fb9ac0d2590 2024-09-13T17:57:25.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.499+0000 7fb9b365a640 1 -- 172.21.15.111:0/4157781355 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ac15d7b0 con 0x7fb9ac0d2590 2024-09-13T17:57:25.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.500+0000 7fb9b365a640 1 -- 172.21.15.111:0/4157781355 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ac1f2430 con 0x7fb9ac0d2590 2024-09-13T17:57:25.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.500+0000 7fb9a9ffb640 1 -- 172.21.15.111:0/4157781355 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb9a00129d0 con 0x7fb9ac0d2590 2024-09-13T17:57:25.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.501+0000 7fb9a9ffb640 1 --2- 172.21.15.111:0/4157781355 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb99806ac00 0x7fb99806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:25.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.501+0000 7fb9b13f6640 1 --2- 172.21.15.111:0/4157781355 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb99806ac00 0x7fb99806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:25.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.501+0000 7fb9a9ffb640 1 -- 172.21.15.111:0/4157781355 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fb9a0088cd0 con 0x7fb9ac0d2590 2024-09-13T17:57:25.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.501+0000 7fb9b13f6640 1 --2- 172.21.15.111:0/4157781355 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb99806ac00 0x7fb99806d0c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb99c034000 tx=0x7fb99c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:25.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.506+0000 7fb9a9ffb640 1 -- 172.21.15.111:0/4157781355 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fb9ac1f2430 con 0x7fb9ac0d2590 2024-09-13T17:57:25.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.701+0000 7fb9b365a640 1 -- 172.21.15.111:0/4157781355 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb9ac1f2430 con 0x7fb9ac0d2590 2024-09-13T17:57:25.708 INFO:teuthology.orchestra.run.smithi111.stderr:new fs with metadata pool 2 and data pool 3 2024-09-13T17:57:25.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.706+0000 7fb9a9ffb640 1 -- 172.21.15.111:0/4157781355 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb9ac1f2430 con 0x7fb9ac0d2590 2024-09-13T17:57:25.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.714+0000 7fb9b365a640 1 -- 172.21.15.111:0/4157781355 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb99806ac00 msgr2=0x7fb99806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:25.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.714+0000 7fb9b365a640 1 --2- 172.21.15.111:0/4157781355 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb99806ac00 0x7fb99806d0c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb99c034000 tx=0x7fb99c02e000 comp rx=0 tx=0).stop 2024-09-13T17:57:25.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.714+0000 7fb9b365a640 1 -- 172.21.15.111:0/4157781355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9ac0d2590 msgr2=0x7fb9ac154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:25.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.714+0000 7fb9b365a640 1 --2- 172.21.15.111:0/4157781355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9ac0d2590 0x7fb9ac154210 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000c480 tx=0x7fb9a000c940 comp rx=0 tx=0).stop 2024-09-13T17:57:25.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.714+0000 7fb9b365a640 1 -- 172.21.15.111:0/4157781355 shutdown_connections 2024-09-13T17:57:25.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.714+0000 7fb9b365a640 1 --2- 172.21.15.111:0/4157781355 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb99806ac00 0x7fb99806d0c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.714+0000 7fb9b365a640 1 --2- 172.21.15.111:0/4157781355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9ac15c500 0x7fb9ac1f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.714+0000 7fb9b365a640 1 --2- 172.21.15.111:0/4157781355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9ac0d2590 0x7fb9ac154210 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.714+0000 7fb9b365a640 1 --2- 172.21.15.111:0/4157781355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9ac0cdff0 0x7fb9ac15d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.714+0000 7fb9b365a640 1 -- 172.21.15.111:0/4157781355 >> 172.21.15.111:0/4157781355 conn(0x7fb9ac089f60 msgr2=0x7fb9ac06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:25.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.720+0000 7fb9b365a640 1 -- 172.21.15.111:0/4157781355 shutdown_connections 2024-09-13T17:57:25.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.721+0000 7fb9b365a640 1 -- 172.21.15.111:0/4157781355 wait complete. 2024-09-13T17:57:25.738 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-13T17:57:25.738 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-13T17:57:25.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.839+0000 7fac82d1f640 1 Processor -- start 2024-09-13T17:57:25.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.839+0000 7fac82d1f640 1 -- start start 2024-09-13T17:57:25.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.839+0000 7fac82d1f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fac7c0d22d0 0x7fac7c0d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:25.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.839+0000 7fac82d1f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fac7c06bb50 0x7fac7c06bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:25.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.839+0000 7fac82d1f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac7c06b6c0 con 0x7fac7c15fbc0 2024-09-13T17:57:25.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.839+0000 7fac82d1f640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac7c0ce500 con 0x7fac7c0d22d0 2024-09-13T17:57:25.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.839+0000 7fac82d1f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac7c0cdc60 con 0x7fac7c06bb50 2024-09-13T17:57:25.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.840+0000 7fac7bfff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fac7c0d22d0 0x7fac7c0d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:25.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.840+0000 7fac7bfff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fac7c0d22d0 0x7fac7c0d2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36432/0 (socket says 172.21.15.111:36432) 2024-09-13T17:57:25.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.840+0000 7fac7bfff640 1 -- 172.21.15.111:0/1466515705 learned_addr learned my addr 172.21.15.111:0/1466515705 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:25.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.840+0000 7fac812bc640 1 --2- 172.21.15.111:0/1466515705 >> v2:172.21.15.188:3300/0 conn(0x7fac7c06bb50 0x7fac7c06bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:25.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.840+0000 7fac7bfff640 1 -- 172.21.15.111:0/1466515705 >> v2:172.21.15.188:3300/0 conn(0x7fac7c06bb50 msgr2=0x7fac7c06bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:25.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.840+0000 7fac7bfff640 1 --2- 172.21.15.111:0/1466515705 >> v2:172.21.15.188:3300/0 conn(0x7fac7c06bb50 0x7fac7c06bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.840+0000 7fac7bfff640 1 -- 172.21.15.111:0/1466515705 >> v1:172.21.15.188:6789/0 conn(0x7fac7c15fbc0 legacy=0x7fac7c0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.840+0000 7fac7bfff640 1 -- 172.21.15.111:0/1466515705 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c0cdf40 con 0x7fac7c0d22d0 2024-09-13T17:57:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.841+0000 7fac7bfff640 1 --2- 172.21.15.111:0/1466515705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac7c0d22d0 0x7fac7c0d2710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0ceb10 tx=0x7fac6c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fcedeb36df5d60 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.841+0000 7fac7b7fe640 1 -- 172.21.15.111:0/1466515705 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c031600 con 0x7fac7c0d22d0 2024-09-13T17:57:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.841+0000 7fac7b7fe640 1 -- 172.21.15.111:0/1466515705 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c02d050 con 0x7fac7c0d22d0 2024-09-13T17:57:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.841+0000 7fac7b7fe640 1 -- 172.21.15.111:0/1466515705 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c031980 con 0x7fac7c0d22d0 2024-09-13T17:57:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.841+0000 7fac82d1f640 1 -- 172.21.15.111:0/1466515705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac7c0d22d0 msgr2=0x7fac7c0d2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.841+0000 7fac82d1f640 1 --2- 172.21.15.111:0/1466515705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac7c0d22d0 0x7fac7c0d2710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0ceb10 tx=0x7fac6c02a730 comp rx=0 tx=0).stop 2024-09-13T17:57:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.841+0000 7fac82d1f640 1 -- 172.21.15.111:0/1466515705 shutdown_connections 2024-09-13T17:57:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.841+0000 7fac82d1f640 1 --2- 172.21.15.111:0/1466515705 >> v2:172.21.15.188:3300/0 conn(0x7fac7c06bb50 0x7fac7c06bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.841+0000 7fac82d1f640 1 --2- 172.21.15.111:0/1466515705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac7c0d22d0 0x7fac7c0d2710 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.841+0000 7fac82d1f640 1 -- 172.21.15.111:0/1466515705 >> 172.21.15.111:0/1466515705 conn(0x7fac7c09b830 msgr2=0x7fac7c089f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.841+0000 7fac82d1f640 1 -- 172.21.15.111:0/1466515705 shutdown_connections 2024-09-13T17:57:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.841+0000 7fac82d1f640 1 -- 172.21.15.111:0/1466515705 wait complete. 2024-09-13T17:57:25.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.842+0000 7fac82d1f640 1 Processor -- start 2024-09-13T17:57:25.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.843+0000 7fac82d1f640 1 -- start start 2024-09-13T17:57:25.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.843+0000 7fac82d1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac7c06bb50 0x7fac7c156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:25.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.843+0000 7fac82d1f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac7c0d22d0 0x7fac7c155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:25.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.843+0000 7fac82d1f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac7c15fbc0 0x7fac7c158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:25.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.843+0000 7fac82d1f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c1557a0 con 0x7fac7c06bb50 2024-09-13T17:57:25.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.843+0000 7fac82d1f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c1594d0 con 0x7fac7c15fbc0 2024-09-13T17:57:25.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.843+0000 7fac82d1f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c157af0 con 0x7fac7c0d22d0 2024-09-13T17:57:25.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.843+0000 7fac80abb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac7c06bb50 0x7fac7c156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:25.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.844+0000 7fac80abb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac7c06bb50 0x7fac7c156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36438/0 (socket says 172.21.15.111:36438) 2024-09-13T17:57:25.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.844+0000 7fac80abb640 1 -- 172.21.15.111:0/2629696976 learned_addr learned my addr 172.21.15.111:0/2629696976 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:25.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.844+0000 7fac812bc640 1 --2- 172.21.15.111:0/2629696976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac7c15fbc0 0x7fac7c158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:25.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.844+0000 7fac7bfff640 1 --2- 172.21.15.111:0/2629696976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac7c0d22d0 0x7fac7c155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:25.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.844+0000 7fac80abb640 1 -- 172.21.15.111:0/2629696976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac7c0d22d0 msgr2=0x7fac7c155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:25.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.844+0000 7fac80abb640 1 --2- 172.21.15.111:0/2629696976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac7c0d22d0 0x7fac7c155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.844+0000 7fac80abb640 1 -- 172.21.15.111:0/2629696976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac7c15fbc0 msgr2=0x7fac7c158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:25.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.844+0000 7fac80abb640 1 --2- 172.21.15.111:0/2629696976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac7c15fbc0 0x7fac7c158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:25.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.844+0000 7fac80abb640 1 -- 172.21.15.111:0/2629696976 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c009b30 con 0x7fac7c06bb50 2024-09-13T17:57:25.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.844+0000 7fac812bc640 1 --2- 172.21.15.111:0/2629696976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac7c15fbc0 0x7fac7c158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:25.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.844+0000 7fac7bfff640 1 --2- 172.21.15.111:0/2629696976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac7c0d22d0 0x7fac7c155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:25.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.844+0000 7fac80abb640 1 --2- 172.21.15.111:0/2629696976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac7c06bb50 0x7fac7c156b70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac6400c430 tx=0x7fac6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:25.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.844+0000 7fac797fa640 1 -- 172.21.15.111:0/2629696976 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64011790 con 0x7fac7c06bb50 2024-09-13T17:57:25.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.844+0000 7fac797fa640 1 -- 172.21.15.111:0/2629696976 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac640118f0 con 0x7fac7c06bb50 2024-09-13T17:57:25.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.844+0000 7fac82d1f640 1 -- 172.21.15.111:0/2629696976 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7c157250 con 0x7fac7c06bb50 2024-09-13T17:57:25.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.844+0000 7fac797fa640 1 -- 172.21.15.111:0/2629696976 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64011ce0 con 0x7fac7c06bb50 2024-09-13T17:57:25.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.844+0000 7fac82d1f640 1 -- 172.21.15.111:0/2629696976 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7c157810 con 0x7fac7c06bb50 2024-09-13T17:57:25.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.845+0000 7fac82d1f640 1 -- 172.21.15.111:0/2629696976 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac7c0b69d0 con 0x7fac7c06bb50 2024-09-13T17:57:25.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.846+0000 7fac797fa640 1 -- 172.21.15.111:0/2629696976 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fac6401c460 con 0x7fac7c06bb50 2024-09-13T17:57:25.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.846+0000 7fac797fa640 1 --2- 172.21.15.111:0/2629696976 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fac5c06ac00 0x7fac5c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:25.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.846+0000 7fac7bfff640 1 --2- 172.21.15.111:0/2629696976 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fac5c06ac00 0x7fac5c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:25.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.846+0000 7fac797fa640 1 -- 172.21.15.111:0/2629696976 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac640892f0 con 0x7fac7c06bb50 2024-09-13T17:57:25.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.847+0000 7fac7bfff640 1 --2- 172.21.15.111:0/2629696976 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fac5c06ac00 0x7fac5c06d0c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fac6c033000 tx=0x7fac6c009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:25.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:25.851+0000 7fac797fa640 1 -- 172.21.15.111:0/2629696976 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fac7c0b69d0 con 0x7fac7c06bb50 2024-09-13T17:57:26.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.018+0000 7fac82d1f640 1 -- 172.21.15.111:0/2629696976 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fac7c0b69d0 con 0x7fac7c06bb50 2024-09-13T17:57:26.261 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:57:26.264 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:57:26.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.712+0000 7fac797fa640 1 -- 172.21.15.111:0/2629696976 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7fac7c0b69d0 con 0x7fac7c06bb50 2024-09-13T17:57:26.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.714+0000 7fac82d1f640 1 -- 172.21.15.111:0/2629696976 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fac5c06ac00 msgr2=0x7fac5c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:26.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.714+0000 7fac82d1f640 1 --2- 172.21.15.111:0/2629696976 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fac5c06ac00 0x7fac5c06d0c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fac6c033000 tx=0x7fac6c009000 comp rx=0 tx=0).stop 2024-09-13T17:57:26.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.714+0000 7fac82d1f640 1 -- 172.21.15.111:0/2629696976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac7c06bb50 msgr2=0x7fac7c156b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:26.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.714+0000 7fac82d1f640 1 --2- 172.21.15.111:0/2629696976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac7c06bb50 0x7fac7c156b70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac6400c430 tx=0x7fac6400c8f0 comp rx=0 tx=0).stop 2024-09-13T17:57:26.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.715+0000 7fac82d1f640 1 -- 172.21.15.111:0/2629696976 shutdown_connections 2024-09-13T17:57:26.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.715+0000 7fac82d1f640 1 --2- 172.21.15.111:0/2629696976 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fac5c06ac00 0x7fac5c06d0c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:26.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.715+0000 7fac82d1f640 1 --2- 172.21.15.111:0/2629696976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac7c15fbc0 0x7fac7c158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:26.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.715+0000 7fac82d1f640 1 --2- 172.21.15.111:0/2629696976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac7c0d22d0 0x7fac7c155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:26.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.715+0000 7fac82d1f640 1 --2- 172.21.15.111:0/2629696976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac7c06bb50 0x7fac7c156b70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:26.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.715+0000 7fac82d1f640 1 -- 172.21.15.111:0/2629696976 >> 172.21.15.111:0/2629696976 conn(0x7fac7c09b830 msgr2=0x7fac7c0ca760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:26.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.715+0000 7fac82d1f640 1 -- 172.21.15.111:0/2629696976 shutdown_connections 2024-09-13T17:57:26.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.715+0000 7fac82d1f640 1 -- 172.21.15.111:0/2629696976 wait complete. 2024-09-13T17:57:26.728 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T17:57:26.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.834+0000 7fccc56e4640 1 Processor -- start 2024-09-13T17:57:26.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.834+0000 7fccc56e4640 1 -- start start 2024-09-13T17:57:26.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.835+0000 7fccc56e4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fccc00cd500 0x7fccc00b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:26.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.835+0000 7fccc56e4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fccc00d1650 0x7fccc00d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:26.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.835+0000 7fccc56e4640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc00d1f40 con 0x7fccc00b1370 2024-09-13T17:57:26.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.835+0000 7fccc56e4640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc00d2190 con 0x7fccc00cd500 2024-09-13T17:57:26.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.835+0000 7fccc56e4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc00ccb20 con 0x7fccc00d1650 2024-09-13T17:57:26.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.835+0000 7fccbeffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fccc00cd500 0x7fccc00b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:26.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.835+0000 7fccbe7fc640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fccc00b1370 0x7fccc00b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58074/0 (socket says 172.21.15.111:58074) 2024-09-13T17:57:26.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.835+0000 7fccbe7fc640 1 -- 172.21.15.111:0/2586550154 learned_addr learned my addr 172.21.15.111:0/2586550154 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:26.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.835+0000 7fccbdffb640 1 -- 172.21.15.111:0/2586550154 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657369187 0 0) 0x7fccc00d1f40 con 0x7fccc00b1370 2024-09-13T17:57:26.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.835+0000 7fccbdffb640 1 -- 172.21.15.111:0/2586550154 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcca8003520 con 0x7fccc00b1370 2024-09-13T17:57:26.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.835+0000 7fccbeffd640 1 -- 172.21.15.111:0/2586550154 >> v2:172.21.15.188:3300/0 conn(0x7fccc00d1650 msgr2=0x7fccc00d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:26.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.835+0000 7fccbeffd640 1 --2- 172.21.15.111:0/2586550154 >> v2:172.21.15.188:3300/0 conn(0x7fccc00d1650 0x7fccc00d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:26.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.835+0000 7fccbeffd640 1 -- 172.21.15.111:0/2586550154 >> v1:172.21.15.111:6789/0 conn(0x7fccc00b1370 legacy=0x7fccc00b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:26.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.835+0000 7fccbeffd640 1 -- 172.21.15.111:0/2586550154 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc00cce00 con 0x7fccc00cd500 2024-09-13T17:57:26.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.836+0000 7fccbeffd640 1 --2- 172.21.15.111:0/2586550154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccc00cd500 0x7fccc00b3850 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009d90 tx=0x7fccb002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61003176235f59da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:26.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.836+0000 7fccbdffb640 1 -- 172.21.15.111:0/2586550154 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0007740 con 0x7fccc00cd500 2024-09-13T17:57:26.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.836+0000 7fccbdffb640 1 -- 172.21.15.111:0/2586550154 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb002d050 con 0x7fccc00cd500 2024-09-13T17:57:26.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.836+0000 7fccbdffb640 1 -- 172.21.15.111:0/2586550154 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0007ac0 con 0x7fccc00cd500 2024-09-13T17:57:26.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.837+0000 7fccc56e4640 1 -- 172.21.15.111:0/2586550154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccc00cd500 msgr2=0x7fccc00b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:26.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.837+0000 7fccc56e4640 1 --2- 172.21.15.111:0/2586550154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccc00cd500 0x7fccc00b3850 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009d90 tx=0x7fccb002a6a0 comp rx=0 tx=0).stop 2024-09-13T17:57:26.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.837+0000 7fccc56e4640 1 -- 172.21.15.111:0/2586550154 shutdown_connections 2024-09-13T17:57:26.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.837+0000 7fccc56e4640 1 --2- 172.21.15.111:0/2586550154 >> v2:172.21.15.188:3300/0 conn(0x7fccc00d1650 0x7fccc00d2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:26.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.837+0000 7fccc56e4640 1 --2- 172.21.15.111:0/2586550154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccc00cd500 0x7fccc00b3850 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:26.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.837+0000 7fccc56e4640 1 -- 172.21.15.111:0/2586550154 >> 172.21.15.111:0/2586550154 conn(0x7fccc0089f60 msgr2=0x7fccc00d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:26.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.837+0000 7fccc56e4640 1 -- 172.21.15.111:0/2586550154 shutdown_connections 2024-09-13T17:57:26.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.837+0000 7fccc56e4640 1 -- 172.21.15.111:0/2586550154 wait complete. 2024-09-13T17:57:26.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.838+0000 7fccc56e4640 1 Processor -- start 2024-09-13T17:57:26.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.838+0000 7fccc56e4640 1 -- start start 2024-09-13T17:57:26.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.839+0000 7fccc56e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccc00cd500 0x7fccc00b7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:26.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.839+0000 7fccc56e4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccc00d1650 0x7fccc00b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:26.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.839+0000 7fccc56e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccc00b07b0 0x7fccc014c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:26.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.839+0000 7fccc56e4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc00c6420 con 0x7fccc00d1650 2024-09-13T17:57:26.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.839+0000 7fccc56e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc00c6700 con 0x7fccc00cd500 2024-09-13T17:57:26.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.839+0000 7fccc56e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc00c69e0 con 0x7fccc00b07b0 2024-09-13T17:57:26.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.839+0000 7fccbe7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccc00d1650 0x7fccc00b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:26.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.839+0000 7fccbe7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccc00d1650 0x7fccc00b0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36450/0 (socket says 172.21.15.111:36450) 2024-09-13T17:57:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.839+0000 7fccbe7fc640 1 -- 172.21.15.111:0/472375205 learned_addr learned my addr 172.21.15.111:0/472375205 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.839+0000 7fccbf7fe640 1 --2- 172.21.15.111:0/472375205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccc00b07b0 0x7fccc014c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.839+0000 7fccbeffd640 1 --2- 172.21.15.111:0/472375205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccc00cd500 0x7fccc00b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.839+0000 7fccbe7fc640 1 -- 172.21.15.111:0/472375205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccc00b07b0 msgr2=0x7fccc014c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.839+0000 7fccbe7fc640 1 --2- 172.21.15.111:0/472375205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccc00b07b0 0x7fccc014c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.839+0000 7fccbe7fc640 1 -- 172.21.15.111:0/472375205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccc00cd500 msgr2=0x7fccc00b7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.839+0000 7fccbe7fc640 1 --2- 172.21.15.111:0/472375205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccc00cd500 0x7fccc00b7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.839+0000 7fccbe7fc640 1 -- 172.21.15.111:0/472375205 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb0009a70 con 0x7fccc00d1650 2024-09-13T17:57:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.839+0000 7fccbf7fe640 1 --2- 172.21.15.111:0/472375205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccc00b07b0 0x7fccc014c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.840+0000 7fccbe7fc640 1 --2- 172.21.15.111:0/472375205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccc00d1650 0x7fccc00b0270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fccb400c500 tx=0x7fccb400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.840+0000 7fcca77fe640 1 -- 172.21.15.111:0/472375205 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb4005ce0 con 0x7fccc00d1650 2024-09-13T17:57:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.840+0000 7fcca77fe640 1 -- 172.21.15.111:0/472375205 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb4005e40 con 0x7fccc00d1650 2024-09-13T17:57:26.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.840+0000 7fccc56e4640 1 -- 172.21.15.111:0/472375205 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc00c6cc0 con 0x7fccc00d1650 2024-09-13T17:57:26.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.840+0000 7fcca77fe640 1 -- 172.21.15.111:0/472375205 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb4007330 con 0x7fccc00d1650 2024-09-13T17:57:26.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.840+0000 7fccc56e4640 1 -- 172.21.15.111:0/472375205 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc00c6060 con 0x7fccc00d1650 2024-09-13T17:57:26.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.841+0000 7fccc56e4640 1 -- 172.21.15.111:0/472375205 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc00c71e0 con 0x7fccc00d1650 2024-09-13T17:57:26.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.841+0000 7fcca77fe640 1 -- 172.21.15.111:0/472375205 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fccb4007550 con 0x7fccc00d1650 2024-09-13T17:57:26.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.841+0000 7fcca77fe640 1 --2- 172.21.15.111:0/472375205 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fccac06ac00 0x7fccac06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:26.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.842+0000 7fcca77fe640 1 -- 172.21.15.111:0/472375205 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5935+0+0 (secure 0 0 0) 0x7fccb4088a90 con 0x7fccc00d1650 2024-09-13T17:57:26.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.842+0000 7fccbeffd640 1 --2- 172.21.15.111:0/472375205 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fccac06ac00 0x7fccac06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:26.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.842+0000 7fccbeffd640 1 --2- 172.21.15.111:0/472375205 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fccac06ac00 0x7fccac06d0c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fccb0034000 tx=0x7fccb002f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:26.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:26.846+0000 7fcca77fe640 1 -- 172.21.15.111:0/472375205 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fccc00c71e0 con 0x7fccc00d1650 2024-09-13T17:57:27.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:27.007+0000 7fccc56e4640 1 -- 172.21.15.111:0/472375205 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fccc00c71e0 con 0x7fccc00d1650 2024-09-13T17:57:28.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.718+0000 7fcca77fe640 1 -- 172.21.15.111:0/472375205 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7fccc00c71e0 con 0x7fccc00d1650 2024-09-13T17:57:28.721 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data_ec' created 2024-09-13T17:57:28.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.724+0000 7fccc56e4640 1 -- 172.21.15.111:0/472375205 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fccac06ac00 msgr2=0x7fccac06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:28.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.724+0000 7fccc56e4640 1 --2- 172.21.15.111:0/472375205 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fccac06ac00 0x7fccac06d0c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fccb0034000 tx=0x7fccb002f000 comp rx=0 tx=0).stop 2024-09-13T17:57:28.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.724+0000 7fccc56e4640 1 -- 172.21.15.111:0/472375205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccc00d1650 msgr2=0x7fccc00b0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:28.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.724+0000 7fccc56e4640 1 --2- 172.21.15.111:0/472375205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccc00d1650 0x7fccc00b0270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fccb400c500 tx=0x7fccb400c9c0 comp rx=0 tx=0).stop 2024-09-13T17:57:28.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.725+0000 7fccc56e4640 1 -- 172.21.15.111:0/472375205 shutdown_connections 2024-09-13T17:57:28.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.725+0000 7fccc56e4640 1 --2- 172.21.15.111:0/472375205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccc00b07b0 0x7fccc014c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:28.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.725+0000 7fccc56e4640 1 --2- 172.21.15.111:0/472375205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccc00d1650 0x7fccc00b0270 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fccb400c500 tx=0x7fccb400c9c0 comp rx=0 tx=0).stop 2024-09-13T17:57:28.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.725+0000 7fccc56e4640 1 --2- 172.21.15.111:0/472375205 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fccac06ac00 0x7fccac06d0c0 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fccb0034000 tx=0x7fccb002f000 comp rx=0 tx=0).stop 2024-09-13T17:57:28.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.725+0000 7fccc56e4640 1 --2- 172.21.15.111:0/472375205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccc00cd500 0x7fccc00b7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:28.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.725+0000 7fccc56e4640 1 -- 172.21.15.111:0/472375205 >> 172.21.15.111:0/472375205 conn(0x7fccc0089f60 msgr2=0x7fccc015bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:28.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.725+0000 7fccc56e4640 1 -- 172.21.15.111:0/472375205 shutdown_connections 2024-09-13T17:57:28.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.725+0000 7fccc56e4640 1 -- 172.21.15.111:0/472375205 wait complete. 2024-09-13T17:57:28.743 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-13T17:57:28.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.847+0000 7f4bd06e5640 1 Processor -- start 2024-09-13T17:57:28.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.847+0000 7f4bd06e5640 1 -- start start 2024-09-13T17:57:28.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.848+0000 7f4bd06e5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4bc8159310 0x7f4bc80ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:28.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.848+0000 7f4bd06e5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4bc80b2b40 0x7f4bc80ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:28.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.848+0000 7f4bd06e5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc80ccbe0 con 0x7f4bc80ab680 2024-09-13T17:57:28.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.848+0000 7f4bd06e5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc80ad940 con 0x7f4bc80b2b40 2024-09-13T17:57:28.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.848+0000 7f4bd06e5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc80b1810 con 0x7f4bc8159310 2024-09-13T17:57:28.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.848+0000 7f4bcdc80640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4bc80b2b40 0x7f4bc80ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:28.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.849+0000 7f4bcec82640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4bc80ab680 0x7f4bc80ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58076/0 (socket says 172.21.15.111:58076) 2024-09-13T17:57:28.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.849+0000 7f4bcdc80640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4bc80b2b40 0x7f4bc80ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36452/0 (socket says 172.21.15.111:36452) 2024-09-13T17:57:28.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.849+0000 7f4bcec82640 1 -- 172.21.15.111:0/279321869 learned_addr learned my addr 172.21.15.111:0/279321869 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:28.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.849+0000 7f4bce481640 1 --2- 172.21.15.111:0/279321869 >> v2:172.21.15.188:3300/0 conn(0x7f4bc8159310 0x7f4bc80ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:28.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.849+0000 7f4bcdc80640 1 -- 172.21.15.111:0/279321869 >> v2:172.21.15.188:3300/0 conn(0x7f4bc8159310 msgr2=0x7f4bc80ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:28.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.849+0000 7f4bcdc80640 1 --2- 172.21.15.111:0/279321869 >> v2:172.21.15.188:3300/0 conn(0x7f4bc8159310 0x7f4bc80ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:28.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.849+0000 7f4bcdc80640 1 -- 172.21.15.111:0/279321869 >> v1:172.21.15.111:6789/0 conn(0x7f4bc80ab680 legacy=0x7f4bc80ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:28.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.849+0000 7f4bcd47f640 1 -- 172.21.15.111:0/279321869 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1245886481 0 0) 0x7f4bc80ccbe0 con 0x7f4bc80ab680 2024-09-13T17:57:28.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.849+0000 7f4bcdc80640 1 -- 172.21.15.111:0/279321869 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc806d9e0 con 0x7f4bc80b2b40 2024-09-13T17:57:28.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.849+0000 7f4bcdc80640 1 --2- 172.21.15.111:0/279321869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc80b2b40 0x7f4bc80ab140 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80cac90 tx=0x7f4bb802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223f619755432113 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:28.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.850+0000 7f4bcd47f640 1 -- 172.21.15.111:0/279321869 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb802ae60 con 0x7f4bc80b2b40 2024-09-13T17:57:28.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.850+0000 7f4bcd47f640 1 -- 172.21.15.111:0/279321869 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb8032440 con 0x7f4bc80b2b40 2024-09-13T17:57:28.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.850+0000 7f4bcd47f640 1 -- 172.21.15.111:0/279321869 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80327d0 con 0x7f4bc80b2b40 2024-09-13T17:57:28.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.850+0000 7f4bd06e5640 1 -- 172.21.15.111:0/279321869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc80b2b40 msgr2=0x7f4bc80ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:28.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.850+0000 7f4bd06e5640 1 --2- 172.21.15.111:0/279321869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc80b2b40 0x7f4bc80ab140 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80cac90 tx=0x7f4bb802a110 comp rx=0 tx=0).stop 2024-09-13T17:57:28.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.850+0000 7f4bd06e5640 1 -- 172.21.15.111:0/279321869 shutdown_connections 2024-09-13T17:57:28.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.850+0000 7f4bd06e5640 1 --2- 172.21.15.111:0/279321869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc80b2b40 0x7f4bc80ab140 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:28.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.850+0000 7f4bd06e5640 1 --2- 172.21.15.111:0/279321869 >> v2:172.21.15.188:3300/0 conn(0x7f4bc8159310 0x7f4bc80ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:28.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.850+0000 7f4bd06e5640 1 -- 172.21.15.111:0/279321869 >> 172.21.15.111:0/279321869 conn(0x7f4bc806a0b0 msgr2=0x7f4bc806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:28.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.850+0000 7f4bd06e5640 1 -- 172.21.15.111:0/279321869 shutdown_connections 2024-09-13T17:57:28.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.850+0000 7f4bd06e5640 1 -- 172.21.15.111:0/279321869 wait complete. 2024-09-13T17:57:28.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.851+0000 7f4bd06e5640 1 Processor -- start 2024-09-13T17:57:28.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.851+0000 7f4bd06e5640 1 -- start start 2024-09-13T17:57:28.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.852+0000 7f4bd06e5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc80ab680 0x7f4bc80c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:28.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.852+0000 7f4bd06e5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bc80b2b40 0x7f4bc809a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:28.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.852+0000 7f4bd06e5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc8159310 0x7f4bc80aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:28.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.852+0000 7f4bd06e5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc8166180 con 0x7f4bc80ab680 2024-09-13T17:57:28.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.852+0000 7f4bd06e5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc81657e0 con 0x7f4bc8159310 2024-09-13T17:57:28.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.852+0000 7f4bd06e5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc8165ac0 con 0x7f4bc80b2b40 2024-09-13T17:57:28.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.852+0000 7f4bce481640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc80ab680 0x7f4bc80c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:28.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.852+0000 7f4bce481640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc80ab680 0x7f4bc80c3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36468/0 (socket says 172.21.15.111:36468) 2024-09-13T17:57:28.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.852+0000 7f4bce481640 1 -- 172.21.15.111:0/2145324738 learned_addr learned my addr 172.21.15.111:0/2145324738 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:28.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.852+0000 7f4bce481640 1 -- 172.21.15.111:0/2145324738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bc80b2b40 msgr2=0x7f4bc809a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:28.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.852+0000 7f4bce481640 1 --2- 172.21.15.111:0/2145324738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bc80b2b40 0x7f4bc809a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:28.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.852+0000 7f4bce481640 1 -- 172.21.15.111:0/2145324738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc8159310 msgr2=0x7f4bc80aba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:28.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.852+0000 7f4bce481640 1 --2- 172.21.15.111:0/2145324738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc8159310 0x7f4bc80aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:28.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.852+0000 7f4bce481640 1 -- 172.21.15.111:0/2145324738 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb8009400 con 0x7f4bc80ab680 2024-09-13T17:57:28.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.853+0000 7f4bce481640 1 --2- 172.21.15.111:0/2145324738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc80ab680 0x7f4bc80c3d10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400e870 tx=0x7f4bc400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:28.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.853+0000 7f4bbeffd640 1 -- 172.21.15.111:0/2145324738 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc40094f0 con 0x7f4bc80ab680 2024-09-13T17:57:28.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.853+0000 7f4bbeffd640 1 -- 172.21.15.111:0/2145324738 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc400c940 con 0x7f4bc80ab680 2024-09-13T17:57:28.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.853+0000 7f4bd06e5640 1 -- 172.21.15.111:0/2145324738 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc8165da0 con 0x7f4bc80ab680 2024-09-13T17:57:28.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.853+0000 7f4bbeffd640 1 -- 172.21.15.111:0/2145324738 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc400ccf0 con 0x7f4bc80ab680 2024-09-13T17:57:28.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.853+0000 7f4bd06e5640 1 -- 172.21.15.111:0/2145324738 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc80c4250 con 0x7f4bc80ab680 2024-09-13T17:57:28.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.854+0000 7f4bd06e5640 1 -- 172.21.15.111:0/2145324738 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc80ab140 con 0x7f4bc80ab680 2024-09-13T17:57:28.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.854+0000 7f4bbeffd640 1 -- 172.21.15.111:0/2145324738 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4bc4013470 con 0x7f4bc80ab680 2024-09-13T17:57:28.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.855+0000 7f4bbeffd640 1 --2- 172.21.15.111:0/2145324738 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4bac06ac00 0x7f4bac06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:28.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.855+0000 7f4bcdc80640 1 --2- 172.21.15.111:0/2145324738 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4bac06ac00 0x7f4bac06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:28.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.855+0000 7f4bbeffd640 1 -- 172.21.15.111:0/2145324738 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f4bc4089680 con 0x7f4bc80ab680 2024-09-13T17:57:28.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.855+0000 7f4bcdc80640 1 --2- 172.21.15.111:0/2145324738 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4bac06ac00 0x7f4bac06d0c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8034000 tx=0x7f4bb802c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:28.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:28.859+0000 7f4bbeffd640 1 -- 172.21.15.111:0/2145324738 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4bc4054620 con 0x7f4bc80ab680 2024-09-13T17:57:29.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.021+0000 7f4bd06e5640 1 -- 172.21.15.111:0/2145324738 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f4bc806e4b0 con 0x7f4bc80ab680 2024-09-13T17:57:29.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.721+0000 7f4bbeffd640 1 -- 172.21.15.111:0/2145324738 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4bc4057c40 con 0x7f4bc80ab680 2024-09-13T17:57:29.723 INFO:teuthology.orchestra.run.smithi111.stderr:set pool 4 allow_ec_overwrites to true 2024-09-13T17:57:29.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.736+0000 7f4bd06e5640 1 -- 172.21.15.111:0/2145324738 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4bac06ac00 msgr2=0x7f4bac06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:29.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.736+0000 7f4bd06e5640 1 --2- 172.21.15.111:0/2145324738 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4bac06ac00 0x7f4bac06d0c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8034000 tx=0x7f4bb802c000 comp rx=0 tx=0).stop 2024-09-13T17:57:29.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.736+0000 7f4bd06e5640 1 -- 172.21.15.111:0/2145324738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc80ab680 msgr2=0x7f4bc80c3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:29.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.736+0000 7f4bd06e5640 1 --2- 172.21.15.111:0/2145324738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc80ab680 0x7f4bc80c3d10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400e870 tx=0x7f4bc400ed30 comp rx=0 tx=0).stop 2024-09-13T17:57:29.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.739+0000 7f4bd06e5640 1 -- 172.21.15.111:0/2145324738 shutdown_connections 2024-09-13T17:57:29.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.739+0000 7f4bd06e5640 1 --2- 172.21.15.111:0/2145324738 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4bac06ac00 0x7f4bac06d0c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:29.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.739+0000 7f4bd06e5640 1 --2- 172.21.15.111:0/2145324738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc8159310 0x7f4bc80aba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:29.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.739+0000 7f4bd06e5640 1 --2- 172.21.15.111:0/2145324738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bc80b2b40 0x7f4bc809a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:29.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.739+0000 7f4bd06e5640 1 --2- 172.21.15.111:0/2145324738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc80ab680 0x7f4bc80c3d10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:29.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.739+0000 7f4bd06e5640 1 -- 172.21.15.111:0/2145324738 >> 172.21.15.111:0/2145324738 conn(0x7f4bc806a0b0 msgr2=0x7f4bc81585b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:29.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.743+0000 7f4bd06e5640 1 -- 172.21.15.111:0/2145324738 shutdown_connections 2024-09-13T17:57:29.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.744+0000 7f4bd06e5640 1 -- 172.21.15.111:0/2145324738 wait complete. 2024-09-13T17:57:29.759 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-13T17:57:29.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.857+0000 7fb0804d7640 1 Processor -- start 2024-09-13T17:57:29.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.857+0000 7fb0804d7640 1 -- start start 2024-09-13T17:57:29.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.858+0000 7fb0804d7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb07815a480 0x7fb07806ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:29.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.858+0000 7fb0804d7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb078069ed0 0x7fb07806a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:29.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.858+0000 7fb0804d7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0780d0210 con 0x7fb07806aa70 2024-09-13T17:57:29.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.858+0000 7fb0804d7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0780ccd20 con 0x7fb07815a480 2024-09-13T17:57:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.858+0000 7fb0804d7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07806a8a0 con 0x7fb078069ed0 2024-09-13T17:57:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.858+0000 7fb07e273640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb07815a480 0x7fb07806ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.858+0000 7fb07ea74640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb078069ed0 0x7fb07806a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.858+0000 7fb07da72640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fb07806aa70 0x7fb0780cc610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:46710/0 (socket says 172.21.15.111:46710) 2024-09-13T17:57:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.858+0000 7fb07da72640 1 -- 172.21.15.111:0/2295591547 learned_addr learned my addr 172.21.15.111:0/2295591547 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.858+0000 7fb07e273640 1 -- 172.21.15.111:0/2295591547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb078069ed0 msgr2=0x7fb07806a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.858+0000 7fb07e273640 1 --2- 172.21.15.111:0/2295591547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb078069ed0 0x7fb07806a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.858+0000 7fb07e273640 1 -- 172.21.15.111:0/2295591547 >> v1:172.21.15.188:6789/0 conn(0x7fb07806aa70 legacy=0x7fb0780cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.858+0000 7fb07e273640 1 -- 172.21.15.111:0/2295591547 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0780cc180 con 0x7fb07815a480 2024-09-13T17:57:29.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.858+0000 7fb07d271640 1 -- 172.21.15.111:0/2295591547 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267425300 0 0) 0x7fb0780d0210 con 0x7fb07806aa70 2024-09-13T17:57:29.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.859+0000 7fb07e273640 1 --2- 172.21.15.111:0/2295591547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb07815a480 0x7fb07806ba60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb0740097a0 tx=0x7fb07402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27fa64fc300ed126 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:29.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.859+0000 7fb07d271640 1 -- 172.21.15.111:0/2295591547 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07402ae50 con 0x7fb07815a480 2024-09-13T17:57:29.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.859+0000 7fb07d271640 1 -- 172.21.15.111:0/2295591547 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb074032440 con 0x7fb07815a480 2024-09-13T17:57:29.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.859+0000 7fb07d271640 1 -- 172.21.15.111:0/2295591547 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0740327d0 con 0x7fb07815a480 2024-09-13T17:57:29.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.860+0000 7fb0804d7640 1 -- 172.21.15.111:0/2295591547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb07815a480 msgr2=0x7fb07806ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:29.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.860+0000 7fb0804d7640 1 --2- 172.21.15.111:0/2295591547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb07815a480 0x7fb07806ba60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb0740097a0 tx=0x7fb07402a100 comp rx=0 tx=0).stop 2024-09-13T17:57:29.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.860+0000 7fb0804d7640 1 -- 172.21.15.111:0/2295591547 shutdown_connections 2024-09-13T17:57:29.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.860+0000 7fb0804d7640 1 --2- 172.21.15.111:0/2295591547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb078069ed0 0x7fb07806a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:29.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.860+0000 7fb0804d7640 1 --2- 172.21.15.111:0/2295591547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb07815a480 0x7fb07806ba60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:29.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.860+0000 7fb0804d7640 1 -- 172.21.15.111:0/2295591547 >> 172.21.15.111:0/2295591547 conn(0x7fb0780717e0 msgr2=0x7fb0780989d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:29.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.860+0000 7fb0804d7640 1 -- 172.21.15.111:0/2295591547 shutdown_connections 2024-09-13T17:57:29.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.860+0000 7fb0804d7640 1 -- 172.21.15.111:0/2295591547 wait complete. 2024-09-13T17:57:29.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.861+0000 7fb0804d7640 1 Processor -- start 2024-09-13T17:57:29.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.861+0000 7fb0804d7640 1 -- start start 2024-09-13T17:57:29.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.862+0000 7fb0804d7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb078069ed0 0x7fb07815d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:29.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.862+0000 7fb0804d7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb07806aa70 0x7fb078151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:29.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.862+0000 7fb0804d7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb07815a480 0x7fb07813d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:29.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.862+0000 7fb0804d7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb078154a50 con 0x7fb07806aa70 2024-09-13T17:57:29.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.862+0000 7fb0804d7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07815d8d0 con 0x7fb078069ed0 2024-09-13T17:57:29.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.862+0000 7fb0804d7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb078153dd0 con 0x7fb07815a480 2024-09-13T17:57:29.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.862+0000 7fb07da72640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb07806aa70 0x7fb078151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:29.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.862+0000 7fb07da72640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb07806aa70 0x7fb078151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36480/0 (socket says 172.21.15.111:36480) 2024-09-13T17:57:29.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.862+0000 7fb07da72640 1 -- 172.21.15.111:0/3997494014 learned_addr learned my addr 172.21.15.111:0/3997494014 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:29.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.862+0000 7fb07e273640 1 --2- 172.21.15.111:0/3997494014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb078069ed0 0x7fb07815d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:29.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.862+0000 7fb07ea74640 1 --2- 172.21.15.111:0/3997494014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb07815a480 0x7fb07813d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:29.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.862+0000 7fb07da72640 1 -- 172.21.15.111:0/3997494014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb07815a480 msgr2=0x7fb07813d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:29.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.862+0000 7fb07da72640 1 --2- 172.21.15.111:0/3997494014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb07815a480 0x7fb07813d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:29.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.862+0000 7fb07da72640 1 -- 172.21.15.111:0/3997494014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb078069ed0 msgr2=0x7fb07815d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:29.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.862+0000 7fb07da72640 1 --2- 172.21.15.111:0/3997494014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb078069ed0 0x7fb07815d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:29.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.862+0000 7fb07da72640 1 -- 172.21.15.111:0/3997494014 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb074009400 con 0x7fb07806aa70 2024-09-13T17:57:29.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.862+0000 7fb07da72640 1 --2- 172.21.15.111:0/3997494014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb07806aa70 0x7fb078151ac0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb06800c4d0 tx=0x7fb06800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:29.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.863+0000 7fb06effd640 1 -- 172.21.15.111:0/3997494014 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0680074e0 con 0x7fb07806aa70 2024-09-13T17:57:29.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.863+0000 7fb06effd640 1 -- 172.21.15.111:0/3997494014 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb068007660 con 0x7fb07806aa70 2024-09-13T17:57:29.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.863+0000 7fb06effd640 1 -- 172.21.15.111:0/3997494014 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb068004a20 con 0x7fb07806aa70 2024-09-13T17:57:29.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.863+0000 7fb0804d7640 1 -- 172.21.15.111:0/3997494014 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0781540b0 con 0x7fb07806aa70 2024-09-13T17:57:29.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.863+0000 7fb0804d7640 1 -- 172.21.15.111:0/3997494014 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb078154670 con 0x7fb07806aa70 2024-09-13T17:57:29.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.864+0000 7fb0804d7640 1 -- 172.21.15.111:0/3997494014 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07806c090 con 0x7fb07806aa70 2024-09-13T17:57:29.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.864+0000 7fb06effd640 1 -- 172.21.15.111:0/3997494014 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb068004b80 con 0x7fb07806aa70 2024-09-13T17:57:29.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.865+0000 7fb06effd640 1 --2- 172.21.15.111:0/3997494014 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb05c06ac00 0x7fb05c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:29.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.865+0000 7fb07e273640 1 --2- 172.21.15.111:0/3997494014 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb05c06ac00 0x7fb05c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:29.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.865+0000 7fb06effd640 1 -- 172.21.15.111:0/3997494014 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7fb0680888a0 con 0x7fb07806aa70 2024-09-13T17:57:29.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.865+0000 7fb07e273640 1 --2- 172.21.15.111:0/3997494014 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb05c06ac00 0x7fb05c06d0c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb074034000 tx=0x7fb07402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:29.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:29.869+0000 7fb06effd640 1 -- 172.21.15.111:0/3997494014 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fb068053720 con 0x7fb07806aa70 2024-09-13T17:57:30.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.064+0000 7fb0804d7640 1 -- 172.21.15.111:0/3997494014 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fb07815a860 con 0x7fb07806aa70 2024-09-13T17:57:30.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.068+0000 7fb06effd640 1 -- 172.21.15.111:0/3997494014 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7fb068018020 con 0x7fb07806aa70 2024-09-13T17:57:30.073 INFO:teuthology.orchestra.run.smithi111.stderr:added data pool 4 to fsmap 2024-09-13T17:57:30.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.070+0000 7fb0804d7640 1 -- 172.21.15.111:0/3997494014 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb05c06ac00 msgr2=0x7fb05c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.070+0000 7fb0804d7640 1 --2- 172.21.15.111:0/3997494014 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb05c06ac00 0x7fb05c06d0c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb074034000 tx=0x7fb07402e000 comp rx=0 tx=0).stop 2024-09-13T17:57:30.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.070+0000 7fb0804d7640 1 -- 172.21.15.111:0/3997494014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb07806aa70 msgr2=0x7fb078151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.070+0000 7fb0804d7640 1 --2- 172.21.15.111:0/3997494014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb07806aa70 0x7fb078151ac0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb06800c4d0 tx=0x7fb06800c990 comp rx=0 tx=0).stop 2024-09-13T17:57:30.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.070+0000 7fb0804d7640 1 -- 172.21.15.111:0/3997494014 shutdown_connections 2024-09-13T17:57:30.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.070+0000 7fb0804d7640 1 --2- 172.21.15.111:0/3997494014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb07815a480 0x7fb07813d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.070+0000 7fb0804d7640 1 --2- 172.21.15.111:0/3997494014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb07806aa70 0x7fb078151ac0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.070+0000 7fb0804d7640 1 --2- 172.21.15.111:0/3997494014 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb05c06ac00 0x7fb05c06d0c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.070+0000 7fb0804d7640 1 --2- 172.21.15.111:0/3997494014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb078069ed0 0x7fb07815d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.070+0000 7fb0804d7640 1 -- 172.21.15.111:0/3997494014 >> 172.21.15.111:0/3997494014 conn(0x7fb0780717e0 msgr2=0x7fb078157c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:30.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.070+0000 7fb0804d7640 1 -- 172.21.15.111:0/3997494014 shutdown_connections 2024-09-13T17:57:30.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.070+0000 7fb0804d7640 1 -- 172.21.15.111:0/3997494014 wait complete. 2024-09-13T17:57:30.084 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:57:30.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.180+0000 7fde1a19d640 1 Processor -- start 2024-09-13T17:57:30.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.181+0000 7fde1a19d640 1 -- start start 2024-09-13T17:57:30.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.181+0000 7fde1a19d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fde140cdf80 0x7fde140ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde1a19d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde140d0cf0 con 0x7fde140cfc40 2024-09-13T17:57:30.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde1a19d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde140d00e0 con 0x7fde1406e8c0 2024-09-13T17:57:30.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde1a19d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1406cff0 con 0x7fde140cdf80 2024-09-13T17:57:30.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde1899a640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fde140cfc40 0x7fde1406e1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58084/0 (socket says 172.21.15.111:58084) 2024-09-13T17:57:30.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde1899a640 1 -- 172.21.15.111:0/2779213794 learned_addr learned my addr 172.21.15.111:0/2779213794 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:30.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde07fff640 1 -- 172.21.15.111:0/2779213794 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078496875 0 0) 0x7fde140d0cf0 con 0x7fde140cfc40 2024-09-13T17:57:30.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde07fff640 1 -- 172.21.15.111:0/2779213794 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde00003520 con 0x7fde140cfc40 2024-09-13T17:57:30.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde1919b640 1 --2- 172.21.15.111:0/2779213794 >> v2:172.21.15.188:3300/0 conn(0x7fde140cdf80 0x7fde140ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde07fff640 1 -- 172.21.15.111:0/2779213794 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1676954500 0 0) 0x7fde00003520 con 0x7fde140cfc40 2024-09-13T17:57:30.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde07fff640 1 -- 172.21.15.111:0/2779213794 >> v2:172.21.15.188:3300/0 conn(0x7fde140cdf80 msgr2=0x7fde140ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:30.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde07fff640 1 --2- 172.21.15.111:0/2779213794 >> v2:172.21.15.188:3300/0 conn(0x7fde140cdf80 0x7fde140ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde07fff640 1 -- 172.21.15.111:0/2779213794 >> v1:172.21.15.188:6789/0 conn(0x7fde1406e8c0 legacy=0x7fde14166320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde07fff640 1 -- 172.21.15.111:0/2779213794 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde140d03c0 con 0x7fde140cfc40 2024-09-13T17:57:30.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde07fff640 1 -- 172.21.15.111:0/2779213794 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7fde08001670 con 0x7fde140cfc40 2024-09-13T17:57:30.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde07fff640 1 -- 172.21.15.111:0/2779213794 >> v1:172.21.15.111:6789/0 conn(0x7fde140cfc40 legacy=0x7fde1406e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde07fff640 1 --2- 172.21.15.111:0/2779213794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde00003970 0x7fde00003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde07fff640 1 --2- 172.21.15.111:0/2779213794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde000047c0 0x7fde00004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde07fff640 1 --2- 172.21.15.111:0/2779213794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde00005180 0x7fde0000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde07fff640 1 -- 172.21.15.111:0/2779213794 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1406cff0 con 0x7fde000047c0 2024-09-13T17:57:30.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde07fff640 1 -- 172.21.15.111:0/2779213794 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0000c0d0 con 0x7fde00005180 2024-09-13T17:57:30.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde07fff640 1 -- 172.21.15.111:0/2779213794 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0000c290 con 0x7fde00003970 2024-09-13T17:57:30.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde1899a640 1 --2- 172.21.15.111:0/2779213794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde000047c0 0x7fde00004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde1999c640 1 --2- 172.21.15.111:0/2779213794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde00005180 0x7fde0000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde1919b640 1 --2- 172.21.15.111:0/2779213794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde00003970 0x7fde00003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.182+0000 7fde1899a640 1 -- 172.21.15.111:0/2779213794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde00003970 msgr2=0x7fde00003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:30.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.183+0000 7fde1899a640 1 --2- 172.21.15.111:0/2779213794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde00003970 0x7fde00003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.183+0000 7fde1899a640 1 -- 172.21.15.111:0/2779213794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde00005180 msgr2=0x7fde0000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:30.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.183+0000 7fde1899a640 1 --2- 172.21.15.111:0/2779213794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde00005180 0x7fde0000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.183+0000 7fde1899a640 1 -- 172.21.15.111:0/2779213794 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde00003520 con 0x7fde000047c0 2024-09-13T17:57:30.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.183+0000 7fde1999c640 1 -- 172.21.15.111:0/2779213794 reap_dead start 2024-09-13T17:57:30.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.183+0000 7fde1919b640 1 --2- 172.21.15.111:0/2779213794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde00003970 0x7fde00003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:30.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.183+0000 7fde1899a640 1 --2- 172.21.15.111:0/2779213794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde000047c0 0x7fde00004c40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fde0800c240 tx=0x7fde000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d9a856860fa917 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:30.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.183+0000 7fde07fff640 1 -- 172.21.15.111:0/2779213794 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0802e070 con 0x7fde000047c0 2024-09-13T17:57:30.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.183+0000 7fde07fff640 1 -- 172.21.15.111:0/2779213794 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde080058d0 con 0x7fde000047c0 2024-09-13T17:57:30.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.183+0000 7fde07fff640 1 -- 172.21.15.111:0/2779213794 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08005ca0 con 0x7fde000047c0 2024-09-13T17:57:30.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.184+0000 7fde1a19d640 1 -- 172.21.15.111:0/2779213794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde000047c0 msgr2=0x7fde00004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.184+0000 7fde1a19d640 1 --2- 172.21.15.111:0/2779213794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde000047c0 0x7fde00004c40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fde0800c240 tx=0x7fde000057f0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.184+0000 7fde1a19d640 1 -- 172.21.15.111:0/2779213794 shutdown_connections 2024-09-13T17:57:30.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.184+0000 7fde1a19d640 1 --2- 172.21.15.111:0/2779213794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde000047c0 0x7fde00004c40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.184+0000 7fde1a19d640 1 -- 172.21.15.111:0/2779213794 >> 172.21.15.111:0/2779213794 conn(0x7fde14069ed0 msgr2=0x7fde1406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:30.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.184+0000 7fde1a19d640 1 -- 172.21.15.111:0/2779213794 shutdown_connections 2024-09-13T17:57:30.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.184+0000 7fde1a19d640 1 -- 172.21.15.111:0/2779213794 wait complete. 2024-09-13T17:57:30.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.185+0000 7fde1a19d640 1 Processor -- start 2024-09-13T17:57:30.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.185+0000 7fde1a19d640 1 -- start start 2024-09-13T17:57:30.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde1a19d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde000047c0 0x7fde140ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde1a19d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde1414f9b0 0x7fde1415e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde1a19d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde14160a00 0x7fde1415e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde1a19d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde140adda0 con 0x7fde14160a00 2024-09-13T17:57:30.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde1a19d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde140ae080 con 0x7fde000047c0 2024-09-13T17:57:30.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde1a19d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde140ae360 con 0x7fde1414f9b0 2024-09-13T17:57:30.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde1999c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde14160a00 0x7fde1415e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde1999c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde14160a00 0x7fde1415e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36506/0 (socket says 172.21.15.111:36506) 2024-09-13T17:57:30.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde1999c640 1 -- 172.21.15.111:0/2451651284 learned_addr learned my addr 172.21.15.111:0/2451651284 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:30.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde1919b640 1 --2- 172.21.15.111:0/2451651284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde000047c0 0x7fde140ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde1899a640 1 --2- 172.21.15.111:0/2451651284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde1414f9b0 0x7fde1415e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde1999c640 1 -- 172.21.15.111:0/2451651284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde1414f9b0 msgr2=0x7fde1415e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde1999c640 1 --2- 172.21.15.111:0/2451651284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde1414f9b0 0x7fde1415e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde1999c640 1 -- 172.21.15.111:0/2451651284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde000047c0 msgr2=0x7fde140ad4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde1999c640 1 --2- 172.21.15.111:0/2451651284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde000047c0 0x7fde140ad4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde1999c640 1 -- 172.21.15.111:0/2451651284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde140ae640 con 0x7fde14160a00 2024-09-13T17:57:30.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde1999c640 1 --2- 172.21.15.111:0/2451651284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde14160a00 0x7fde1415e620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fde1406ace0 tx=0x7fde0c00fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:30.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.186+0000 7fde05ffb640 1 -- 172.21.15.111:0/2451651284 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c006d60 con 0x7fde14160a00 2024-09-13T17:57:30.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.187+0000 7fde05ffb640 1 -- 172.21.15.111:0/2451651284 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0c003970 con 0x7fde14160a00 2024-09-13T17:57:30.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.187+0000 7fde1a19d640 1 -- 172.21.15.111:0/2451651284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde140ae920 con 0x7fde14160a00 2024-09-13T17:57:30.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.187+0000 7fde05ffb640 1 -- 172.21.15.111:0/2451651284 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c003d20 con 0x7fde14160a00 2024-09-13T17:57:30.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.187+0000 7fde1a19d640 1 -- 172.21.15.111:0/2451651284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde140aee40 con 0x7fde14160a00 2024-09-13T17:57:30.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.188+0000 7fde1a19d640 1 -- 172.21.15.111:0/2451651284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde14066b00 con 0x7fde14160a00 2024-09-13T17:57:30.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.188+0000 7fde05ffb640 1 -- 172.21.15.111:0/2451651284 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fde0c005110 con 0x7fde14160a00 2024-09-13T17:57:30.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.188+0000 7fde05ffb640 1 --2- 172.21.15.111:0/2451651284 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fddf806ac00 0x7fddf806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.189+0000 7fde1919b640 1 --2- 172.21.15.111:0/2451651284 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fddf806ac00 0x7fddf806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.189+0000 7fde05ffb640 1 -- 172.21.15.111:0/2451651284 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fde0c088d30 con 0x7fde14160a00 2024-09-13T17:57:30.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.189+0000 7fde1919b640 1 --2- 172.21.15.111:0/2451651284 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fddf806ac00 0x7fddf806d0c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde1404f9f0 tx=0x7fde10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:30.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.193+0000 7fde05ffb640 1 -- 172.21.15.111:0/2451651284 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fde0c053d60 con 0x7fde14160a00 2024-09-13T17:57:30.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.388+0000 7fde1a19d640 1 -- 172.21.15.111:0/2451651284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde140cef40 con 0x7fde14160a00 2024-09-13T17:57:30.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.389+0000 7fde05ffb640 1 -- 172.21.15.111:0/2451651284 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4742 (secure 0 0 0) 0x7fde0c01a090 con 0x7fde14160a00 2024-09-13T17:57:30.391 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:30.391 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4001406804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":4001406804},{"type":"v1","addr":"172.21.15.188:6835","nonce":4001406804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2565466103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2565466103},{"type":"v1","addr":"172.21.15.111:6837","nonce":2565466103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/1252690799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1252690799},{"type":"v1","addr":"172.21.15.111:6836","nonce":1252690799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:57:25.702997+0000","modified":"2024-09-13T17:57:30.065468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/1266310843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1266310843},{"type":"v1","addr":"172.21.15.188:6834","nonce":1266310843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-09-13T17:57:30.391 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 7 2024-09-13T17:57:30.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.391+0000 7fde1a19d640 1 -- 172.21.15.111:0/2451651284 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fddf806ac00 msgr2=0x7fddf806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.391+0000 7fde1a19d640 1 --2- 172.21.15.111:0/2451651284 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fddf806ac00 0x7fddf806d0c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde1404f9f0 tx=0x7fde10008040 comp rx=0 tx=0).stop 2024-09-13T17:57:30.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.391+0000 7fde1a19d640 1 -- 172.21.15.111:0/2451651284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde14160a00 msgr2=0x7fde1415e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.391+0000 7fde1a19d640 1 --2- 172.21.15.111:0/2451651284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde14160a00 0x7fde1415e620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fde1406ace0 tx=0x7fde0c00fc90 comp rx=0 tx=0).stop 2024-09-13T17:57:30.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.391+0000 7fde1a19d640 1 -- 172.21.15.111:0/2451651284 shutdown_connections 2024-09-13T17:57:30.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.391+0000 7fde1a19d640 1 --2- 172.21.15.111:0/2451651284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde14160a00 0x7fde1415e620 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.391+0000 7fde1a19d640 1 --2- 172.21.15.111:0/2451651284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde1414f9b0 0x7fde1415e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.391+0000 7fde1a19d640 1 --2- 172.21.15.111:0/2451651284 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fddf806ac00 0x7fddf806d0c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.391+0000 7fde1a19d640 1 --2- 172.21.15.111:0/2451651284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde000047c0 0x7fde140ad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.391+0000 7fde1a19d640 1 -- 172.21.15.111:0/2451651284 >> 172.21.15.111:0/2451651284 conn(0x7fde14069ed0 msgr2=0x7fde140b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:30.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.391+0000 7fde1a19d640 1 -- 172.21.15.111:0/2451651284 shutdown_connections 2024-09-13T17:57:30.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.391+0000 7fde1a19d640 1 -- 172.21.15.111:0/2451651284 wait complete. 2024-09-13T17:57:30.404 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:30.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.499+0000 7fdbe8696640 1 Processor -- start 2024-09-13T17:57:30.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.499+0000 7fdbe8696640 1 -- start start 2024-09-13T17:57:30.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.500+0000 7fdbe8696640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbe006b5f0 0x7fdbe00abae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.500+0000 7fdbe8696640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdbe015c660 0x7fdbe00ac020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.500+0000 7fdbe8696640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe0066090 con 0x7fdbe015c280 2024-09-13T17:57:30.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.500+0000 7fdbe8696640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe00ac6f0 con 0x7fdbe015c660 2024-09-13T17:57:30.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.500+0000 7fdbe8696640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe00ac9d0 con 0x7fdbe006b5f0 2024-09-13T17:57:30.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.500+0000 7fdbe6c33640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdbe015c660 0x7fdbe00ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.500+0000 7fdbe5c31640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbe006b5f0 0x7fdbe00abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.500+0000 7fdbe6c33640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdbe015c660 0x7fdbe00ac020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:45830/0 (socket says 172.21.15.111:45830) 2024-09-13T17:57:30.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.500+0000 7fdbe6c33640 1 -- 172.21.15.111:0/163020058 learned_addr learned my addr 172.21.15.111:0/163020058 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:30.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.500+0000 7fdbe6432640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fdbe015c280 0x7fdbe006aee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:46726/0 (socket says 172.21.15.111:46726) 2024-09-13T17:57:30.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.501+0000 7fdbe5430640 1 -- 172.21.15.111:0/163020058 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3530697649 0 0) 0x7fdbe0066090 con 0x7fdbe015c280 2024-09-13T17:57:30.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.501+0000 7fdbe5c31640 1 -- 172.21.15.111:0/163020058 >> v2:172.21.15.188:3300/0 conn(0x7fdbe015c660 msgr2=0x7fdbe00ac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:30.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.501+0000 7fdbe5c31640 1 --2- 172.21.15.111:0/163020058 >> v2:172.21.15.188:3300/0 conn(0x7fdbe015c660 0x7fdbe00ac020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.501+0000 7fdbe5c31640 1 -- 172.21.15.111:0/163020058 >> v1:172.21.15.188:6789/0 conn(0x7fdbe015c280 legacy=0x7fdbe006aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.501+0000 7fdbe5c31640 1 -- 172.21.15.111:0/163020058 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe00accb0 con 0x7fdbe006b5f0 2024-09-13T17:57:30.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.501+0000 7fdbe5c31640 1 --2- 172.21.15.111:0/163020058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbe006b5f0 0x7fdbe00abae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00097a0 tx=0x7fdbd002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f645d9beeec5222 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:30.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.501+0000 7fdbe5430640 1 -- 172.21.15.111:0/163020058 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0032440 con 0x7fdbe006b5f0 2024-09-13T17:57:30.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.502+0000 7fdbe5430640 1 -- 172.21.15.111:0/163020058 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd00325a0 con 0x7fdbe006b5f0 2024-09-13T17:57:30.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.502+0000 7fdbe5430640 1 -- 172.21.15.111:0/163020058 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0032930 con 0x7fdbe006b5f0 2024-09-13T17:57:30.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.502+0000 7fdbe8696640 1 -- 172.21.15.111:0/163020058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbe006b5f0 msgr2=0x7fdbe00abae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.502+0000 7fdbe8696640 1 --2- 172.21.15.111:0/163020058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbe006b5f0 0x7fdbe00abae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00097a0 tx=0x7fdbd002a180 comp rx=0 tx=0).stop 2024-09-13T17:57:30.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.502+0000 7fdbe8696640 1 -- 172.21.15.111:0/163020058 shutdown_connections 2024-09-13T17:57:30.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.502+0000 7fdbe8696640 1 --2- 172.21.15.111:0/163020058 >> v2:172.21.15.188:3300/0 conn(0x7fdbe015c660 0x7fdbe00ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.502+0000 7fdbe8696640 1 --2- 172.21.15.111:0/163020058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbe006b5f0 0x7fdbe00abae0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.502+0000 7fdbe8696640 1 -- 172.21.15.111:0/163020058 >> 172.21.15.111:0/163020058 conn(0x7fdbe00709f0 msgr2=0x7fdbe0070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:30.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.502+0000 7fdbe8696640 1 -- 172.21.15.111:0/163020058 shutdown_connections 2024-09-13T17:57:30.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.502+0000 7fdbe8696640 1 -- 172.21.15.111:0/163020058 wait complete. 2024-09-13T17:57:30.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.503+0000 7fdbe8696640 1 Processor -- start 2024-09-13T17:57:30.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.503+0000 7fdbe8696640 1 -- start start 2024-09-13T17:57:30.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.504+0000 7fdbe8696640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbe006b5f0 0x7fdbe014c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.504+0000 7fdbe8696640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbe015c280 0x7fdbe00c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.504+0000 7fdbe8696640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbe015c660 0x7fdbe00c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.504+0000 7fdbe8696640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe014cb30 con 0x7fdbe015c660 2024-09-13T17:57:30.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.504+0000 7fdbe8696640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe00c4600 con 0x7fdbe015c280 2024-09-13T17:57:30.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.504+0000 7fdbe8696640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe00c48e0 con 0x7fdbe006b5f0 2024-09-13T17:57:30.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.504+0000 7fdbe6c33640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbe015c660 0x7fdbe00c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.504+0000 7fdbe6c33640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbe015c660 0x7fdbe00c3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36516/0 (socket says 172.21.15.111:36516) 2024-09-13T17:57:30.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.504+0000 7fdbe6c33640 1 -- 172.21.15.111:0/2014623904 learned_addr learned my addr 172.21.15.111:0/2014623904 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:30.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.504+0000 7fdbe6432640 1 --2- 172.21.15.111:0/2014623904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbe006b5f0 0x7fdbe014c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.504+0000 7fdbe5c31640 1 --2- 172.21.15.111:0/2014623904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbe015c280 0x7fdbe00c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.504+0000 7fdbe6c33640 1 -- 172.21.15.111:0/2014623904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbe006b5f0 msgr2=0x7fdbe014c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.504+0000 7fdbe6c33640 1 --2- 172.21.15.111:0/2014623904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbe006b5f0 0x7fdbe014c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.504+0000 7fdbe6c33640 1 -- 172.21.15.111:0/2014623904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbe015c280 msgr2=0x7fdbe00c3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.504+0000 7fdbe6c33640 1 --2- 172.21.15.111:0/2014623904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbe015c280 0x7fdbe00c3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.504+0000 7fdbe6c33640 1 -- 172.21.15.111:0/2014623904 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd0009400 con 0x7fdbe015c660 2024-09-13T17:57:30.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.505+0000 7fdbe5c31640 1 --2- 172.21.15.111:0/2014623904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbe015c280 0x7fdbe00c3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:30.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.505+0000 7fdbe6c33640 1 --2- 172.21.15.111:0/2014623904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbe015c660 0x7fdbe00c3fd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800e840 tx=0x7fdbd800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:30.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.505+0000 7fdbd6ffd640 1 -- 172.21.15.111:0/2014623904 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd800c730 con 0x7fdbe015c660 2024-09-13T17:57:30.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.505+0000 7fdbd6ffd640 1 -- 172.21.15.111:0/2014623904 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd800c8b0 con 0x7fdbe015c660 2024-09-13T17:57:30.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.505+0000 7fdbe8696640 1 -- 172.21.15.111:0/2014623904 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe00c4bc0 con 0x7fdbe015c660 2024-09-13T17:57:30.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.505+0000 7fdbd6ffd640 1 -- 172.21.15.111:0/2014623904 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd800cc60 con 0x7fdbe015c660 2024-09-13T17:57:30.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.505+0000 7fdbe8696640 1 -- 172.21.15.111:0/2014623904 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe00c5180 con 0x7fdbe015c660 2024-09-13T17:57:30.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.506+0000 7fdbe8696640 1 -- 172.21.15.111:0/2014623904 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe00c5370 con 0x7fdbe015c660 2024-09-13T17:57:30.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.506+0000 7fdbd6ffd640 1 -- 172.21.15.111:0/2014623904 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdbd8013470 con 0x7fdbe015c660 2024-09-13T17:57:30.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.507+0000 7fdbd6ffd640 1 --2- 172.21.15.111:0/2014623904 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdbc406ac00 0x7fdbc406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.507+0000 7fdbd6ffd640 1 -- 172.21.15.111:0/2014623904 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdbd8089800 con 0x7fdbe015c660 2024-09-13T17:57:30.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.507+0000 7fdbe6432640 1 --2- 172.21.15.111:0/2014623904 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdbc406ac00 0x7fdbc406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.507+0000 7fdbe6432640 1 --2- 172.21.15.111:0/2014623904 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdbc406ac00 0x7fdbc406d0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0012e0 tx=0x7fdbdc007ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:30.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.511+0000 7fdbd6ffd640 1 -- 172.21.15.111:0/2014623904 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fdbd8054660 con 0x7fdbe015c660 2024-09-13T17:57:30.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.672+0000 7fdbe8696640 1 -- 172.21.15.111:0/2014623904 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbe006aee0 con 0x7fdbe015c660 2024-09-13T17:57:30.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.673+0000 7fdbd6ffd640 1 -- 172.21.15.111:0/2014623904 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7fdbd8087e60 con 0x7fdbe015c660 2024-09-13T17:57:30.676 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:30.676 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:30.065235+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:30.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.675+0000 7fdbe8696640 1 -- 172.21.15.111:0/2014623904 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdbc406ac00 msgr2=0x7fdbc406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.675+0000 7fdbe8696640 1 --2- 172.21.15.111:0/2014623904 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdbc406ac00 0x7fdbc406d0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0012e0 tx=0x7fdbdc007ed0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.675+0000 7fdbe8696640 1 -- 172.21.15.111:0/2014623904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbe015c660 msgr2=0x7fdbe00c3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.675+0000 7fdbe8696640 1 --2- 172.21.15.111:0/2014623904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbe015c660 0x7fdbe00c3fd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800e840 tx=0x7fdbd800ed00 comp rx=0 tx=0).stop 2024-09-13T17:57:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.675+0000 7fdbe8696640 1 -- 172.21.15.111:0/2014623904 shutdown_connections 2024-09-13T17:57:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.675+0000 7fdbe8696640 1 --2- 172.21.15.111:0/2014623904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbe015c660 0x7fdbe00c3fd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.675+0000 7fdbe8696640 1 --2- 172.21.15.111:0/2014623904 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdbc406ac00 0x7fdbc406d0c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.675+0000 7fdbe8696640 1 --2- 172.21.15.111:0/2014623904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbe015c280 0x7fdbe00c3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.675+0000 7fdbe8696640 1 --2- 172.21.15.111:0/2014623904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbe006b5f0 0x7fdbe014c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.675+0000 7fdbe8696640 1 -- 172.21.15.111:0/2014623904 >> 172.21.15.111:0/2014623904 conn(0x7fdbe00709f0 msgr2=0x7fdbe00aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.676+0000 7fdbe8696640 1 -- 172.21.15.111:0/2014623904 shutdown_connections 2024-09-13T17:57:30.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.676+0000 7fdbe8696640 1 -- 172.21.15.111:0/2014623904 wait complete. 2024-09-13T17:57:30.689 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:30.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.785+0000 7fd197b2c640 1 Processor -- start 2024-09-13T17:57:30.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.785+0000 7fd197b2c640 1 -- start start 2024-09-13T17:57:30.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.786+0000 7fd197b2c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd19015c280 0x7fd19006aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.786+0000 7fd197b2c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd190066090 con 0x7fd19015c660 2024-09-13T17:57:30.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.786+0000 7fd197b2c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1900ac6f0 con 0x7fd1900ac1f0 2024-09-13T17:57:30.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.786+0000 7fd197b2c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900ac9d0 con 0x7fd19015c280 2024-09-13T17:57:30.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.786+0000 7fd1950c7640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd19015c660 0x7fd1900abae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58086/0 (socket says 172.21.15.111:58086) 2024-09-13T17:57:30.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.786+0000 7fd1950c7640 1 -- 172.21.15.111:0/666154157 learned_addr learned my addr 172.21.15.111:0/666154157 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:30.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.786+0000 7fd1948c6640 1 -- 172.21.15.111:0/666154157 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951757394 0 0) 0x7fd190066090 con 0x7fd19015c660 2024-09-13T17:57:30.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.786+0000 7fd1948c6640 1 -- 172.21.15.111:0/666154157 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd17c003520 con 0x7fd19015c660 2024-09-13T17:57:30.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.786+0000 7fd1958c8640 1 --2- 172.21.15.111:0/666154157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd19015c280 0x7fd19006aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1948c6640 1 -- 172.21.15.111:0/666154157 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 963072689 0 0) 0x7fd17c003520 con 0x7fd19015c660 2024-09-13T17:57:30.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1948c6640 1 -- 172.21.15.111:0/666154157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd19015c280 msgr2=0x7fd19006aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:30.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1948c6640 1 --2- 172.21.15.111:0/666154157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd19015c280 0x7fd19006aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1948c6640 1 -- 172.21.15.111:0/666154157 >> v1:172.21.15.188:6789/0 conn(0x7fd1900ac1f0 legacy=0x7fd1900b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1948c6640 1 -- 172.21.15.111:0/666154157 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1900accb0 con 0x7fd19015c660 2024-09-13T17:57:30.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1948c6640 1 -- 172.21.15.111:0/666154157 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7fd180001670 con 0x7fd19015c660 2024-09-13T17:57:30.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1948c6640 1 -- 172.21.15.111:0/666154157 >> v1:172.21.15.111:6789/0 conn(0x7fd19015c660 legacy=0x7fd1900abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1948c6640 1 --2- 172.21.15.111:0/666154157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd17c0039a0 0x7fd17c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1948c6640 1 --2- 172.21.15.111:0/666154157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd17c0047c0 0x7fd17c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1948c6640 1 --2- 172.21.15.111:0/666154157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd19015c280 0x7fd17c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1948c6640 1 -- 172.21.15.111:0/666154157 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900ac9d0 con 0x7fd17c0039a0 2024-09-13T17:57:30.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1948c6640 1 -- 172.21.15.111:0/666154157 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c00ba30 con 0x7fd17c0047c0 2024-09-13T17:57:30.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1948c6640 1 -- 172.21.15.111:0/666154157 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c00bbf0 con 0x7fd19015c280 2024-09-13T17:57:30.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1958c8640 1 --2- 172.21.15.111:0/666154157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd17c0039a0 0x7fd17c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1950c7640 1 --2- 172.21.15.111:0/666154157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd17c0047c0 0x7fd17c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1960c9640 1 --2- 172.21.15.111:0/666154157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd19015c280 0x7fd17c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1958c8640 1 -- 172.21.15.111:0/666154157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd19015c280 msgr2=0x7fd17c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:30.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1958c8640 1 --2- 172.21.15.111:0/666154157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd19015c280 0x7fd17c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1958c8640 1 -- 172.21.15.111:0/666154157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd17c0047c0 msgr2=0x7fd17c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:30.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1958c8640 1 --2- 172.21.15.111:0/666154157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd17c0047c0 0x7fd17c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1958c8640 1 -- 172.21.15.111:0/666154157 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c003520 con 0x7fd17c0039a0 2024-09-13T17:57:30.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1960c9640 1 --2- 172.21.15.111:0/666154157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd19015c280 0x7fd17c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:30.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.787+0000 7fd1950c7640 1 --2- 172.21.15.111:0/666154157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd17c0047c0 0x7fd17c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:30.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.788+0000 7fd1958c8640 1 --2- 172.21.15.111:0/666154157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd17c0039a0 0x7fd17c003dc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009f90 tx=0x7fd17c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a085b2fdd140279f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:30.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.788+0000 7fd1948c6640 1 -- 172.21.15.111:0/666154157 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c0315f0 con 0x7fd17c0039a0 2024-09-13T17:57:30.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.788+0000 7fd1948c6640 1 -- 172.21.15.111:0/666154157 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18c00f040 con 0x7fd17c0039a0 2024-09-13T17:57:30.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.788+0000 7fd1948c6640 1 -- 172.21.15.111:0/666154157 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c031990 con 0x7fd17c0039a0 2024-09-13T17:57:30.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.788+0000 7fd197b2c640 1 -- 172.21.15.111:0/666154157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd17c0039a0 msgr2=0x7fd17c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.788+0000 7fd197b2c640 1 --2- 172.21.15.111:0/666154157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd17c0039a0 0x7fd17c003dc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009f90 tx=0x7fd17c0057f0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.788+0000 7fd1960c9640 1 -- 172.21.15.111:0/666154157 reap_dead start 2024-09-13T17:57:30.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.789+0000 7fd197b2c640 1 -- 172.21.15.111:0/666154157 shutdown_connections 2024-09-13T17:57:30.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.789+0000 7fd197b2c640 1 -- 172.21.15.111:0/666154157 >> 172.21.15.111:0/666154157 conn(0x7fd1900709f0 msgr2=0x7fd190070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:30.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.789+0000 7fd197b2c640 1 -- 172.21.15.111:0/666154157 shutdown_connections 2024-09-13T17:57:30.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.789+0000 7fd197b2c640 1 -- 172.21.15.111:0/666154157 wait complete. 2024-09-13T17:57:30.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.790+0000 7fd197b2c640 1 Processor -- start 2024-09-13T17:57:30.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.790+0000 7fd197b2c640 1 -- start start 2024-09-13T17:57:30.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.790+0000 7fd197b2c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd17c0039a0 0x7fd190158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd197b2c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd190152700 0x7fd19015edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd197b2c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd190153c30 0x7fd1901483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd197b2c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd190158940 con 0x7fd17c0039a0 2024-09-13T17:57:30.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd197b2c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd190151b70 con 0x7fd190152700 2024-09-13T17:57:30.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd197b2c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd190152b90 con 0x7fd190153c30 2024-09-13T17:57:30.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd1958c8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd17c0039a0 0x7fd190158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd1958c8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd17c0039a0 0x7fd190158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36530/0 (socket says 172.21.15.111:36530) 2024-09-13T17:57:30.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd1958c8640 1 -- 172.21.15.111:0/1976404601 learned_addr learned my addr 172.21.15.111:0/1976404601 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:30.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd1950c7640 1 --2- 172.21.15.111:0/1976404601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd190152700 0x7fd19015edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd1960c9640 1 --2- 172.21.15.111:0/1976404601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd190153c30 0x7fd1901483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd1958c8640 1 -- 172.21.15.111:0/1976404601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd190153c30 msgr2=0x7fd1901483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd1958c8640 1 --2- 172.21.15.111:0/1976404601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd190153c30 0x7fd1901483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd1958c8640 1 -- 172.21.15.111:0/1976404601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd190152700 msgr2=0x7fd19015edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd1958c8640 1 --2- 172.21.15.111:0/1976404601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd190152700 0x7fd19015edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd1958c8640 1 -- 172.21.15.111:0/1976404601 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd190152e70 con 0x7fd17c0039a0 2024-09-13T17:57:30.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd1950c7640 1 --2- 172.21.15.111:0/1976404601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd190152700 0x7fd19015edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:30.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd1958c8640 1 --2- 172.21.15.111:0/1976404601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd17c0039a0 0x7fd190158400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd18c033040 tx=0x7fd18c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:30.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd1867fc640 1 -- 172.21.15.111:0/1976404601 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c045630 con 0x7fd17c0039a0 2024-09-13T17:57:30.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd1867fc640 1 -- 172.21.15.111:0/1976404601 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18c040070 con 0x7fd17c0039a0 2024-09-13T17:57:30.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd197b2c640 1 -- 172.21.15.111:0/1976404601 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd190153150 con 0x7fd17c0039a0 2024-09-13T17:57:30.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd1867fc640 1 -- 172.21.15.111:0/1976404601 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c045990 con 0x7fd17c0039a0 2024-09-13T17:57:30.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.791+0000 7fd197b2c640 1 -- 172.21.15.111:0/1976404601 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd190153710 con 0x7fd17c0039a0 2024-09-13T17:57:30.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.792+0000 7fd197b2c640 1 -- 172.21.15.111:0/1976404601 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1901538a0 con 0x7fd17c0039a0 2024-09-13T17:57:30.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.793+0000 7fd1867fc640 1 -- 172.21.15.111:0/1976404601 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd18c042030 con 0x7fd17c0039a0 2024-09-13T17:57:30.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.793+0000 7fd1867fc640 1 --2- 172.21.15.111:0/1976404601 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd17406ac00 0x7fd17406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.793+0000 7fd1867fc640 1 -- 172.21.15.111:0/1976404601 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd18c02e050 con 0x7fd17c0039a0 2024-09-13T17:57:30.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.793+0000 7fd1950c7640 1 --2- 172.21.15.111:0/1976404601 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd17406ac00 0x7fd17406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.794+0000 7fd1950c7640 1 --2- 172.21.15.111:0/1976404601 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd17406ac00 0x7fd17406d0c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd190151820 tx=0x7fd180009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:30.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.797+0000 7fd1867fc640 1 -- 172.21.15.111:0/1976404601 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fd18c075730 con 0x7fd17c0039a0 2024-09-13T17:57:30.807 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-09-13T17:57:30.806+0000 7f1289ce4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:57:30.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.958+0000 7fd197b2c640 1 -- 172.21.15.111:0/1976404601 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1900ad3f0 con 0x7fd17c0039a0 2024-09-13T17:57:30.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.959+0000 7fd1867fc640 1 -- 172.21.15.111:0/1976404601 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7fd18c078d50 con 0x7fd17c0039a0 2024-09-13T17:57:30.961 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:30.961 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:30.065235+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:30.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.961+0000 7fd197b2c640 1 -- 172.21.15.111:0/1976404601 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd17406ac00 msgr2=0x7fd17406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.961+0000 7fd197b2c640 1 --2- 172.21.15.111:0/1976404601 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd17406ac00 0x7fd17406d0c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd190151820 tx=0x7fd180009040 comp rx=0 tx=0).stop 2024-09-13T17:57:30.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.961+0000 7fd197b2c640 1 -- 172.21.15.111:0/1976404601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd17c0039a0 msgr2=0x7fd190158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.961+0000 7fd197b2c640 1 --2- 172.21.15.111:0/1976404601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd17c0039a0 0x7fd190158400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd18c033040 tx=0x7fd18c009750 comp rx=0 tx=0).stop 2024-09-13T17:57:30.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.961+0000 7fd197b2c640 1 -- 172.21.15.111:0/1976404601 shutdown_connections 2024-09-13T17:57:30.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.961+0000 7fd197b2c640 1 --2- 172.21.15.111:0/1976404601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd190153c30 0x7fd1901483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.961+0000 7fd197b2c640 1 --2- 172.21.15.111:0/1976404601 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd17406ac00 0x7fd17406d0c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.961+0000 7fd197b2c640 1 --2- 172.21.15.111:0/1976404601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd190152700 0x7fd19015edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.961+0000 7fd197b2c640 1 --2- 172.21.15.111:0/1976404601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd17c0039a0 0x7fd190158400 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.961+0000 7fd197b2c640 1 -- 172.21.15.111:0/1976404601 >> 172.21.15.111:0/1976404601 conn(0x7fd1900709f0 msgr2=0x7fd1900ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:30.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.961+0000 7fd197b2c640 1 -- 172.21.15.111:0/1976404601 shutdown_connections 2024-09-13T17:57:30.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:30.961+0000 7fd197b2c640 1 -- 172.21.15.111:0/1976404601 wait complete. 2024-09-13T17:57:30.991 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:57:31.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.087+0000 7f04ba7ae640 1 Processor -- start 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.087+0000 7f04ba7ae640 1 -- start start 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.088+0000 7f04ba7ae640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f04b4071d30 0x7f04b4070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.088+0000 7f04ba7ae640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04b40c8920 0x7f04b40c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.088+0000 7f04ba7ae640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b40abdf0 con 0x7f04b4072110 2024-09-13T17:57:31.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.088+0000 7f04ba7ae640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04b40c9310 con 0x7f04b4071d30 2024-09-13T17:57:31.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.088+0000 7f04ba7ae640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b40b2280 con 0x7f04b40c8920 2024-09-13T17:57:31.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.088+0000 7f04abfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f04b4072110 0x7f04b40b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58094/0 (socket says 172.21.15.111:58094) 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.088+0000 7f04abfff640 1 -- 172.21.15.111:0/4087602694 learned_addr learned my addr 172.21.15.111:0/4087602694 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.088+0000 7f04b3fff640 1 --2- 172.21.15.111:0/4087602694 >> v2:172.21.15.188:3300/0 conn(0x7f04b4071d30 0x7f04b4070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.088+0000 7f04b8d4b640 1 --2- 172.21.15.111:0/4087602694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04b40c8920 0x7f04b40c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.088+0000 7f04b37fe640 1 -- 172.21.15.111:0/4087602694 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745999890 0 0) 0x7f04b40abdf0 con 0x7f04b4072110 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.088+0000 7f04b37fe640 1 -- 172.21.15.111:0/4087602694 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04a0005730 con 0x7f04b4072110 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04b37fe640 1 -- 172.21.15.111:0/4087602694 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1037510669 0 0) 0x7f04a0005730 con 0x7f04b4072110 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04b37fe640 1 -- 172.21.15.111:0/4087602694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04b40c8920 msgr2=0x7f04b40c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04b37fe640 1 --2- 172.21.15.111:0/4087602694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04b40c8920 0x7f04b40c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04b37fe640 1 -- 172.21.15.111:0/4087602694 >> v2:172.21.15.188:3300/0 conn(0x7f04b4071d30 msgr2=0x7f04b4070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04b37fe640 1 --2- 172.21.15.111:0/4087602694 >> v2:172.21.15.188:3300/0 conn(0x7f04b4071d30 0x7f04b4070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04b37fe640 1 -- 172.21.15.111:0/4087602694 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b40ab1e0 con 0x7f04b4072110 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04b37fe640 1 -- 172.21.15.111:0/4087602694 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f049c001670 con 0x7f04b4072110 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04b37fe640 1 -- 172.21.15.111:0/4087602694 >> v1:172.21.15.111:6789/0 conn(0x7f04b4072110 legacy=0x7f04b40b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04b37fe640 1 --2- 172.21.15.111:0/4087602694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04a0005b80 0x7f04a0005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04b37fe640 1 --2- 172.21.15.111:0/4087602694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04a00069d0 0x7f04a0006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04b37fe640 1 --2- 172.21.15.111:0/4087602694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04b40c8920 0x7f04a0007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04b37fe640 1 -- 172.21.15.111:0/4087602694 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b40c9310 con 0x7f04a00069d0 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04b37fe640 1 -- 172.21.15.111:0/4087602694 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b40b2280 con 0x7f04a0005b80 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04b37fe640 1 -- 172.21.15.111:0/4087602694 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a000db90 con 0x7f04b40c8920 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04abfff640 1 --2- 172.21.15.111:0/4087602694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04a00069d0 0x7f04a0006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04b3fff640 1 --2- 172.21.15.111:0/4087602694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04a0005b80 0x7f04a0005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04b8d4b640 1 --2- 172.21.15.111:0/4087602694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04b40c8920 0x7f04a0007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04abfff640 1 -- 172.21.15.111:0/4087602694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04b40c8920 msgr2=0x7f04a0007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04abfff640 1 --2- 172.21.15.111:0/4087602694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04b40c8920 0x7f04a0007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04abfff640 1 -- 172.21.15.111:0/4087602694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04a0005b80 msgr2=0x7f04a0005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04abfff640 1 --2- 172.21.15.111:0/4087602694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04a0005b80 0x7f04a0005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04abfff640 1 -- 172.21.15.111:0/4087602694 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a0005730 con 0x7f04a00069d0 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.089+0000 7f04b8d4b640 1 --2- 172.21.15.111:0/4087602694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04b40c8920 0x7f04a0007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.090+0000 7f04abfff640 1 --2- 172.21.15.111:0/4087602694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04a00069d0 0x7f04a0006e50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f049c00c290 tx=0x7f04a0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1c51171daed5c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.090+0000 7f04b37fe640 1 -- 172.21.15.111:0/4087602694 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c02e070 con 0x7f04a00069d0 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.090+0000 7f04b37fe640 1 -- 172.21.15.111:0/4087602694 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049c0058d0 con 0x7f04a00069d0 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.090+0000 7f04b37fe640 1 -- 172.21.15.111:0/4087602694 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c005c80 con 0x7f04a00069d0 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.090+0000 7f04ba7ae640 1 -- 172.21.15.111:0/4087602694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04a00069d0 msgr2=0x7f04a0006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.090+0000 7f04ba7ae640 1 --2- 172.21.15.111:0/4087602694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04a00069d0 0x7f04a0006e50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f049c00c290 tx=0x7f04a0007a00 comp rx=0 tx=0).stop 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.090+0000 7f04b8d4b640 1 -- 172.21.15.111:0/4087602694 reap_dead start 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.091+0000 7f04ba7ae640 1 -- 172.21.15.111:0/4087602694 shutdown_connections 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.091+0000 7f04ba7ae640 1 -- 172.21.15.111:0/4087602694 >> 172.21.15.111:0/4087602694 conn(0x7f04b4069ed0 msgr2=0x7f04b406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.091+0000 7f04ba7ae640 1 -- 172.21.15.111:0/4087602694 shutdown_connections 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.091+0000 7f04ba7ae640 1 -- 172.21.15.111:0/4087602694 wait complete. 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.092+0000 7f04ba7ae640 1 Processor -- start 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.092+0000 7f04ba7ae640 1 -- start start 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.092+0000 7f04ba7ae640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04a00069d0 0x7f04b41eded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.092+0000 7f04ba7ae640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04b414fee0 0x7f04b415b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.092+0000 7f04ba7ae640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04b4150360 0x7f04b415b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.092+0000 7f04ba7ae640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b40c3ac0 con 0x7f04b4150360 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.092+0000 7f04ba7ae640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b415aa60 con 0x7f04b414fee0 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.092+0000 7f04ba7ae640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b41ee410 con 0x7f04a00069d0 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.093+0000 7f04b8d4b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04b4150360 0x7f04b415b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.093+0000 7f04b8d4b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04b4150360 0x7f04b415b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36538/0 (socket says 172.21.15.111:36538) 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.093+0000 7f04b8d4b640 1 -- 172.21.15.111:0/2874062940 learned_addr learned my addr 172.21.15.111:0/2874062940 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.093+0000 7f04b3fff640 1 --2- 172.21.15.111:0/2874062940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04a00069d0 0x7f04b41eded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.093+0000 7f04abfff640 1 --2- 172.21.15.111:0/2874062940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04b414fee0 0x7f04b415b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.093+0000 7f04b8d4b640 1 -- 172.21.15.111:0/2874062940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04a00069d0 msgr2=0x7f04b41eded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.093+0000 7f04b8d4b640 1 --2- 172.21.15.111:0/2874062940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04a00069d0 0x7f04b41eded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.093+0000 7f04b8d4b640 1 -- 172.21.15.111:0/2874062940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04b414fee0 msgr2=0x7f04b415b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.093+0000 7f04b8d4b640 1 --2- 172.21.15.111:0/2874062940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04b414fee0 0x7f04b415b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.093+0000 7f04b8d4b640 1 -- 172.21.15.111:0/2874062940 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b40c2eb0 con 0x7f04b4150360 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.093+0000 7f04b8d4b640 1 --2- 172.21.15.111:0/2874062940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04b4150360 0x7f04b415b790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f04b406ff50 tx=0x7f04ac010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.093+0000 7f04b17fa640 1 -- 172.21.15.111:0/2874062940 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac01b070 con 0x7f04b4150360 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.093+0000 7f04b17fa640 1 -- 172.21.15.111:0/2874062940 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04ac00f040 con 0x7f04b4150360 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.093+0000 7f04b17fa640 1 -- 172.21.15.111:0/2874062940 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac00e050 con 0x7f04b4150360 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.093+0000 7f04ba7ae640 1 -- 172.21.15.111:0/2874062940 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b40c3190 con 0x7f04b4150360 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.093+0000 7f04ba7ae640 1 -- 172.21.15.111:0/2874062940 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b40c3750 con 0x7f04b4150360 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.094+0000 7f04ba7ae640 1 -- 172.21.15.111:0/2874062940 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b40c3910 con 0x7f04b4150360 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.095+0000 7f04b17fa640 1 -- 172.21.15.111:0/2874062940 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f04ac015440 con 0x7f04b4150360 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.095+0000 7f04b17fa640 1 --2- 172.21.15.111:0/2874062940 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f049406ac00 0x7f049406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.095+0000 7f04b3fff640 1 --2- 172.21.15.111:0/2874062940 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f049406ac00 0x7f049406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.095+0000 7f04b17fa640 1 -- 172.21.15.111:0/2874062940 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f04ac089920 con 0x7f04b4150360 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.096+0000 7f04b3fff640 1 --2- 172.21.15.111:0/2874062940 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f049406ac00 0x7f049406d0c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009b40 tx=0x7f04a4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.099+0000 7f04b17fa640 1 -- 172.21.15.111:0/2874062940 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f04ac054810 con 0x7f04b4150360 2024-09-13T17:57:31.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.293+0000 7f04ba7ae640 1 -- 172.21.15.111:0/2874062940 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04b40c8dd0 con 0x7f04b4150360 2024-09-13T17:57:31.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.294+0000 7f04b17fa640 1 -- 172.21.15.111:0/2874062940 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4742 (secure 0 0 0) 0x7f04ac057e30 con 0x7f04b4150360 2024-09-13T17:57:31.297 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:31.297 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4001406804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":4001406804},{"type":"v1","addr":"172.21.15.188:6835","nonce":4001406804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2565466103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2565466103},{"type":"v1","addr":"172.21.15.111:6837","nonce":2565466103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/1252690799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1252690799},{"type":"v1","addr":"172.21.15.111:6836","nonce":1252690799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:57:25.702997+0000","modified":"2024-09-13T17:57:30.065468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/1266310843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1266310843},{"type":"v1","addr":"172.21.15.188:6834","nonce":1266310843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-09-13T17:57:31.297 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 7 2024-09-13T17:57:31.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.296+0000 7f04ba7ae640 1 -- 172.21.15.111:0/2874062940 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f049406ac00 msgr2=0x7f049406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.296+0000 7f04ba7ae640 1 --2- 172.21.15.111:0/2874062940 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f049406ac00 0x7f049406d0c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009b40 tx=0x7f04a4009190 comp rx=0 tx=0).stop 2024-09-13T17:57:31.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.296+0000 7f04ba7ae640 1 -- 172.21.15.111:0/2874062940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04b4150360 msgr2=0x7f04b415b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.296+0000 7f04ba7ae640 1 --2- 172.21.15.111:0/2874062940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04b4150360 0x7f04b415b790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f04b406ff50 tx=0x7f04ac010590 comp rx=0 tx=0).stop 2024-09-13T17:57:31.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.296+0000 7f04ba7ae640 1 -- 172.21.15.111:0/2874062940 shutdown_connections 2024-09-13T17:57:31.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.296+0000 7f04ba7ae640 1 --2- 172.21.15.111:0/2874062940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04b4150360 0x7f04b415b790 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.296+0000 7f04ba7ae640 1 --2- 172.21.15.111:0/2874062940 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f049406ac00 0x7f049406d0c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.296+0000 7f04ba7ae640 1 --2- 172.21.15.111:0/2874062940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04b414fee0 0x7f04b415b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.296+0000 7f04ba7ae640 1 --2- 172.21.15.111:0/2874062940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04a00069d0 0x7f04b41eded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.296+0000 7f04ba7ae640 1 -- 172.21.15.111:0/2874062940 >> 172.21.15.111:0/2874062940 conn(0x7f04b4069ed0 msgr2=0x7f04b40cb570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:31.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.296+0000 7f04ba7ae640 1 -- 172.21.15.111:0/2874062940 shutdown_connections 2024-09-13T17:57:31.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.296+0000 7f04ba7ae640 1 -- 172.21.15.111:0/2874062940 wait complete. 2024-09-13T17:57:31.308 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:31.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.404+0000 7ff01859c640 1 Processor -- start 2024-09-13T17:57:31.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.404+0000 7ff01859c640 1 -- start start 2024-09-13T17:57:31.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.405+0000 7ff01859c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff010072110 0x7ff010070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.405+0000 7ff01859c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff0100c8920 0x7ff0100c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.405+0000 7ff01859c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0100abdf0 con 0x7ff010071d30 2024-09-13T17:57:31.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.405+0000 7ff01859c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0100c92f0 con 0x7ff010072110 2024-09-13T17:57:31.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.405+0000 7ff01859c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0100b2280 con 0x7ff0100c8920 2024-09-13T17:57:31.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.405+0000 7ff015b37640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff010072110 0x7ff010070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.405+0000 7ff015b37640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff010072110 0x7ff010070190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36544/0 (socket says 172.21.15.111:36544) 2024-09-13T17:57:31.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.405+0000 7ff015b37640 1 -- 172.21.15.111:0/1422902438 learned_addr learned my addr 172.21.15.111:0/1422902438 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:31.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.405+0000 7ff016b39640 1 --2- 172.21.15.111:0/1422902438 >> v2:172.21.15.188:3300/0 conn(0x7ff0100c8920 0x7ff0100c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.405+0000 7ff015b37640 1 -- 172.21.15.111:0/1422902438 >> v2:172.21.15.188:3300/0 conn(0x7ff0100c8920 msgr2=0x7ff0100c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:31.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.405+0000 7ff015b37640 1 --2- 172.21.15.111:0/1422902438 >> v2:172.21.15.188:3300/0 conn(0x7ff0100c8920 0x7ff0100c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.405+0000 7ff015b37640 1 -- 172.21.15.111:0/1422902438 >> v1:172.21.15.188:6789/0 conn(0x7ff010071d30 legacy=0x7ff0100b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:31.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.405+0000 7ff015b37640 1 -- 172.21.15.111:0/1422902438 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0100ab1e0 con 0x7ff010072110 2024-09-13T17:57:31.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.406+0000 7ff015b37640 1 --2- 172.21.15.111:0/1422902438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff010072110 0x7ff010070190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff01006ff50 tx=0x7ff00402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2c2cf9d3c94ede server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:31.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.406+0000 7ff015336640 1 -- 172.21.15.111:0/1422902438 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004031600 con 0x7ff010072110 2024-09-13T17:57:31.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.406+0000 7ff015336640 1 -- 172.21.15.111:0/1422902438 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00402d050 con 0x7ff010072110 2024-09-13T17:57:31.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.406+0000 7ff015336640 1 -- 172.21.15.111:0/1422902438 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004031980 con 0x7ff010072110 2024-09-13T17:57:31.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.406+0000 7ff01859c640 1 -- 172.21.15.111:0/1422902438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff010072110 msgr2=0x7ff010070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.406+0000 7ff01859c640 1 --2- 172.21.15.111:0/1422902438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff010072110 0x7ff010070190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff01006ff50 tx=0x7ff00402a6f0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.407+0000 7ff01859c640 1 -- 172.21.15.111:0/1422902438 shutdown_connections 2024-09-13T17:57:31.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.407+0000 7ff01859c640 1 --2- 172.21.15.111:0/1422902438 >> v2:172.21.15.188:3300/0 conn(0x7ff0100c8920 0x7ff0100c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.407+0000 7ff01859c640 1 --2- 172.21.15.111:0/1422902438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff010072110 0x7ff010070190 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.407+0000 7ff01859c640 1 -- 172.21.15.111:0/1422902438 >> 172.21.15.111:0/1422902438 conn(0x7ff010069ed0 msgr2=0x7ff01006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:31.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.407+0000 7ff01859c640 1 -- 172.21.15.111:0/1422902438 shutdown_connections 2024-09-13T17:57:31.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.407+0000 7ff01859c640 1 -- 172.21.15.111:0/1422902438 wait complete. 2024-09-13T17:57:31.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.408+0000 7ff01859c640 1 Processor -- start 2024-09-13T17:57:31.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.408+0000 7ff01859c640 1 -- start start 2024-09-13T17:57:31.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.408+0000 7ff01859c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff010071d30 0x7ff0101ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.408+0000 7ff01859c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff010072110 0x7ff01015bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.408+0000 7ff01859c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0100c8920 0x7ff01015c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.408+0000 7ff01859c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01014de30 con 0x7ff0100c8920 2024-09-13T17:57:31.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.408+0000 7ff01859c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01014cd90 con 0x7ff010071d30 2024-09-13T17:57:31.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.408+0000 7ff01859c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01014d070 con 0x7ff010072110 2024-09-13T17:57:31.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.408+0000 7ff016b39640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0100c8920 0x7ff01015c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.409+0000 7ff016b39640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0100c8920 0x7ff01015c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36548/0 (socket says 172.21.15.111:36548) 2024-09-13T17:57:31.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.409+0000 7ff016b39640 1 -- 172.21.15.111:0/3448595987 learned_addr learned my addr 172.21.15.111:0/3448595987 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:31.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.409+0000 7ff016338640 1 --2- 172.21.15.111:0/3448595987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff010071d30 0x7ff0101ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.409+0000 7ff015b37640 1 --2- 172.21.15.111:0/3448595987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff010072110 0x7ff01015bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.409+0000 7ff016b39640 1 -- 172.21.15.111:0/3448595987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff010072110 msgr2=0x7ff01015bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.409+0000 7ff016b39640 1 --2- 172.21.15.111:0/3448595987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff010072110 0x7ff01015bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.409+0000 7ff016b39640 1 -- 172.21.15.111:0/3448595987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff010071d30 msgr2=0x7ff0101ef8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.409+0000 7ff016b39640 1 --2- 172.21.15.111:0/3448595987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff010071d30 0x7ff0101ef8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.409+0000 7ff016b39640 1 -- 172.21.15.111:0/3448595987 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff004009b30 con 0x7ff0100c8920 2024-09-13T17:57:31.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.409+0000 7ff016b39640 1 --2- 172.21.15.111:0/3448595987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0100c8920 0x7ff01015c200 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff00800d5d0 tx=0x7ff00800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:31.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.409+0000 7ff00effd640 1 -- 172.21.15.111:0/3448595987 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0080125f0 con 0x7ff0100c8920 2024-09-13T17:57:31.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.409+0000 7ff00effd640 1 -- 172.21.15.111:0/3448595987 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff008012770 con 0x7ff0100c8920 2024-09-13T17:57:31.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.409+0000 7ff00effd640 1 -- 172.21.15.111:0/3448595987 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff008012b20 con 0x7ff0100c8920 2024-09-13T17:57:31.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.409+0000 7ff01859c640 1 -- 172.21.15.111:0/3448595987 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff01014d350 con 0x7ff0100c8920 2024-09-13T17:57:31.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.409+0000 7ff01859c640 1 -- 172.21.15.111:0/3448595987 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01014d910 con 0x7ff0100c8920 2024-09-13T17:57:31.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.410+0000 7ff01859c640 1 -- 172.21.15.111:0/3448595987 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff010070190 con 0x7ff0100c8920 2024-09-13T17:57:31.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.411+0000 7ff00effd640 1 -- 172.21.15.111:0/3448595987 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff008012c80 con 0x7ff0100c8920 2024-09-13T17:57:31.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.411+0000 7ff00effd640 1 --2- 172.21.15.111:0/3448595987 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7feffc06ac30 0x7feffc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.411+0000 7ff016338640 1 --2- 172.21.15.111:0/3448595987 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7feffc06ac30 0x7feffc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.411+0000 7ff00effd640 1 -- 172.21.15.111:0/3448595987 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff008089980 con 0x7ff0100c8920 2024-09-13T17:57:31.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.412+0000 7ff016338640 1 --2- 172.21.15.111:0/3448595987 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7feffc06ac30 0x7feffc06d0f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff000004f60 tx=0x7ff000007e30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:31.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.415+0000 7ff00effd640 1 -- 172.21.15.111:0/3448595987 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff0080548a0 con 0x7ff0100c8920 2024-09-13T17:57:31.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.576+0000 7ff01859c640 1 -- 172.21.15.111:0/3448595987 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff01015cb20 con 0x7ff0100c8920 2024-09-13T17:57:31.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.577+0000 7ff00effd640 1 -- 172.21.15.111:0/3448595987 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7ff008057ec0 con 0x7ff0100c8920 2024-09-13T17:57:31.579 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:31.579 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:30.065235+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:31.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.579+0000 7ff01859c640 1 -- 172.21.15.111:0/3448595987 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7feffc06ac30 msgr2=0x7feffc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.579+0000 7ff01859c640 1 --2- 172.21.15.111:0/3448595987 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7feffc06ac30 0x7feffc06d0f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff000004f60 tx=0x7ff000007e30 comp rx=0 tx=0).stop 2024-09-13T17:57:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.579+0000 7ff01859c640 1 -- 172.21.15.111:0/3448595987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0100c8920 msgr2=0x7ff01015c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.579+0000 7ff01859c640 1 --2- 172.21.15.111:0/3448595987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0100c8920 0x7ff01015c200 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff00800d5d0 tx=0x7ff00800da90 comp rx=0 tx=0).stop 2024-09-13T17:57:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.579+0000 7ff01859c640 1 -- 172.21.15.111:0/3448595987 shutdown_connections 2024-09-13T17:57:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.579+0000 7ff01859c640 1 --2- 172.21.15.111:0/3448595987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0100c8920 0x7ff01015c200 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.579+0000 7ff01859c640 1 --2- 172.21.15.111:0/3448595987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff010072110 0x7ff01015bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.579+0000 7ff01859c640 1 --2- 172.21.15.111:0/3448595987 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7feffc06ac30 0x7feffc06d0f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.579+0000 7ff01859c640 1 --2- 172.21.15.111:0/3448595987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff010071d30 0x7ff0101ef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.579+0000 7ff01859c640 1 -- 172.21.15.111:0/3448595987 >> 172.21.15.111:0/3448595987 conn(0x7ff010069ed0 msgr2=0x7ff0100b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.579+0000 7ff01859c640 1 -- 172.21.15.111:0/3448595987 shutdown_connections 2024-09-13T17:57:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:31.579+0000 7ff01859c640 1 -- 172.21.15.111:0/3448595987 wait complete. 2024-09-13T17:57:31.995 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:57:25.702997+0000', 'modified': '2024-09-13T17:57:30.065468+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4291': {'gid': 4291, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6834/1266310843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 1266310843}, {'type': 'v1', 'addr': '172.21.15.188:6834', 'nonce': 1266310843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-09-13T17:57:31.995 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T17:57:31.995 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-13T17:57:32.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.092+0000 7f53dbf83640 1 Processor -- start 2024-09-13T17:57:32.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.093+0000 7f53dbf83640 1 -- start start 2024-09-13T17:57:32.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.093+0000 7f53dbf83640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f53d40cace0 0x7f53d40cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.093+0000 7f53dbf83640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f53d406d030 0x7f53d406d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.093+0000 7f53dbf83640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d40c8920 con 0x7f53d415a2e0 2024-09-13T17:57:32.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.093+0000 7f53dbf83640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d40cb690 con 0x7f53d40cace0 2024-09-13T17:57:32.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.093+0000 7f53dbf83640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d415a6c0 con 0x7f53d406d030 2024-09-13T17:57:32.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.093+0000 7f53d9d1f640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f53d415a2e0 0x7f53d40c9a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58102/0 (socket says 172.21.15.111:58102) 2024-09-13T17:57:32.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.093+0000 7f53d951e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f53d40cace0 0x7f53d40cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.093+0000 7f53d9d1f640 1 -- 172.21.15.111:0/3379493817 learned_addr learned my addr 172.21.15.111:0/3379493817 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:32.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.094+0000 7f53da520640 1 --2- 172.21.15.111:0/3379493817 >> v2:172.21.15.188:3300/0 conn(0x7f53d406d030 0x7f53d406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.094+0000 7f53d951e640 1 -- 172.21.15.111:0/3379493817 >> v2:172.21.15.188:3300/0 conn(0x7f53d406d030 msgr2=0x7f53d406d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:32.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.094+0000 7f53d951e640 1 --2- 172.21.15.111:0/3379493817 >> v2:172.21.15.188:3300/0 conn(0x7f53d406d030 0x7f53d406d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.094+0000 7f53d951e640 1 -- 172.21.15.111:0/3379493817 >> v1:172.21.15.111:6789/0 conn(0x7f53d415a2e0 legacy=0x7f53d40c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:32.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.094+0000 7f53d951e640 1 -- 172.21.15.111:0/3379493817 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d40ca500 con 0x7f53d40cace0 2024-09-13T17:57:32.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.094+0000 7f53d951e640 1 --2- 172.21.15.111:0/3379493817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53d40cace0 0x7f53d40cb150 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f53d406c5b0 tx=0x7f53c402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c7065085c4c2bf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:32.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.095+0000 7f53d8d1d640 1 -- 172.21.15.111:0/3379493817 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4032440 con 0x7f53d40cace0 2024-09-13T17:57:32.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.095+0000 7f53d8d1d640 1 -- 172.21.15.111:0/3379493817 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c40325a0 con 0x7f53d40cace0 2024-09-13T17:57:32.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.095+0000 7f53d8d1d640 1 -- 172.21.15.111:0/3379493817 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4032930 con 0x7f53d40cace0 2024-09-13T17:57:32.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.095+0000 7f53dbf83640 1 -- 172.21.15.111:0/3379493817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53d40cace0 msgr2=0x7f53d40cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:32.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.095+0000 7f53dbf83640 1 --2- 172.21.15.111:0/3379493817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53d40cace0 0x7f53d40cb150 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f53d406c5b0 tx=0x7f53c402a1a0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.095+0000 7f53dbf83640 1 -- 172.21.15.111:0/3379493817 shutdown_connections 2024-09-13T17:57:32.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.095+0000 7f53dbf83640 1 --2- 172.21.15.111:0/3379493817 >> v2:172.21.15.188:3300/0 conn(0x7f53d406d030 0x7f53d406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.095+0000 7f53dbf83640 1 --2- 172.21.15.111:0/3379493817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53d40cace0 0x7f53d40cb150 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.095+0000 7f53dbf83640 1 -- 172.21.15.111:0/3379493817 >> 172.21.15.111:0/3379493817 conn(0x7f53d4069ed0 msgr2=0x7f53d406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:32.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.095+0000 7f53dbf83640 1 -- 172.21.15.111:0/3379493817 shutdown_connections 2024-09-13T17:57:32.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.095+0000 7f53dbf83640 1 -- 172.21.15.111:0/3379493817 wait complete. 2024-09-13T17:57:32.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.096+0000 7f53dbf83640 1 Processor -- start 2024-09-13T17:57:32.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.096+0000 7f53dbf83640 1 -- start start 2024-09-13T17:57:32.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.097+0000 7f53dbf83640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d406d030 0x7f53d41f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.097+0000 7f53dbf83640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53d40cace0 0x7f53d4158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.097+0000 7f53dbf83640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53d415a2e0 0x7f53d4158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.097+0000 7f53dbf83640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d41f1a70 con 0x7f53d40cace0 2024-09-13T17:57:32.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.097+0000 7f53dbf83640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d40c1100 con 0x7f53d406d030 2024-09-13T17:57:32.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.097+0000 7f53dbf83640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d41f1c60 con 0x7f53d415a2e0 2024-09-13T17:57:32.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.097+0000 7f53d951e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53d40cace0 0x7f53d4158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.097+0000 7f53d951e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53d40cace0 0x7f53d4158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36564/0 (socket says 172.21.15.111:36564) 2024-09-13T17:57:32.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.097+0000 7f53d951e640 1 -- 172.21.15.111:0/2493594468 learned_addr learned my addr 172.21.15.111:0/2493594468 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:32.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.097+0000 7f53d9d1f640 1 --2- 172.21.15.111:0/2493594468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d406d030 0x7f53d41f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.097+0000 7f53da520640 1 --2- 172.21.15.111:0/2493594468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53d415a2e0 0x7f53d4158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.097+0000 7f53d951e640 1 -- 172.21.15.111:0/2493594468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53d415a2e0 msgr2=0x7f53d4158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:32.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.097+0000 7f53d951e640 1 --2- 172.21.15.111:0/2493594468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53d415a2e0 0x7f53d4158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.097+0000 7f53d951e640 1 -- 172.21.15.111:0/2493594468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d406d030 msgr2=0x7f53d41f2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:32.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.097+0000 7f53d951e640 1 --2- 172.21.15.111:0/2493594468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d406d030 0x7f53d41f2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.097+0000 7f53d951e640 1 -- 172.21.15.111:0/2493594468 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c4009400 con 0x7f53d40cace0 2024-09-13T17:57:32.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.098+0000 7f53d9d1f640 1 --2- 172.21.15.111:0/2493594468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d406d030 0x7f53d41f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:32.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.098+0000 7f53d951e640 1 --2- 172.21.15.111:0/2493594468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53d40cace0 0x7f53d4158930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f53c4000c00 tx=0x7f53c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:32.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.098+0000 7f53ca7fc640 1 -- 172.21.15.111:0/2493594468 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4030b80 con 0x7f53d40cace0 2024-09-13T17:57:32.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.098+0000 7f53ca7fc640 1 -- 172.21.15.111:0/2493594468 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c4030ce0 con 0x7f53d40cace0 2024-09-13T17:57:32.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.098+0000 7f53ca7fc640 1 -- 172.21.15.111:0/2493594468 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c403c640 con 0x7f53d40cace0 2024-09-13T17:57:32.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.098+0000 7f53dbf83640 1 -- 172.21.15.111:0/2493594468 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d41f1f40 con 0x7f53d40cace0 2024-09-13T17:57:32.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.098+0000 7f53dbf83640 1 -- 172.21.15.111:0/2493594468 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d4156ea0 con 0x7f53d40cace0 2024-09-13T17:57:32.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.100+0000 7f53ca7fc640 1 -- 172.21.15.111:0/2493594468 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f53c403c7a0 con 0x7f53d40cace0 2024-09-13T17:57:32.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.100+0000 7f53ca7fc640 1 --2- 172.21.15.111:0/2493594468 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f53b806ac00 0x7f53b806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.100+0000 7f53ca7fc640 1 -- 172.21.15.111:0/2493594468 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f53c40aa530 con 0x7f53d40cace0 2024-09-13T17:57:32.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.100+0000 7f53d9d1f640 1 --2- 172.21.15.111:0/2493594468 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f53b806ac00 0x7f53b806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.100+0000 7f53d9d1f640 1 --2- 172.21.15.111:0/2493594468 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f53b806ac00 0x7f53b806d0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f53d4157270 tx=0x7f53d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:32.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.101+0000 7f53decab740 1 Processor -- start 2024-09-13T17:57:32.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.101+0000 7f53decab740 1 -- start start 2024-09-13T17:57:32.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53decab740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec140a0 0x55cb5ec14480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53decab740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec11860 0x55cb5ec61570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53decab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec3b550 0x55cb5ec61ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53decab740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb5eb6e7c0 con 0x55cb5ec11860 2024-09-13T17:57:32.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53decab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb5eb7db70 con 0x55cb5ec3b550 2024-09-13T17:57:32.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53decab740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb5eb7b300 con 0x55cb5ec140a0 2024-09-13T17:57:32.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53d951e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec11860 0x55cb5ec61570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53d951e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec11860 0x55cb5ec61570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36568/0 (socket says 172.21.15.111:36568) 2024-09-13T17:57:32.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53d951e640 1 -- 172.21.15.111:0/2974681868 learned_addr learned my addr 172.21.15.111:0/2974681868 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:32.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53da520640 1 --2- 172.21.15.111:0/2974681868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec3b550 0x55cb5ec61ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53d9d1f640 1 --2- 172.21.15.111:0/2974681868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec140a0 0x55cb5ec14480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53d951e640 1 -- 172.21.15.111:0/2974681868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec140a0 msgr2=0x55cb5ec14480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:32.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53d951e640 1 --2- 172.21.15.111:0/2974681868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec140a0 0x55cb5ec14480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53d951e640 1 -- 172.21.15.111:0/2974681868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec3b550 msgr2=0x55cb5ec61ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:32.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53d951e640 1 --2- 172.21.15.111:0/2974681868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec3b550 0x55cb5ec61ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53d951e640 1 -- 172.21.15.111:0/2974681868 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb5eb034b0 con 0x55cb5ec11860 2024-09-13T17:57:32.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53d9d1f640 1 --2- 172.21.15.111:0/2974681868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec140a0 0x55cb5ec14480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:32.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53d951e640 1 --2- 172.21.15.111:0/2974681868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec11860 0x55cb5ec61570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f53c4075620 tx=0x7f53c4006e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec137cb6b677348b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:32.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53b77fe640 1 -- 172.21.15.111:0/2974681868 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c407cfa0 con 0x55cb5ec11860 2024-09-13T17:57:32.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53b77fe640 1 -- 172.21.15.111:0/2974681868 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c4081560 con 0x55cb5ec11860 2024-09-13T17:57:32.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.102+0000 7f53b77fe640 1 -- 172.21.15.111:0/2974681868 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4075880 con 0x55cb5ec11860 2024-09-13T17:57:32.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.103+0000 7f53decab740 1 -- 172.21.15.111:0/2974681868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec11860 msgr2=0x55cb5ec61570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:32.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.103+0000 7f53decab740 1 --2- 172.21.15.111:0/2974681868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec11860 0x55cb5ec61570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f53c4075620 tx=0x7f53c4006e10 comp rx=0 tx=0).stop 2024-09-13T17:57:32.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.103+0000 7f53decab740 1 -- 172.21.15.111:0/2974681868 shutdown_connections 2024-09-13T17:57:32.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.103+0000 7f53decab740 1 --2- 172.21.15.111:0/2974681868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec3b550 0x55cb5ec61ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.103+0000 7f53decab740 1 --2- 172.21.15.111:0/2974681868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec11860 0x55cb5ec61570 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.103+0000 7f53decab740 1 --2- 172.21.15.111:0/2974681868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec140a0 0x55cb5ec14480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.103+0000 7f53decab740 1 -- 172.21.15.111:0/2974681868 >> 172.21.15.111:0/2974681868 conn(0x55cb5ec107d0 msgr2=0x55cb5ec58c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:32.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.103+0000 7f53decab740 1 -- 172.21.15.111:0/2974681868 shutdown_connections 2024-09-13T17:57:32.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.103+0000 7f53decab740 1 -- 172.21.15.111:0/2974681868 wait complete. 2024-09-13T17:57:32.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.104+0000 7f53decab740 1 -- start start 2024-09-13T17:57:32.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.104+0000 7f53decab740 1 Processor -- start 2024-09-13T17:57:32.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53decab740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec140a0 0x55cb5ec31f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53decab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec3b550 0x55cb5ec0d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53decab740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec32bf0 0x55cb5ec35c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53decab740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb5eb76bf0 con 0x55cb5ec140a0 2024-09-13T17:57:32.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53decab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb5eb35250 con 0x55cb5ec3b550 2024-09-13T17:57:32.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53decab740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb5eb34ef0 con 0x55cb5ec32bf0 2024-09-13T17:57:32.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53decab740 10 client.0 _ll_register_callbacks cb 0x55cb5eb67740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:57:32.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53d9d1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec140a0 0x55cb5ec31f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53d9d1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec140a0 0x55cb5ec31f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36572/0 (socket says 172.21.15.111:36572) 2024-09-13T17:57:32.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53d9d1f640 1 -- 172.21.15.111:0/3740508675 learned_addr learned my addr 172.21.15.111:0/3740508675 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:32.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53d9d1f640 1 -- 172.21.15.111:0/3740508675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec32bf0 msgr2=0x55cb5ec35c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:32.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53d9d1f640 1 --2- 172.21.15.111:0/3740508675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec32bf0 0x55cb5ec35c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53d951e640 1 --2- 172.21.15.111:0/3740508675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec3b550 0x55cb5ec0d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53d9d1f640 1 -- 172.21.15.111:0/3740508675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec3b550 msgr2=0x55cb5ec0d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:32.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53d9d1f640 1 --2- 172.21.15.111:0/3740508675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec3b550 0x55cb5ec0d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53d9d1f640 1 -- 172.21.15.111:0/3740508675 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d4156ea0 con 0x55cb5ec140a0 2024-09-13T17:57:32.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53d9d1f640 1 --2- 172.21.15.111:0/3740508675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec140a0 0x55cb5ec31f90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f53d00071d0 tx=0x7f53d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c397fa204ef0065e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:32.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-13T17:57:32.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53b4ff9640 1 -- 172.21.15.111:0/3740508675 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d0007750 con 0x55cb5ec140a0 2024-09-13T17:57:32.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53b4ff9640 1 -- 172.21.15.111:0/3740508675 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f53d0022d80 con 0x55cb5ec140a0 2024-09-13T17:57:32.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53decab740 1 -- 172.21.15.111:0/3740508675 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb5e86a3c0 con 0x55cb5ec140a0 2024-09-13T17:57:32.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53b4ff9640 1 -- 172.21.15.111:0/3740508675 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53d000ec80 con 0x55cb5ec140a0 2024-09-13T17:57:32.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53decab740 10 client.4407 fetch_fsmap learned FSMap version 7 2024-09-13T17:57:32.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.105+0000 7f53decab740 1 -- 172.21.15.111:0/3740508675 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55cb5eb75270 con 0x55cb5ec140a0 2024-09-13T17:57:32.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.106+0000 7f53b4ff9640 1 -- 172.21.15.111:0/3740508675 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb5e86a3c0 con 0x55cb5ec140a0 2024-09-13T17:57:32.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.106+0000 7f53b4ff9640 1 -- 172.21.15.111:0/3740508675 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f53d002a070 con 0x55cb5ec140a0 2024-09-13T17:57:32.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.106+0000 7f53decab740 10 client.4407 fetch_fsmap finished waiting for FSMap version 7 2024-09-13T17:57:32.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.106+0000 7f53decab740 10 client.4407 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T17:57:32.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.106+0000 7f53decab740 20 client.4407 populate_metadata read hostname 'smithi111' 2024-09-13T17:57:32.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.106+0000 7f53decab740 1 --2- 172.21.15.111:0/3740508675 >> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] conn(0x55cb5ec11d70 0x55cb5ec12150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.106+0000 7f53decab740 4 client.4407 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:57:32.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.106+0000 7f53decab740 1 -- 172.21.15.111:0/3740508675 --> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb5e87b6c0 con 0x55cb5ec11d70 2024-09-13T17:57:32.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.106+0000 7f53d951e640 1 --2- 172.21.15.111:0/3740508675 >> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] conn(0x55cb5ec11d70 0x55cb5ec12150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.107+0000 7f53d951e640 1 --2- 172.21.15.111:0/3740508675 >> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] conn(0x55cb5ec11d70 0x55cb5ec12150 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51186b8e701a5c79 server_cookie=afbd7305bea1d18b in_seq=0 out_seq=0 2024-09-13T17:57:32.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.107+0000 7f53b4ff9640 10 client.4407 ms_handle_connect on v2:172.21.15.188:6832/1266310843 2024-09-13T17:57:32.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.107+0000 7f53b4ff9640 1 -- 172.21.15.111:0/3740508675 <== mds.0 v2:172.21.15.188:6832/1266310843 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55cb5e87b6c0 con 0x55cb5ec11d70 2024-09-13T17:57:32.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.107+0000 7f53b4ff9640 10 client.4407 handle_command_reply: tid=0 2024-09-13T17:57:32.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.107+0000 7f53decab740 1 client.4407 shutdown 2024-09-13T17:57:32.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.107+0000 7f53decab740 1 -- 172.21.15.111:0/3740508675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec140a0 msgr2=0x55cb5ec31f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:32.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.107+0000 7f53decab740 1 --2- 172.21.15.111:0/3740508675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec140a0 0x55cb5ec31f90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f53d00071d0 tx=0x7f53d0000bc0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.107+0000 7f53decab740 1 -- 172.21.15.111:0/3740508675 shutdown_connections 2024-09-13T17:57:32.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.107+0000 7f53decab740 1 --2- 172.21.15.111:0/3740508675 >> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] conn(0x55cb5ec11d70 0x55cb5ec12150 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.107+0000 7f53decab740 1 --2- 172.21.15.111:0/3740508675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec32bf0 0x55cb5ec35c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.107+0000 7f53b4ff9640 0 client.4407 ms_handle_reset on v2:172.21.15.188:6832/1266310843 2024-09-13T17:57:32.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.107+0000 7f53decab740 1 --2- 172.21.15.111:0/3740508675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec3b550 0x55cb5ec0d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.107+0000 7f53decab740 1 --2- 172.21.15.111:0/3740508675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec140a0 0x55cb5ec31f90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.107+0000 7f53decab740 1 -- 172.21.15.111:0/3740508675 >> 172.21.15.111:0/3740508675 conn(0x55cb5ec11860 msgr2=0x55cb5ec56fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:32.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.108+0000 7f53decab740 1 -- 172.21.15.111:0/3740508675 shutdown_connections 2024-09-13T17:57:32.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.108+0000 7f53decab740 1 -- 172.21.15.111:0/3740508675 wait complete. 2024-09-13T17:57:32.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.108+0000 7f53decab740 20 client.4407 trim_cache size 0 max 16384 2024-09-13T17:57:32.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.122+0000 7f53decab740 1 Processor -- start 2024-09-13T17:57:32.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.122+0000 7f53decab740 1 -- start start 2024-09-13T17:57:32.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53decab740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec140a0 0x55cb5ec35660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53decab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec32bf0 0x55cb5ec35ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53decab740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec3b550 0x55cb5ec379d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53decab740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb5eafa460 con 0x55cb5ec3b550 2024-09-13T17:57:32.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53decab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb5ec322a0 con 0x55cb5ec32bf0 2024-09-13T17:57:32.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53decab740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb5ec5aee0 con 0x55cb5ec140a0 2024-09-13T17:57:32.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53da520640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec3b550 0x55cb5ec379d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53da520640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec3b550 0x55cb5ec379d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36574/0 (socket says 172.21.15.111:36574) 2024-09-13T17:57:32.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53da520640 1 -- 172.21.15.111:0/2604064433 learned_addr learned my addr 172.21.15.111:0/2604064433 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:32.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53d951e640 1 --2- 172.21.15.111:0/2604064433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec32bf0 0x55cb5ec35ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53da520640 1 -- 172.21.15.111:0/2604064433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec140a0 msgr2=0x55cb5ec35660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:32.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53da520640 1 --2- 172.21.15.111:0/2604064433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec140a0 0x55cb5ec35660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53da520640 1 -- 172.21.15.111:0/2604064433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec32bf0 msgr2=0x55cb5ec35ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:32.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53da520640 1 --2- 172.21.15.111:0/2604064433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec32bf0 0x55cb5ec35ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53da520640 1 -- 172.21.15.111:0/2604064433 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb5eb175f0 con 0x55cb5ec3b550 2024-09-13T17:57:32.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53d951e640 1 --2- 172.21.15.111:0/2604064433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec32bf0 0x55cb5ec35ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:32.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53da520640 1 --2- 172.21.15.111:0/2604064433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec3b550 0x55cb5ec379d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00f560 tx=0x7f53cc00fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b27500db756149e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:32.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.123+0000 7f53b7fff640 1 -- 172.21.15.111:0/2604064433 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc014070 con 0x55cb5ec3b550 2024-09-13T17:57:32.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.124+0000 7f53b7fff640 1 -- 172.21.15.111:0/2604064433 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53cc00e780 con 0x55cb5ec3b550 2024-09-13T17:57:32.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.124+0000 7f53b7fff640 1 -- 172.21.15.111:0/2604064433 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc00eb30 con 0x55cb5ec3b550 2024-09-13T17:57:32.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.124+0000 7f53decab740 1 -- 172.21.15.111:0/2604064433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec3b550 msgr2=0x55cb5ec379d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:32.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.124+0000 7f53decab740 1 --2- 172.21.15.111:0/2604064433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec3b550 0x55cb5ec379d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00f560 tx=0x7f53cc00fa20 comp rx=0 tx=0).stop 2024-09-13T17:57:32.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.124+0000 7f53decab740 1 -- 172.21.15.111:0/2604064433 shutdown_connections 2024-09-13T17:57:32.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.124+0000 7f53decab740 1 --2- 172.21.15.111:0/2604064433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec3b550 0x55cb5ec379d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.124+0000 7f53decab740 1 --2- 172.21.15.111:0/2604064433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec32bf0 0x55cb5ec35ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.124+0000 7f53decab740 1 --2- 172.21.15.111:0/2604064433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec140a0 0x55cb5ec35660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.124+0000 7f53decab740 1 -- 172.21.15.111:0/2604064433 >> 172.21.15.111:0/2604064433 conn(0x55cb5ec11860 msgr2=0x55cb5ec31ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:32.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.124+0000 7f53decab740 1 -- 172.21.15.111:0/2604064433 shutdown_connections 2024-09-13T17:57:32.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.124+0000 7f53decab740 1 -- 172.21.15.111:0/2604064433 wait complete. 2024-09-13T17:57:32.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.125+0000 7f53decab740 1 -- start start 2024-09-13T17:57:32.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.125+0000 7f53decab740 1 Processor -- start 2024-09-13T17:57:32.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.125+0000 7f53decab740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec140a0 0x55cb5ecbce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.125+0000 7f53decab740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec32bf0 0x55cb5ec0dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.125+0000 7f53decab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec3b550 0x55cb5ecc34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.125+0000 7f53decab740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb5ecc2d80 con 0x55cb5ec140a0 2024-09-13T17:57:32.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.125+0000 7f53decab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb5ec0e600 con 0x55cb5ec3b550 2024-09-13T17:57:32.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53decab740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb5ec33df0 con 0x55cb5ec32bf0 2024-09-13T17:57:32.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53decab740 10 client.0 _ll_register_callbacks cb 0x7f53c40754b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:57:32.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53d9d1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec140a0 0x55cb5ecbce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53d9d1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec140a0 0x55cb5ecbce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36578/0 (socket says 172.21.15.111:36578) 2024-09-13T17:57:32.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53d9d1f640 1 -- 172.21.15.111:0/3536423566 learned_addr learned my addr 172.21.15.111:0/3536423566 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:32.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53d951e640 1 --2- 172.21.15.111:0/3536423566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec32bf0 0x55cb5ec0dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53da520640 1 --2- 172.21.15.111:0/3536423566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec3b550 0x55cb5ecc34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53d9d1f640 1 -- 172.21.15.111:0/3536423566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec32bf0 msgr2=0x55cb5ec0dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:32.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53d9d1f640 1 --2- 172.21.15.111:0/3536423566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec32bf0 0x55cb5ec0dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53d9d1f640 1 -- 172.21.15.111:0/3536423566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec3b550 msgr2=0x55cb5ecc34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:32.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53d9d1f640 1 --2- 172.21.15.111:0/3536423566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec3b550 0x55cb5ecc34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53d9d1f640 1 -- 172.21.15.111:0/3536423566 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53cc007da0 con 0x55cb5ec140a0 2024-09-13T17:57:32.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53d9d1f640 1 --2- 172.21.15.111:0/3536423566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec140a0 0x55cb5ecbce20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f53d00226e0 tx=0x7f53d0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d70a70e080dca14 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:32.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-13T17:57:32.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53b4ff9640 1 -- 172.21.15.111:0/3536423566 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d000ec80 con 0x55cb5ec140a0 2024-09-13T17:57:32.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53b4ff9640 1 -- 172.21.15.111:0/3536423566 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f53d000e1b0 con 0x55cb5ec140a0 2024-09-13T17:57:32.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53decab740 1 -- 172.21.15.111:0/3536423566 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53c4081560 con 0x55cb5ec140a0 2024-09-13T17:57:32.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53b4ff9640 1 -- 172.21.15.111:0/3536423566 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53d0007750 con 0x55cb5ec140a0 2024-09-13T17:57:32.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53decab740 10 client.4413 fetch_fsmap learned FSMap version 7 2024-09-13T17:57:32.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53decab740 1 -- 172.21.15.111:0/3536423566 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55cb5eb3cd00 con 0x55cb5ec140a0 2024-09-13T17:57:32.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53b4ff9640 1 -- 172.21.15.111:0/3536423566 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c4081560 con 0x55cb5ec140a0 2024-09-13T17:57:32.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.126+0000 7f53b4ff9640 1 -- 172.21.15.111:0/3536423566 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f53d002a040 con 0x55cb5ec140a0 2024-09-13T17:57:32.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53decab740 10 client.4413 fetch_fsmap finished waiting for FSMap version 7 2024-09-13T17:57:32.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53decab740 10 client.4413 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T17:57:32.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53decab740 20 client.4413 populate_metadata read hostname 'smithi111' 2024-09-13T17:57:32.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53decab740 1 --2- 172.21.15.111:0/3536423566 >> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] conn(0x55cb5ec35660 0x55cb5ec35a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53decab740 4 client.4413 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:57:32.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53decab740 1 -- 172.21.15.111:0/3536423566 --> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cb5e8ced10 con 0x55cb5ec35660 2024-09-13T17:57:32.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53d951e640 1 --2- 172.21.15.111:0/3536423566 >> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] conn(0x55cb5ec35660 0x55cb5ec35a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53d951e640 1 --2- 172.21.15.111:0/3536423566 >> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] conn(0x55cb5ec35660 0x55cb5ec35a40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd1c52cc2891223 server_cookie=13e29ce935cd2e0c in_seq=0 out_seq=0 2024-09-13T17:57:32.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53b4ff9640 10 client.4413 ms_handle_connect on v2:172.21.15.188:6832/1266310843 2024-09-13T17:57:32.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53b4ff9640 1 -- 172.21.15.111:0/3536423566 <== mds.0 v2:172.21.15.188:6832/1266310843 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55cb5e8ced10 con 0x55cb5ec35660 2024-09-13T17:57:32.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53b4ff9640 10 client.4413 handle_command_reply: tid=0 2024-09-13T17:57:32.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53decab740 1 client.4413 shutdown 2024-09-13T17:57:32.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53decab740 1 -- 172.21.15.111:0/3536423566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec140a0 msgr2=0x55cb5ecbce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:32.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53decab740 1 --2- 172.21.15.111:0/3536423566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec140a0 0x55cb5ecbce20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f53d00226e0 tx=0x7f53d0000ed0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53decab740 1 -- 172.21.15.111:0/3536423566 shutdown_connections 2024-09-13T17:57:32.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53decab740 1 --2- 172.21.15.111:0/3536423566 >> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] conn(0x55cb5ec35660 0x55cb5ec35a40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53decab740 1 --2- 172.21.15.111:0/3536423566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb5ec3b550 0x55cb5ecc34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53b4ff9640 0 client.4413 ms_handle_reset on v2:172.21.15.188:6832/1266310843 2024-09-13T17:57:32.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53decab740 1 --2- 172.21.15.111:0/3536423566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb5ec32bf0 0x55cb5ec0dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53decab740 1 --2- 172.21.15.111:0/3536423566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb5ec140a0 0x55cb5ecbce20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.127+0000 7f53decab740 1 -- 172.21.15.111:0/3536423566 >> 172.21.15.111:0/3536423566 conn(0x55cb5ec11860 msgr2=0x55cb5ec0f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:32.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.128+0000 7f53decab740 1 -- 172.21.15.111:0/3536423566 shutdown_connections 2024-09-13T17:57:32.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.128+0000 7f53decab740 1 -- 172.21.15.111:0/3536423566 wait complete. 2024-09-13T17:57:32.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.128+0000 7f53decab740 20 client.4413 trim_cache size 0 max 16384 2024-09-13T17:57:32.141 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T17:57:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "2a8c13ef-d427-4b48-90b3-65459ec79a47", 2024-09-13T17:57:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2024-09-13T17:57:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4291, 2024-09-13T17:57:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-09-13T17:57:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-09-13T17:57:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-09-13T17:57:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 6.4231688250000003, 2024-09-13T17:57:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 7, 2024-09-13T17:57:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 24, 2024-09-13T17:57:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 23, 2024-09-13T17:57:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 12.317854561000001 2024-09-13T17:57:32.142 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T17:57:32.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.128+0000 7f53dbf83640 1 -- 172.21.15.111:0/2493594468 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f53b806ac00 msgr2=0x7f53b806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:32.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.128+0000 7f53dbf83640 1 --2- 172.21.15.111:0/2493594468 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f53b806ac00 0x7f53b806d0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f53d4157270 tx=0x7f53d0009040 comp rx=0 tx=0).stop 2024-09-13T17:57:32.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.128+0000 7f53dbf83640 1 -- 172.21.15.111:0/2493594468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53d40cace0 msgr2=0x7f53d4158930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:32.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.128+0000 7f53dbf83640 1 --2- 172.21.15.111:0/2493594468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53d40cace0 0x7f53d4158930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f53c4000c00 tx=0x7f53c40046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.128+0000 7f53dbf83640 1 -- 172.21.15.111:0/2493594468 shutdown_connections 2024-09-13T17:57:32.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.128+0000 7f53dbf83640 1 --2- 172.21.15.111:0/2493594468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53d415a2e0 0x7f53d4158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.128+0000 7f53dbf83640 1 --2- 172.21.15.111:0/2493594468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53d40cace0 0x7f53d4158930 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.128+0000 7f53dbf83640 1 --2- 172.21.15.111:0/2493594468 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f53b806ac00 0x7f53b806d0c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.128+0000 7f53dbf83640 1 --2- 172.21.15.111:0/2493594468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d406d030 0x7f53d41f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.128+0000 7f53dbf83640 1 -- 172.21.15.111:0/2493594468 >> 172.21.15.111:0/2493594468 conn(0x7f53d4069ed0 msgr2=0x7f53d4070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:32.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.128+0000 7f53dbf83640 1 -- 172.21.15.111:0/2493594468 shutdown_connections 2024-09-13T17:57:32.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.128+0000 7f53dbf83640 1 -- 172.21.15.111:0/2493594468 wait complete. 2024-09-13T17:57:32.144 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T17:57:32.144 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:57:32.145 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:57:32.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.239+0000 7ff0b95c3640 1 Processor -- start 2024-09-13T17:57:32.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.239+0000 7ff0b95c3640 1 -- start start 2024-09-13T17:57:32.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.240+0000 7ff0b95c3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b415c4c0 0x7ff0b406d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.240+0000 7ff0b95c3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff0b40cd750 0x7ff0b40cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.240+0000 7ff0b95c3640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b40ce0f0 con 0x7ff0b415c8a0 2024-09-13T17:57:32.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.240+0000 7ff0b95c3640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b40cd0e0 con 0x7ff0b40cd750 2024-09-13T17:57:32.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.240+0000 7ff0b95c3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b41601d0 con 0x7ff0b415c4c0 2024-09-13T17:57:32.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.240+0000 7ff0b37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff0b40cd750 0x7ff0b40cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.240+0000 7ff0b27fc640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff0b415c8a0 0x7ff0b40cc9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49158/0 (socket says 172.21.15.111:49158) 2024-09-13T17:57:32.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.240+0000 7ff0b27fc640 1 -- 172.21.15.111:0/1307607553 learned_addr learned my addr 172.21.15.111:0/1307607553 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:32.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.240+0000 7ff0b2ffd640 1 --2- 172.21.15.111:0/1307607553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b415c4c0 0x7ff0b406d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.240+0000 7ff0b37fe640 1 -- 172.21.15.111:0/1307607553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b415c4c0 msgr2=0x7ff0b406d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:32.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.240+0000 7ff0b37fe640 1 --2- 172.21.15.111:0/1307607553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b415c4c0 0x7ff0b406d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.240+0000 7ff0b37fe640 1 -- 172.21.15.111:0/1307607553 >> v1:172.21.15.111:6789/0 conn(0x7ff0b415c8a0 legacy=0x7ff0b40cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:32.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.240+0000 7ff0b37fe640 1 -- 172.21.15.111:0/1307607553 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b406de20 con 0x7ff0b40cd750 2024-09-13T17:57:32.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.240+0000 7ff0b1ffb640 1 -- 172.21.15.111:0/1307607553 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510640607 0 0) 0x7ff0b40ce0f0 con 0x7ff0b415c8a0 2024-09-13T17:57:32.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.240+0000 7ff0b2ffd640 1 --2- 172.21.15.111:0/1307607553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b415c4c0 0x7ff0b406d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:32.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.241+0000 7ff0b37fe640 1 --2- 172.21.15.111:0/1307607553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0b40cd750 0x7ff0b40cdbb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009d90 tx=0x7ff0ac02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8f29cbafc84586 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:32.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.241+0000 7ff0b1ffb640 1 -- 172.21.15.111:0/1307607553 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac02aea0 con 0x7ff0b40cd750 2024-09-13T17:57:32.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.241+0000 7ff0b1ffb640 1 -- 172.21.15.111:0/1307607553 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ac02d050 con 0x7ff0b40cd750 2024-09-13T17:57:32.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.241+0000 7ff0b1ffb640 1 -- 172.21.15.111:0/1307607553 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac007960 con 0x7ff0b40cd750 2024-09-13T17:57:32.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.242+0000 7ff0b95c3640 1 -- 172.21.15.111:0/1307607553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0b40cd750 msgr2=0x7ff0b40cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:32.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.242+0000 7ff0b95c3640 1 --2- 172.21.15.111:0/1307607553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0b40cd750 0x7ff0b40cdbb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009d90 tx=0x7ff0ac02a680 comp rx=0 tx=0).stop 2024-09-13T17:57:32.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.242+0000 7ff0b95c3640 1 -- 172.21.15.111:0/1307607553 shutdown_connections 2024-09-13T17:57:32.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.242+0000 7ff0b95c3640 1 --2- 172.21.15.111:0/1307607553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0b40cd750 0x7ff0b40cdbb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.242+0000 7ff0b95c3640 1 --2- 172.21.15.111:0/1307607553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b415c4c0 0x7ff0b406d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.242+0000 7ff0b95c3640 1 -- 172.21.15.111:0/1307607553 >> 172.21.15.111:0/1307607553 conn(0x7ff0b40989d0 msgr2=0x7ff0b40b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:32.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.242+0000 7ff0b95c3640 1 -- 172.21.15.111:0/1307607553 shutdown_connections 2024-09-13T17:57:32.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.242+0000 7ff0b95c3640 1 -- 172.21.15.111:0/1307607553 wait complete. 2024-09-13T17:57:32.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.243+0000 7ff0b95c3640 1 Processor -- start 2024-09-13T17:57:32.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.243+0000 7ff0b95c3640 1 -- start start 2024-09-13T17:57:32.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b95c3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b40cd750 0x7ff0b415ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b95c3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0b415c4c0 0x7ff0b414bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b95c3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b415c8a0 0x7ff0b40cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b95c3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b414c420 con 0x7ff0b415c4c0 2024-09-13T17:57:32.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b95c3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b415d320 con 0x7ff0b415c8a0 2024-09-13T17:57:32.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b95c3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b415d600 con 0x7ff0b40cd750 2024-09-13T17:57:32.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b27fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0b415c4c0 0x7ff0b414bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b27fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0b415c4c0 0x7ff0b414bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38864/0 (socket says 172.21.15.111:38864) 2024-09-13T17:57:32.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b27fc640 1 -- 172.21.15.111:0/3460020021 learned_addr learned my addr 172.21.15.111:0/3460020021 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b2ffd640 1 --2- 172.21.15.111:0/3460020021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b40cd750 0x7ff0b415ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b37fe640 1 --2- 172.21.15.111:0/3460020021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b415c8a0 0x7ff0b40cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b27fc640 1 -- 172.21.15.111:0/3460020021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b40cd750 msgr2=0x7ff0b415ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b27fc640 1 --2- 172.21.15.111:0/3460020021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b40cd750 0x7ff0b415ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b27fc640 1 -- 172.21.15.111:0/3460020021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b415c8a0 msgr2=0x7ff0b40cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b27fc640 1 --2- 172.21.15.111:0/3460020021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b415c8a0 0x7ff0b40cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b27fc640 1 -- 172.21.15.111:0/3460020021 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac009a70 con 0x7ff0b415c4c0 2024-09-13T17:57:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b27fc640 1 --2- 172.21.15.111:0/3460020021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0b415c4c0 0x7ff0b414bee0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400c410 tx=0x7ff0a4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:32.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff09b7fe640 1 -- 172.21.15.111:0/3460020021 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40074e0 con 0x7ff0b415c4c0 2024-09-13T17:57:32.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff09b7fe640 1 -- 172.21.15.111:0/3460020021 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a4007660 con 0x7ff0b415c4c0 2024-09-13T17:57:32.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff09b7fe640 1 -- 172.21.15.111:0/3460020021 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4004a20 con 0x7ff0b415c4c0 2024-09-13T17:57:32.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b95c3640 1 -- 172.21.15.111:0/3460020021 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b415d8e0 con 0x7ff0b415c4c0 2024-09-13T17:57:32.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.244+0000 7ff0b95c3640 1 -- 172.21.15.111:0/3460020021 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b415dea0 con 0x7ff0b415c4c0 2024-09-13T17:57:32.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.245+0000 7ff0b95c3640 1 -- 172.21.15.111:0/3460020021 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b415e090 con 0x7ff0b415c4c0 2024-09-13T17:57:32.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.246+0000 7ff09b7fe640 1 -- 172.21.15.111:0/3460020021 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff0a4004b80 con 0x7ff0b415c4c0 2024-09-13T17:57:32.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.246+0000 7ff09b7fe640 1 --2- 172.21.15.111:0/3460020021 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff0a006ac00 0x7ff0a006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:32.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.246+0000 7ff09b7fe640 1 -- 172.21.15.111:0/3460020021 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff0a4089470 con 0x7ff0b415c4c0 2024-09-13T17:57:32.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.246+0000 7ff0b2ffd640 1 --2- 172.21.15.111:0/3460020021 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff0a006ac00 0x7ff0a006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:32.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.247+0000 7ff0b2ffd640 1 --2- 172.21.15.111:0/3460020021 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff0a006ac00 0x7ff0a006d0c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800a990 tx=0x7ff0a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:32.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.251+0000 7ff09b7fe640 1 -- 172.21.15.111:0/3460020021 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff0a4054360 con 0x7ff0b415c4c0 2024-09-13T17:57:32.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.445+0000 7ff0b95c3640 1 -- 172.21.15.111:0/3460020021 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff0b40b51b0 con 0x7ff0b415c4c0 2024-09-13T17:57:32.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.445+0000 7ff09b7fe640 1 -- 172.21.15.111:0/3460020021 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff0a4057980 con 0x7ff0b415c4c0 2024-09-13T17:57:32.447 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T17:57:32.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.447+0000 7ff0b95c3640 1 -- 172.21.15.111:0/3460020021 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff0a006ac00 msgr2=0x7ff0a006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:32.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.447+0000 7ff0b95c3640 1 --2- 172.21.15.111:0/3460020021 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff0a006ac00 0x7ff0a006d0c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800a990 tx=0x7ff0a8008040 comp rx=0 tx=0).stop 2024-09-13T17:57:32.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.447+0000 7ff0b95c3640 1 -- 172.21.15.111:0/3460020021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0b415c4c0 msgr2=0x7ff0b414bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:32.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.447+0000 7ff0b95c3640 1 --2- 172.21.15.111:0/3460020021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0b415c4c0 0x7ff0b414bee0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400c410 tx=0x7ff0a4001e70 comp rx=0 tx=0).stop 2024-09-13T17:57:32.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.447+0000 7ff0b95c3640 1 -- 172.21.15.111:0/3460020021 shutdown_connections 2024-09-13T17:57:32.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.447+0000 7ff0b95c3640 1 --2- 172.21.15.111:0/3460020021 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff0a006ac00 0x7ff0a006d0c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.447+0000 7ff0b95c3640 1 --2- 172.21.15.111:0/3460020021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b415c8a0 0x7ff0b40cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.447+0000 7ff0b95c3640 1 --2- 172.21.15.111:0/3460020021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0b415c4c0 0x7ff0b414bee0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.447+0000 7ff0b95c3640 1 --2- 172.21.15.111:0/3460020021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b40cd750 0x7ff0b415ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:32.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.447+0000 7ff0b95c3640 1 -- 172.21.15.111:0/3460020021 >> 172.21.15.111:0/3460020021 conn(0x7ff0b40989d0 msgr2=0x7ff0b40b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:32.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.447+0000 7ff0b95c3640 1 -- 172.21.15.111:0/3460020021 shutdown_connections 2024-09-13T17:57:32.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:32.447+0000 7ff0b95c3640 1 -- 172.21.15.111:0/3460020021 wait complete. 2024-09-13T17:57:32.460 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T17:57:32.460 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:57:32.460 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2024-09-13T17:57:32.460 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-13T17:57:32.460 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:57:32.460 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T17:57:32.460 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:57:32.460 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:57:32.460 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-13T17:57:32.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:32.461 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2024-09-13T17:57:32.477 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:57:32.477 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:57:32.477 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:57:32.477 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:32.477 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-09-13T17:57:32.477 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:32.477 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:57:32.477 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:57:32.478 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2024-09-13T17:57:32.478 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:57:32.478 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:57:32.478 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2024-09-13T17:57:32.478 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:57:32.478 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2024-09-13T17:57:32.478 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:57:32.478 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2024-09-13T17:57:32.478 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-13T17:57:32.478 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42807sec preferred_lft 42807sec 2024-09-13T17:57:32.478 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link noprefixroute 2024-09-13T17:57:32.478 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:32.479 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T17:57:32.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:32.479 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2024-09-13T17:57:32.479 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-13T17:57:32.479 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add name ceph-brx type bridge 2024-09-13T17:57:32.480 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr flush dev ceph-brx 2024-09-13T17:57:32.480 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set ceph-brx up 2024-09-13T17:57:32.480 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T17:57:32.480 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-13T17:57:32.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:32.649 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:57:32.673 INFO:teuthology.orchestra.run.smithi111.stdout:1 2024-09-13T17:57:32.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:32.674 DEBUG:teuthology.orchestra.run.smithi111:> route 2024-09-13T17:57:32.738 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2024-09-13T17:57:32.739 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:57:32.739 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:57:32.739 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:57:32.739 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:57:32.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:32.739 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2024-09-13T17:57:32.740 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-13T17:57:32.740 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:57:32.740 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:57:32.740 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:57:32.740 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-13T17:57:33.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:33.019 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-09-13T17:57:33.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:33.035 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list-id 2024-09-13T17:57:33.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:33.091 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2024-09-13T17:57:33.091 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-13T17:57:33.091 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-13T17:57:33.091 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-13T17:57:33.091 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-13T17:57:33.169 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-13T17:57:33.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:33.169 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2024-09-13T17:57:33.169 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-13T17:57:33.169 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-13T17:57:33.170 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T17:57:33.170 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-13T17:57:33.170 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-13T17:57:33.170 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-13T17:57:33.170 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-13T17:57:33.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:33.323 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2024-09-13T17:57:33.323 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-13T17:57:33.323 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2024-09-13T17:57:33.323 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T17:57:33.323 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-13T17:57:33.366 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-13T17:57:33.366 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:57:33.366 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:57:33.423 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-13T17:57:33.423 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:57:33.423 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:57:33.479 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2024-09-13T17:57:33.545 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:57:33.545 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:57:33.609 INFO:teuthology.orchestra.run.smithi111.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:57:33.610 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:57:33.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:33.611 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2024-09-13T17:57:33.666 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T17:57:33.666 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-13T17:57:33.709 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2024-09-13T17:57:33.735 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:57:33.735 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:57:33.799 INFO:teuthology.orchestra.run.smithi111.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:57:33.801 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:57:33.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:33.801 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2024-09-13T17:57:33.806 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi111.stderr:2024-09-13T17:57:33.805+0000 7fd7251ef480 -1 init, newargv = 0x561537f3a730 newargc=15 2024-09-13T17:57:33.806 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi111.stderr:ceph-fuse[43474]: starting ceph client 2024-09-13T17:57:33.813 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi111.stderr:ceph-fuse[43474]: starting fuse 2024-09-13T17:57:33.823 INFO:teuthology.orchestra.run.smithi111.stdout:41 2024-09-13T17:57:33.823 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-13T17:57:33.823 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:57:33.823 DEBUG:teuthology.orchestra.run.smithi111:> import glob 2024-09-13T17:57:33.823 DEBUG:teuthology.orchestra.run.smithi111:> import re 2024-09-13T17:57:33.823 DEBUG:teuthology.orchestra.run.smithi111:> import os 2024-09-13T17:57:33.823 DEBUG:teuthology.orchestra.run.smithi111:> import subprocess 2024-09-13T17:57:33.823 DEBUG:teuthology.orchestra.run.smithi111:> 2024-09-13T17:57:33.823 DEBUG:teuthology.orchestra.run.smithi111:> def _find_admin_socket(client_name): 2024-09-13T17:57:33.824 DEBUG:teuthology.orchestra.run.smithi111:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-13T17:57:33.824 DEBUG:teuthology.orchestra.run.smithi111:> files = glob.glob(asok_path) 2024-09-13T17:57:33.824 DEBUG:teuthology.orchestra.run.smithi111:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-13T17:57:33.824 DEBUG:teuthology.orchestra.run.smithi111:> 2024-09-13T17:57:33.824 DEBUG:teuthology.orchestra.run.smithi111:> # Given a non-glob path, it better be there 2024-09-13T17:57:33.824 DEBUG:teuthology.orchestra.run.smithi111:> if "*" not in asok_path: 2024-09-13T17:57:33.824 DEBUG:teuthology.orchestra.run.smithi111:> assert(len(files) == 1) 2024-09-13T17:57:33.824 DEBUG:teuthology.orchestra.run.smithi111:> return files[0] 2024-09-13T17:57:33.824 DEBUG:teuthology.orchestra.run.smithi111:> 2024-09-13T17:57:33.824 DEBUG:teuthology.orchestra.run.smithi111:> for f in files: 2024-09-13T17:57:33.824 DEBUG:teuthology.orchestra.run.smithi111:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:57:33.824 DEBUG:teuthology.orchestra.run.smithi111:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:57:33.824 DEBUG:teuthology.orchestra.run.smithi111:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:57:33.824 DEBUG:teuthology.orchestra.run.smithi111:> contents = proc_f.read() 2024-09-13T17:57:33.824 DEBUG:teuthology.orchestra.run.smithi111:> if mountpoint in contents: 2024-09-13T17:57:33.825 DEBUG:teuthology.orchestra.run.smithi111:> return f 2024-09-13T17:57:33.825 DEBUG:teuthology.orchestra.run.smithi111:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:57:33.825 DEBUG:teuthology.orchestra.run.smithi111:> 2024-09-13T17:57:33.825 DEBUG:teuthology.orchestra.run.smithi111:> print(_find_admin_socket("client.admin")) 2024-09-13T17:57:33.825 DEBUG:teuthology.orchestra.run.smithi111:> ' 2024-09-13T17:57:33.936 INFO:teuthology.orchestra.run.smithi111.stdout:/var/run/ceph/ceph-client.admin.43474.asok 2024-09-13T17:57:34.123 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.43474.asok 2024-09-13T17:57:34.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:34.123 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.43474.asok status 2024-09-13T17:57:34.256 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T17:57:34.256 INFO:teuthology.orchestra.run.smithi111.stdout: "metadata": { 2024-09-13T17:57:34.256 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:57:34.256 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:57:34.256 INFO:teuthology.orchestra.run.smithi111.stdout: "entity_id": "admin", 2024-09-13T17:57:34.256 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": "smithi111", 2024-09-13T17:57:34.256 INFO:teuthology.orchestra.run.smithi111.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-13T17:57:34.256 INFO:teuthology.orchestra.run.smithi111.stdout: "pid": "43474", 2024-09-13T17:57:34.257 INFO:teuthology.orchestra.run.smithi111.stdout: "root": "/" 2024-09-13T17:57:34.257 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:57:34.257 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_count": 0, 2024-09-13T17:57:34.257 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_pinned_count": 0, 2024-09-13T17:57:34.257 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4431, 2024-09-13T17:57:34.257 INFO:teuthology.orchestra.run.smithi111.stdout: "inst": { 2024-09-13T17:57:34.257 INFO:teuthology.orchestra.run.smithi111.stdout: "name": { 2024-09-13T17:57:34.257 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "client", 2024-09-13T17:57:34.257 INFO:teuthology.orchestra.run.smithi111.stdout: "num": 4431 2024-09-13T17:57:34.257 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:57:34.258 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2024-09-13T17:57:34.258 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2024-09-13T17:57:34.258 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:57:34.258 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 828427648 2024-09-13T17:57:34.258 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:57:34.258 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:57:34.258 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2024-09-13T17:57:34.258 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2024-09-13T17:57:34.258 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:57:34.258 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 828427648 2024-09-13T17:57:34.258 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:57:34.258 INFO:teuthology.orchestra.run.smithi111.stdout: "inst_str": "client.4431 192.168.0.1:0/828427648", 2024-09-13T17:57:34.259 INFO:teuthology.orchestra.run.smithi111.stdout: "addr_str": "192.168.0.1:0/828427648", 2024-09-13T17:57:34.259 INFO:teuthology.orchestra.run.smithi111.stdout: "inode_count": 1, 2024-09-13T17:57:34.259 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_epoch": 7, 2024-09-13T17:57:34.259 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch": 24, 2024-09-13T17:57:34.259 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:57:34.259 INFO:teuthology.orchestra.run.smithi111.stdout: "blocklisted": false, 2024-09-13T17:57:34.259 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs" 2024-09-13T17:57:34.259 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T17:57:34.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:34.264 DEBUG:teuthology.orchestra.run.smithi111:> stat --file-system '--printf=%T 2024-09-13T17:57:34.264 DEBUG:teuthology.orchestra.run.smithi111:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:57:34.280 INFO:teuthology.orchestra.run.smithi111.stdout:fuseblk 2024-09-13T17:57:34.280 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:57:34.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:34.280 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:57:34.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:34.347 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-13T17:57:34.422 INFO:teuthology.orchestra.run.smithi111.stdout:# file: . 2024-09-13T17:57:34.422 INFO:teuthology.orchestra.run.smithi111.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-13T17:57:34.422 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:34.422 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:57:34.423 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:57:34.479 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:34.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.615+0000 7f36ef953640 1 Processor -- start 2024-09-13T17:57:34.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.615+0000 7f36ef953640 1 -- start start 2024-09-13T17:57:34.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.616+0000 7f36ef953640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f36e8072110 0x7f36e8070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:34.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.616+0000 7f36ef953640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e80c8920 0x7f36e80c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:34.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.616+0000 7f36ef953640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e80abdf0 con 0x7f36e8071d30 2024-09-13T17:57:34.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.616+0000 7f36ef953640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e80c9310 con 0x7f36e8072110 2024-09-13T17:57:34.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.616+0000 7f36ef953640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e80b2280 con 0x7f36e80c8920 2024-09-13T17:57:34.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.616+0000 7f36eceee640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f36e8072110 0x7f36e8070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:34.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.616+0000 7f36eceee640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f36e8072110 0x7f36e8070190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38876/0 (socket says 172.21.15.111:38876) 2024-09-13T17:57:34.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.616+0000 7f36eceee640 1 -- 172.21.15.111:0/2947212657 learned_addr learned my addr 172.21.15.111:0/2947212657 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:34.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.616+0000 7f36edef0640 1 --2- 172.21.15.111:0/2947212657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e80c8920 0x7f36e80c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:34.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.616+0000 7f36eceee640 1 -- 172.21.15.111:0/2947212657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e80c8920 msgr2=0x7f36e80c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:34.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.616+0000 7f36eceee640 1 --2- 172.21.15.111:0/2947212657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e80c8920 0x7f36e80c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:34.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.616+0000 7f36eceee640 1 -- 172.21.15.111:0/2947212657 >> v1:172.21.15.188:6789/0 conn(0x7f36e8071d30 legacy=0x7f36e80b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:34.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.616+0000 7f36eceee640 1 -- 172.21.15.111:0/2947212657 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e80ab1e0 con 0x7f36e8072110 2024-09-13T17:57:34.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.617+0000 7f36eceee640 1 --2- 172.21.15.111:0/2947212657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36e8072110 0x7f36e8070190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009870 tx=0x7f36d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd806f86c04f6a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:34.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.617+0000 7f36dffff640 1 -- 172.21.15.111:0/2947212657 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8032440 con 0x7f36e8072110 2024-09-13T17:57:34.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.617+0000 7f36dffff640 1 -- 172.21.15.111:0/2947212657 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d80325a0 con 0x7f36e8072110 2024-09-13T17:57:34.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.617+0000 7f36dffff640 1 -- 172.21.15.111:0/2947212657 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8032930 con 0x7f36e8072110 2024-09-13T17:57:34.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.618+0000 7f36ef953640 1 -- 172.21.15.111:0/2947212657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36e8072110 msgr2=0x7f36e8070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:34.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.618+0000 7f36ef953640 1 --2- 172.21.15.111:0/2947212657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36e8072110 0x7f36e8070190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009870 tx=0x7f36d802a270 comp rx=0 tx=0).stop 2024-09-13T17:57:34.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.618+0000 7f36ef953640 1 -- 172.21.15.111:0/2947212657 shutdown_connections 2024-09-13T17:57:34.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.618+0000 7f36ef953640 1 --2- 172.21.15.111:0/2947212657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36e8072110 0x7f36e8070190 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:34.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.618+0000 7f36ef953640 1 --2- 172.21.15.111:0/2947212657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e80c8920 0x7f36e80c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:34.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.618+0000 7f36ef953640 1 -- 172.21.15.111:0/2947212657 >> 172.21.15.111:0/2947212657 conn(0x7f36e8069ed0 msgr2=0x7f36e806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:34.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.618+0000 7f36ef953640 1 -- 172.21.15.111:0/2947212657 shutdown_connections 2024-09-13T17:57:34.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.618+0000 7f36ef953640 1 -- 172.21.15.111:0/2947212657 wait complete. 2024-09-13T17:57:34.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.619+0000 7f36ef953640 1 Processor -- start 2024-09-13T17:57:34.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.619+0000 7f36ef953640 1 -- start start 2024-09-13T17:57:34.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36ef953640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e8071d30 0x7f36e81f3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:34.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36ef953640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36e8072110 0x7f36e81e43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:34.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36ef953640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e80c8920 0x7f36e80ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:34.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36ef953640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e81e3b20 con 0x7f36e8072110 2024-09-13T17:57:34.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36ef953640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e81e2fa0 con 0x7f36e80c8920 2024-09-13T17:57:34.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36ef953640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e81e3280 con 0x7f36e8071d30 2024-09-13T17:57:34.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36eceee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36e8072110 0x7f36e81e43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:34.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36eceee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36e8072110 0x7f36e81e43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38888/0 (socket says 172.21.15.111:38888) 2024-09-13T17:57:34.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36eceee640 1 -- 172.21.15.111:0/147689418 learned_addr learned my addr 172.21.15.111:0/147689418 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:34.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36ed6ef640 1 --2- 172.21.15.111:0/147689418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e8071d30 0x7f36e81f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:34.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36edef0640 1 --2- 172.21.15.111:0/147689418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e80c8920 0x7f36e80ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:34.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36eceee640 1 -- 172.21.15.111:0/147689418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e8071d30 msgr2=0x7f36e81f3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:34.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36eceee640 1 --2- 172.21.15.111:0/147689418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e8071d30 0x7f36e81f3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:34.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36eceee640 1 -- 172.21.15.111:0/147689418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e80c8920 msgr2=0x7f36e80ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:34.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36eceee640 1 --2- 172.21.15.111:0/147689418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e80c8920 0x7f36e80ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:34.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36eceee640 1 -- 172.21.15.111:0/147689418 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d80094d0 con 0x7f36e8072110 2024-09-13T17:57:34.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36edef0640 1 --2- 172.21.15.111:0/147689418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e80c8920 0x7f36e80ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:34.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36eceee640 1 --2- 172.21.15.111:0/147689418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36e8072110 0x7f36e81e43a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f36d8000c00 tx=0x7f36d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:34.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36ddffb640 1 -- 172.21.15.111:0/147689418 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8032440 con 0x7f36e8072110 2024-09-13T17:57:34.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36ddffb640 1 -- 172.21.15.111:0/147689418 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d8030b80 con 0x7f36e8072110 2024-09-13T17:57:34.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36ef953640 1 -- 172.21.15.111:0/147689418 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e81e3560 con 0x7f36e8072110 2024-09-13T17:57:34.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.620+0000 7f36ddffb640 1 -- 172.21.15.111:0/147689418 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d803d440 con 0x7f36e8072110 2024-09-13T17:57:34.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.621+0000 7f36ef953640 1 -- 172.21.15.111:0/147689418 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e81f35a0 con 0x7f36e8072110 2024-09-13T17:57:34.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.621+0000 7f36ef953640 1 -- 172.21.15.111:0/147689418 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e81e33c0 con 0x7f36e8072110 2024-09-13T17:57:34.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.622+0000 7f36ddffb640 1 -- 172.21.15.111:0/147689418 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f36d8039040 con 0x7f36e8072110 2024-09-13T17:57:34.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.622+0000 7f36ddffb640 1 --2- 172.21.15.111:0/147689418 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f36cc06ac00 0x7f36cc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:34.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.622+0000 7f36ddffb640 1 -- 172.21.15.111:0/147689418 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f36d80aa080 con 0x7f36e8072110 2024-09-13T17:57:34.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.622+0000 7f36ed6ef640 1 --2- 172.21.15.111:0/147689418 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f36cc06ac00 0x7f36cc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:34.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.623+0000 7f36ed6ef640 1 --2- 172.21.15.111:0/147689418 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f36cc06ac00 0x7f36cc06d0c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f36e4000f60 tx=0x7f36e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:34.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.626+0000 7f36ddffb640 1 -- 172.21.15.111:0/147689418 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f36e81e33c0 con 0x7f36e8072110 2024-09-13T17:57:34.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.788+0000 7f36ef953640 1 -- 172.21.15.111:0/147689418 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36e81e33c0 con 0x7f36e8072110 2024-09-13T17:57:34.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.790+0000 7f36ddffb640 1 -- 172.21.15.111:0/147689418 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f36e81e33c0 con 0x7f36e8072110 2024-09-13T17:57:34.792 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:34.792 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:30.065235+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:34.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.792+0000 7f36ef953640 1 -- 172.21.15.111:0/147689418 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f36cc06ac00 msgr2=0x7f36cc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:34.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.792+0000 7f36ef953640 1 --2- 172.21.15.111:0/147689418 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f36cc06ac00 0x7f36cc06d0c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f36e4000f60 tx=0x7f36e4009040 comp rx=0 tx=0).stop 2024-09-13T17:57:34.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.792+0000 7f36ef953640 1 -- 172.21.15.111:0/147689418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36e8072110 msgr2=0x7f36e81e43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:34.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.792+0000 7f36ef953640 1 --2- 172.21.15.111:0/147689418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36e8072110 0x7f36e81e43a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f36d8000c00 tx=0x7f36d80046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:34.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.792+0000 7f36ef953640 1 -- 172.21.15.111:0/147689418 shutdown_connections 2024-09-13T17:57:34.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.792+0000 7f36ef953640 1 --2- 172.21.15.111:0/147689418 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f36cc06ac00 0x7f36cc06d0c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:34.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.792+0000 7f36ef953640 1 --2- 172.21.15.111:0/147689418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e80c8920 0x7f36e80ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:34.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.792+0000 7f36ef953640 1 --2- 172.21.15.111:0/147689418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36e8072110 0x7f36e81e43a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:34.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.792+0000 7f36ef953640 1 --2- 172.21.15.111:0/147689418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e8071d30 0x7f36e81f3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:34.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.792+0000 7f36ef953640 1 -- 172.21.15.111:0/147689418 >> 172.21.15.111:0/147689418 conn(0x7f36e8069ed0 msgr2=0x7f36e80af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:34.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.792+0000 7f36ef953640 1 -- 172.21.15.111:0/147689418 shutdown_connections 2024-09-13T17:57:34.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.792+0000 7f36ef953640 1 -- 172.21.15.111:0/147689418 wait complete. 2024-09-13T17:57:34.804 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:57:34.804 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:57:34.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.901+0000 7f8443dd4640 1 Processor -- start 2024-09-13T17:57:34.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.902+0000 7f8443dd4640 1 -- start start 2024-09-13T17:57:34.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.902+0000 7f8443dd4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f843c15a7f0 0x7f843c0ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:34.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.902+0000 7f8443dd4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f843c0989d0 0x7f843c098e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:34.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.902+0000 7f8443dd4640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843c0cc300 con 0x7f843c0993a0 2024-09-13T17:57:34.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.902+0000 7f8443dd4640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c0b3d20 con 0x7f843c0989d0 2024-09-13T17:57:34.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.902+0000 7f8443dd4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c0997d0 con 0x7f843c15a7f0 2024-09-13T17:57:34.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.903+0000 7f844136f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f843c0989d0 0x7f843c098e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:34.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.903+0000 7f8442371640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f843c0993a0 0x7f843c0caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49162/0 (socket says 172.21.15.111:49162) 2024-09-13T17:57:34.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.903+0000 7f8442371640 1 -- 172.21.15.111:0/868266004 learned_addr learned my addr 172.21.15.111:0/868266004 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:34.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.903+0000 7f8441b70640 1 --2- 172.21.15.111:0/868266004 >> v2:172.21.15.188:3300/0 conn(0x7f843c15a7f0 0x7f843c0ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:34.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.903+0000 7f844136f640 1 -- 172.21.15.111:0/868266004 >> v2:172.21.15.188:3300/0 conn(0x7f843c15a7f0 msgr2=0x7f843c0ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:34.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.903+0000 7f844136f640 1 --2- 172.21.15.111:0/868266004 >> v2:172.21.15.188:3300/0 conn(0x7f843c15a7f0 0x7f843c0ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:34.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.903+0000 7f844136f640 1 -- 172.21.15.111:0/868266004 >> v1:172.21.15.111:6789/0 conn(0x7f843c0993a0 legacy=0x7f843c0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:34.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.903+0000 7f844136f640 1 -- 172.21.15.111:0/868266004 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c0b33f0 con 0x7f843c0989d0 2024-09-13T17:57:34.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.903+0000 7f8440b6e640 1 -- 172.21.15.111:0/868266004 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270252798 0 0) 0x7f843c0cc300 con 0x7f843c0993a0 2024-09-13T17:57:34.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.904+0000 7f844136f640 1 --2- 172.21.15.111:0/868266004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f843c0989d0 0x7f843c098e60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f843c071930 tx=0x7f842c02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f355cffd4040a384 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:34.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.904+0000 7f8440b6e640 1 -- 172.21.15.111:0/868266004 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c02add0 con 0x7f843c0989d0 2024-09-13T17:57:34.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.904+0000 7f8440b6e640 1 -- 172.21.15.111:0/868266004 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c032440 con 0x7f843c0989d0 2024-09-13T17:57:34.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.904+0000 7f8440b6e640 1 -- 172.21.15.111:0/868266004 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c0327d0 con 0x7f843c0989d0 2024-09-13T17:57:34.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.904+0000 7f8443dd4640 1 -- 172.21.15.111:0/868266004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f843c0989d0 msgr2=0x7f843c098e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:34.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.904+0000 7f8443dd4640 1 --2- 172.21.15.111:0/868266004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f843c0989d0 0x7f843c098e60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f843c071930 tx=0x7f842c02a080 comp rx=0 tx=0).stop 2024-09-13T17:57:34.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.904+0000 7f8443dd4640 1 -- 172.21.15.111:0/868266004 shutdown_connections 2024-09-13T17:57:34.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.904+0000 7f8443dd4640 1 --2- 172.21.15.111:0/868266004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f843c0989d0 0x7f843c098e60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:34.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.904+0000 7f8443dd4640 1 --2- 172.21.15.111:0/868266004 >> v2:172.21.15.188:3300/0 conn(0x7f843c15a7f0 0x7f843c0ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:34.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.904+0000 7f8443dd4640 1 -- 172.21.15.111:0/868266004 >> 172.21.15.111:0/868266004 conn(0x7f843c0b1f60 msgr2=0x7f843c0b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:34.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.904+0000 7f8443dd4640 1 -- 172.21.15.111:0/868266004 shutdown_connections 2024-09-13T17:57:34.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.904+0000 7f8443dd4640 1 -- 172.21.15.111:0/868266004 wait complete. 2024-09-13T17:57:34.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.905+0000 7f8443dd4640 1 Processor -- start 2024-09-13T17:57:34.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.906+0000 7f8443dd4640 1 -- start start 2024-09-13T17:57:34.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.906+0000 7f8443dd4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f843c0989d0 0x7f843c1562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:34.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.906+0000 7f8443dd4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f843c0993a0 0x7f843c0c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:34.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.906+0000 7f8443dd4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f843c15a7f0 0x7f843c153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:34.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.906+0000 7f8443dd4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0c4ed0 con 0x7f843c0993a0 2024-09-13T17:57:34.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.906+0000 7f8443dd4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0c4060 con 0x7f843c15a7f0 2024-09-13T17:57:34.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.906+0000 7f8443dd4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0c42c0 con 0x7f843c0989d0 2024-09-13T17:57:34.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.906+0000 7f844136f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f843c0993a0 0x7f843c0c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:34.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.906+0000 7f844136f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f843c0993a0 0x7f843c0c3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38900/0 (socket says 172.21.15.111:38900) 2024-09-13T17:57:34.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.906+0000 7f844136f640 1 -- 172.21.15.111:0/2560237236 learned_addr learned my addr 172.21.15.111:0/2560237236 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:34.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.906+0000 7f8441b70640 1 --2- 172.21.15.111:0/2560237236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f843c0989d0 0x7f843c1562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:34.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.906+0000 7f8442371640 1 --2- 172.21.15.111:0/2560237236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f843c15a7f0 0x7f843c153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:34.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.907+0000 7f844136f640 1 -- 172.21.15.111:0/2560237236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f843c0989d0 msgr2=0x7f843c1562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:34.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.907+0000 7f844136f640 1 --2- 172.21.15.111:0/2560237236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f843c0989d0 0x7f843c1562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:34.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.907+0000 7f844136f640 1 -- 172.21.15.111:0/2560237236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f843c15a7f0 msgr2=0x7f843c153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:34.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.907+0000 7f844136f640 1 --2- 172.21.15.111:0/2560237236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f843c15a7f0 0x7f843c153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:34.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.907+0000 7f844136f640 1 -- 172.21.15.111:0/2560237236 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c009400 con 0x7f843c0993a0 2024-09-13T17:57:34.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.907+0000 7f8441b70640 1 --2- 172.21.15.111:0/2560237236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f843c0989d0 0x7f843c1562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:34.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.907+0000 7f844136f640 1 --2- 172.21.15.111:0/2560237236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f843c0993a0 0x7f843c0c3b20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f842c000c00 tx=0x7f842c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:34.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.907+0000 7f84327fc640 1 -- 172.21.15.111:0/2560237236 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c03d910 con 0x7f843c0993a0 2024-09-13T17:57:34.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.907+0000 7f84327fc640 1 -- 172.21.15.111:0/2560237236 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c03da90 con 0x7f843c0993a0 2024-09-13T17:57:34.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.907+0000 7f84327fc640 1 -- 172.21.15.111:0/2560237236 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c03de40 con 0x7f843c0993a0 2024-09-13T17:57:34.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.907+0000 7f8443dd4640 1 -- 172.21.15.111:0/2560237236 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843c0c45a0 con 0x7f843c0993a0 2024-09-13T17:57:34.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.907+0000 7f8443dd4640 1 -- 172.21.15.111:0/2560237236 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843c0c4b60 con 0x7f843c0993a0 2024-09-13T17:57:34.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.908+0000 7f8443dd4640 1 -- 172.21.15.111:0/2560237236 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843c0c4cf0 con 0x7f843c0993a0 2024-09-13T17:57:34.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.908+0000 7f84327fc640 1 -- 172.21.15.111:0/2560237236 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f842c039040 con 0x7f843c0993a0 2024-09-13T17:57:34.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.909+0000 7f84327fc640 1 --2- 172.21.15.111:0/2560237236 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f842006ac00 0x7f842006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:34.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.909+0000 7f84327fc640 1 -- 172.21.15.111:0/2560237236 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f842c0a9f00 con 0x7f843c0993a0 2024-09-13T17:57:34.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.909+0000 7f8441b70640 1 --2- 172.21.15.111:0/2560237236 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f842006ac00 0x7f842006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:34.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.909+0000 7f8441b70640 1 --2- 172.21.15.111:0/2560237236 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f842006ac00 0x7f842006d0c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f843c0b56f0 tx=0x7f8438008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:34.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:34.913+0000 7f84327fc640 1 -- 172.21.15.111:0/2560237236 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f842c074e80 con 0x7f843c0993a0 2024-09-13T17:57:35.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.072+0000 7f8443dd4640 1 -- 172.21.15.111:0/2560237236 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f843c098e60 con 0x7f843c0993a0 2024-09-13T17:57:35.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.073+0000 7f84327fc640 1 -- 172.21.15.111:0/2560237236 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f842c0784a0 con 0x7f843c0993a0 2024-09-13T17:57:35.075 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:57:35.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.075+0000 7f8443dd4640 1 -- 172.21.15.111:0/2560237236 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f842006ac00 msgr2=0x7f842006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.075+0000 7f8443dd4640 1 --2- 172.21.15.111:0/2560237236 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f842006ac00 0x7f842006d0c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f843c0b56f0 tx=0x7f8438008040 comp rx=0 tx=0).stop 2024-09-13T17:57:35.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.075+0000 7f8443dd4640 1 -- 172.21.15.111:0/2560237236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f843c0993a0 msgr2=0x7f843c0c3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.075+0000 7f8443dd4640 1 --2- 172.21.15.111:0/2560237236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f843c0993a0 0x7f843c0c3b20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f842c000c00 tx=0x7f842c0046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.075+0000 7f8443dd4640 1 -- 172.21.15.111:0/2560237236 shutdown_connections 2024-09-13T17:57:35.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.075+0000 7f8443dd4640 1 --2- 172.21.15.111:0/2560237236 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f842006ac00 0x7f842006d0c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.075+0000 7f8443dd4640 1 --2- 172.21.15.111:0/2560237236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f843c15a7f0 0x7f843c153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.075+0000 7f8443dd4640 1 --2- 172.21.15.111:0/2560237236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f843c0993a0 0x7f843c0c3b20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.075+0000 7f8443dd4640 1 --2- 172.21.15.111:0/2560237236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f843c0989d0 0x7f843c1562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.075+0000 7f8443dd4640 1 -- 172.21.15.111:0/2560237236 >> 172.21.15.111:0/2560237236 conn(0x7f843c0b1f60 msgr2=0x7f843c15d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:35.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.075+0000 7f8443dd4640 1 -- 172.21.15.111:0/2560237236 shutdown_connections 2024-09-13T17:57:35.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.075+0000 7f8443dd4640 1 -- 172.21.15.111:0/2560237236 wait complete. 2024-09-13T17:57:35.087 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:57:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.185+0000 7f476c8aa640 1 Processor -- start 2024-09-13T17:57:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.185+0000 7f476c8aa640 1 -- start start 2024-09-13T17:57:35.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.186+0000 7f476c8aa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4768158b00 0x7f476806c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.186+0000 7f476c8aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f476806c6d0 0x7f4768070b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.186+0000 7f476c8aa640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4768071050 0x7f47680714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.186+0000 7f476c8aa640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f476806cb30 con 0x7f4768071050 2024-09-13T17:57:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.186+0000 7f476c8aa640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47680b2d10 con 0x7f4768158b00 2024-09-13T17:57:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.186+0000 7f476c8aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47680b2ff0 con 0x7f476806c6d0 2024-09-13T17:57:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.186+0000 7f4766d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4768071050 0x7f47680714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.187+0000 7f4766d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4768071050 0x7f47680714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38914/0 (socket says 172.21.15.111:38914) 2024-09-13T17:57:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.187+0000 7f4766d9d640 1 -- 172.21.15.111:0/1553988922 learned_addr learned my addr 172.21.15.111:0/1553988922 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.187+0000 7f476659c640 1 --2- 172.21.15.111:0/1553988922 >> v2:172.21.15.188:3300/0 conn(0x7f4768158b00 0x7f476806c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.187+0000 7f4765d9b640 1 --2- 172.21.15.111:0/1553988922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f476806c6d0 0x7f4768070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.187+0000 7f4766d9d640 1 -- 172.21.15.111:0/1553988922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f476806c6d0 msgr2=0x7f4768070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.187+0000 7f4766d9d640 1 --2- 172.21.15.111:0/1553988922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f476806c6d0 0x7f4768070b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.187+0000 7f4766d9d640 1 -- 172.21.15.111:0/1553988922 >> v2:172.21.15.188:3300/0 conn(0x7f4768158b00 msgr2=0x7f476806c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.187+0000 7f4766d9d640 1 --2- 172.21.15.111:0/1553988922 >> v2:172.21.15.188:3300/0 conn(0x7f4768158b00 0x7f476806c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.187+0000 7f4766d9d640 1 -- 172.21.15.111:0/1553988922 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4768070310 con 0x7f4768071050 2024-09-13T17:57:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.187+0000 7f476659c640 1 --2- 172.21.15.111:0/1553988922 >> v2:172.21.15.188:3300/0 conn(0x7f4768158b00 0x7f476806c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.187+0000 7f4766d9d640 1 --2- 172.21.15.111:0/1553988922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4768071050 0x7f47680714e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f476806ade0 tx=0x7f476002c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed877296f8f7f72 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.188+0000 7f476559a640 1 -- 172.21.15.111:0/1553988922 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476002f070 con 0x7f4768071050 2024-09-13T17:57:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.188+0000 7f476559a640 1 -- 172.21.15.111:0/1553988922 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4760035400 con 0x7f4768071050 2024-09-13T17:57:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.188+0000 7f476559a640 1 -- 172.21.15.111:0/1553988922 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760035790 con 0x7f4768071050 2024-09-13T17:57:35.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.188+0000 7f476c8aa640 1 -- 172.21.15.111:0/1553988922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4768071050 msgr2=0x7f47680714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.188+0000 7f476c8aa640 1 --2- 172.21.15.111:0/1553988922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4768071050 0x7f47680714e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f476806ade0 tx=0x7f476002c8a0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.188+0000 7f476c8aa640 1 -- 172.21.15.111:0/1553988922 shutdown_connections 2024-09-13T17:57:35.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.188+0000 7f476c8aa640 1 --2- 172.21.15.111:0/1553988922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f476806c6d0 0x7f4768070b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.188+0000 7f476c8aa640 1 --2- 172.21.15.111:0/1553988922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4768071050 0x7f47680714e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.188+0000 7f476c8aa640 1 --2- 172.21.15.111:0/1553988922 >> v2:172.21.15.188:3300/0 conn(0x7f4768158b00 0x7f476806c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.188+0000 7f476c8aa640 1 -- 172.21.15.111:0/1553988922 >> 172.21.15.111:0/1553988922 conn(0x7f476806a0b0 msgr2=0x7f476806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:35.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.188+0000 7f476c8aa640 1 -- 172.21.15.111:0/1553988922 shutdown_connections 2024-09-13T17:57:35.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.188+0000 7f476c8aa640 1 -- 172.21.15.111:0/1553988922 wait complete. 2024-09-13T17:57:35.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.190+0000 7f476c8aa640 1 Processor -- start 2024-09-13T17:57:35.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.190+0000 7f476c8aa640 1 -- start start 2024-09-13T17:57:35.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.190+0000 7f476c8aa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f476806c6d0 0x7f47681e2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.190+0000 7f476c8aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4768071050 0x7f47681572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.190+0000 7f476c8aa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4768158b00 0x7f476809b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.190+0000 7f476c8aa640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4768157810 con 0x7f4768158b00 2024-09-13T17:57:35.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.190+0000 7f476c8aa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47681e2ab0 con 0x7f476806c6d0 2024-09-13T17:57:35.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.190+0000 7f476c8aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47681e1ea0 con 0x7f4768071050 2024-09-13T17:57:35.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.190+0000 7f4766d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4768158b00 0x7f476809b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.190+0000 7f4766d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4768158b00 0x7f476809b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38924/0 (socket says 172.21.15.111:38924) 2024-09-13T17:57:35.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.190+0000 7f4766d9d640 1 -- 172.21.15.111:0/3353704657 learned_addr learned my addr 172.21.15.111:0/3353704657 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:35.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.191+0000 7f4765d9b640 1 --2- 172.21.15.111:0/3353704657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4768071050 0x7f47681572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.191+0000 7f476659c640 1 --2- 172.21.15.111:0/3353704657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f476806c6d0 0x7f47681e2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.191+0000 7f4766d9d640 1 -- 172.21.15.111:0/3353704657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4768071050 msgr2=0x7f47681572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.191+0000 7f4766d9d640 1 --2- 172.21.15.111:0/3353704657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4768071050 0x7f47681572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.191+0000 7f4766d9d640 1 -- 172.21.15.111:0/3353704657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f476806c6d0 msgr2=0x7f47681e2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.191+0000 7f4766d9d640 1 --2- 172.21.15.111:0/3353704657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f476806c6d0 0x7f47681e2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.191+0000 7f4766d9d640 1 -- 172.21.15.111:0/3353704657 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476000bd40 con 0x7f4768158b00 2024-09-13T17:57:35.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.191+0000 7f476659c640 1 --2- 172.21.15.111:0/3353704657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f476806c6d0 0x7f47681e2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:35.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.191+0000 7f4766d9d640 1 --2- 172.21.15.111:0/3353704657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4768158b00 0x7f476809b440 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f476002ce90 tx=0x7f4760004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:35.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.191+0000 7f475effd640 1 -- 172.21.15.111:0/3353704657 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760034b60 con 0x7f4768158b00 2024-09-13T17:57:35.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.191+0000 7f475effd640 1 -- 172.21.15.111:0/3353704657 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4760034cc0 con 0x7f4768158b00 2024-09-13T17:57:35.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.191+0000 7f476c8aa640 1 -- 172.21.15.111:0/3353704657 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47681e2180 con 0x7f4768158b00 2024-09-13T17:57:35.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.191+0000 7f475effd640 1 -- 172.21.15.111:0/3353704657 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476003e670 con 0x7f4768158b00 2024-09-13T17:57:35.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.191+0000 7f476c8aa640 1 -- 172.21.15.111:0/3353704657 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47681e2740 con 0x7f4768158b00 2024-09-13T17:57:35.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.192+0000 7f476c8aa640 1 -- 172.21.15.111:0/3353704657 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47681e2c20 con 0x7f4768158b00 2024-09-13T17:57:35.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.193+0000 7f475effd640 1 -- 172.21.15.111:0/3353704657 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f476003e7d0 con 0x7f4768158b00 2024-09-13T17:57:35.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.193+0000 7f475effd640 1 --2- 172.21.15.111:0/3353704657 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f474806ac00 0x7f474806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.193+0000 7f475effd640 1 -- 172.21.15.111:0/3353704657 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f47600abfc0 con 0x7f4768158b00 2024-09-13T17:57:35.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.193+0000 7f476659c640 1 --2- 172.21.15.111:0/3353704657 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f474806ac00 0x7f474806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.193+0000 7f476659c640 1 --2- 172.21.15.111:0/3353704657 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f474806ac00 0x7f474806d0c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4750006fd0 tx=0x7f4750008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:35.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.197+0000 7f475effd640 1 -- 172.21.15.111:0/3353704657 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f47600df9c0 con 0x7f4768158b00 2024-09-13T17:57:35.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.359+0000 7f476c8aa640 1 -- 172.21.15.111:0/3353704657 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f47680714e0 con 0x7f4768158b00 2024-09-13T17:57:35.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.359+0000 7f475effd640 1 -- 172.21.15.111:0/3353704657 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f476007a9d0 con 0x7f4768158b00 2024-09-13T17:57:35.361 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:57:35.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.361+0000 7f476c8aa640 1 -- 172.21.15.111:0/3353704657 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f474806ac00 msgr2=0x7f474806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.361+0000 7f476c8aa640 1 --2- 172.21.15.111:0/3353704657 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f474806ac00 0x7f474806d0c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4750006fd0 tx=0x7f4750008040 comp rx=0 tx=0).stop 2024-09-13T17:57:35.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.361+0000 7f476c8aa640 1 -- 172.21.15.111:0/3353704657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4768158b00 msgr2=0x7f476809b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.361+0000 7f476c8aa640 1 --2- 172.21.15.111:0/3353704657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4768158b00 0x7f476809b440 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f476002ce90 tx=0x7f4760004a40 comp rx=0 tx=0).stop 2024-09-13T17:57:35.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.361+0000 7f476c8aa640 1 -- 172.21.15.111:0/3353704657 shutdown_connections 2024-09-13T17:57:35.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.361+0000 7f476c8aa640 1 --2- 172.21.15.111:0/3353704657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4768158b00 0x7f476809b440 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.361+0000 7f476c8aa640 1 --2- 172.21.15.111:0/3353704657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4768071050 0x7f47681572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.361+0000 7f476c8aa640 1 --2- 172.21.15.111:0/3353704657 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f474806ac00 0x7f474806d0c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.361+0000 7f476c8aa640 1 --2- 172.21.15.111:0/3353704657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f476806c6d0 0x7f47681e2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.361+0000 7f476c8aa640 1 -- 172.21.15.111:0/3353704657 >> 172.21.15.111:0/3353704657 conn(0x7f476806a0b0 msgr2=0x7f47680acaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:35.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.361+0000 7f476c8aa640 1 -- 172.21.15.111:0/3353704657 shutdown_connections 2024-09-13T17:57:35.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.361+0000 7f476c8aa640 1 -- 172.21.15.111:0/3353704657 wait complete. 2024-09-13T17:57:35.373 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:57:35.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.471+0000 7fd04415e640 1 Processor -- start 2024-09-13T17:57:35.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.471+0000 7fd04415e640 1 -- start start 2024-09-13T17:57:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.472+0000 7fd04415e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd03c06c8a0 0x7fd03c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.472+0000 7fd04415e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd03c071b20 0x7fd03c0b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.472+0000 7fd04415e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd03c0b2d10 con 0x7fd03c158b00 2024-09-13T17:57:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.472+0000 7fd04415e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03c0b2ff0 con 0x7fd03c06c8a0 2024-09-13T17:57:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.472+0000 7fd04415e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03c070310 con 0x7fd03c071b20 2024-09-13T17:57:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.472+0000 7fd0416f9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd03c06c8a0 0x7fd03c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.472+0000 7fd0416f9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd03c06c8a0 0x7fd03c0b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38934/0 (socket says 172.21.15.111:38934) 2024-09-13T17:57:35.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.472+0000 7fd0416f9640 1 -- 172.21.15.111:0/2866345047 learned_addr learned my addr 172.21.15.111:0/2866345047 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:35.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.472+0000 7fd0426fb640 1 --2- 172.21.15.111:0/2866345047 >> v2:172.21.15.188:3300/0 conn(0x7fd03c071b20 0x7fd03c0b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.472+0000 7fd0416f9640 1 -- 172.21.15.111:0/2866345047 >> v2:172.21.15.188:3300/0 conn(0x7fd03c071b20 msgr2=0x7fd03c0b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:35.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.472+0000 7fd0416f9640 1 --2- 172.21.15.111:0/2866345047 >> v2:172.21.15.188:3300/0 conn(0x7fd03c071b20 0x7fd03c0b16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.472+0000 7fd0416f9640 1 -- 172.21.15.111:0/2866345047 >> v1:172.21.15.188:6789/0 conn(0x7fd03c158b00 legacy=0x7fd03c06c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.472+0000 7fd0416f9640 1 -- 172.21.15.111:0/2866345047 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c0705f0 con 0x7fd03c06c8a0 2024-09-13T17:57:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.473+0000 7fd0416f9640 1 --2- 172.21.15.111:0/2866345047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c06c8a0 0x7fd03c0b1160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd03c06da20 tx=0x7fd02c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4758d45114542232 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.473+0000 7fd040ef8640 1 -- 172.21.15.111:0/2866345047 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c032440 con 0x7fd03c06c8a0 2024-09-13T17:57:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.473+0000 7fd040ef8640 1 -- 172.21.15.111:0/2866345047 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02c0325a0 con 0x7fd03c06c8a0 2024-09-13T17:57:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.473+0000 7fd040ef8640 1 -- 172.21.15.111:0/2866345047 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c032930 con 0x7fd03c06c8a0 2024-09-13T17:57:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.474+0000 7fd04415e640 1 -- 172.21.15.111:0/2866345047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c06c8a0 msgr2=0x7fd03c0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.474+0000 7fd04415e640 1 --2- 172.21.15.111:0/2866345047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c06c8a0 0x7fd03c0b1160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd03c06da20 tx=0x7fd02c02a270 comp rx=0 tx=0).stop 2024-09-13T17:57:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.474+0000 7fd04415e640 1 -- 172.21.15.111:0/2866345047 shutdown_connections 2024-09-13T17:57:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.474+0000 7fd04415e640 1 --2- 172.21.15.111:0/2866345047 >> v2:172.21.15.188:3300/0 conn(0x7fd03c071b20 0x7fd03c0b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.474+0000 7fd04415e640 1 --2- 172.21.15.111:0/2866345047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c06c8a0 0x7fd03c0b1160 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.474+0000 7fd04415e640 1 -- 172.21.15.111:0/2866345047 >> 172.21.15.111:0/2866345047 conn(0x7fd03c06a0b0 msgr2=0x7fd03c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:35.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.474+0000 7fd04415e640 1 -- 172.21.15.111:0/2866345047 shutdown_connections 2024-09-13T17:57:35.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.474+0000 7fd04415e640 1 -- 172.21.15.111:0/2866345047 wait complete. 2024-09-13T17:57:35.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.475+0000 7fd04415e640 1 Processor -- start 2024-09-13T17:57:35.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.475+0000 7fd04415e640 1 -- start start 2024-09-13T17:57:35.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd04415e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c06c8a0 0x7fd03c157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd04415e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd03c071b20 0x7fd03c1548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd04415e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd03c158b00 0x7fd03c0977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd04415e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c157b90 con 0x7fd03c06c8a0 2024-09-13T17:57:35.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd04415e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c157e70 con 0x7fd03c158b00 2024-09-13T17:57:35.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd04415e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c158150 con 0x7fd03c071b20 2024-09-13T17:57:35.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd041efa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c06c8a0 0x7fd03c157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd041efa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c06c8a0 0x7fd03c157490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38946/0 (socket says 172.21.15.111:38946) 2024-09-13T17:57:35.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd041efa640 1 -- 172.21.15.111:0/3159668722 learned_addr learned my addr 172.21.15.111:0/3159668722 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:35.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd0416f9640 1 --2- 172.21.15.111:0/3159668722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd03c071b20 0x7fd03c1548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd0426fb640 1 --2- 172.21.15.111:0/3159668722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd03c158b00 0x7fd03c0977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd041efa640 1 -- 172.21.15.111:0/3159668722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd03c071b20 msgr2=0x7fd03c1548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd041efa640 1 --2- 172.21.15.111:0/3159668722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd03c071b20 0x7fd03c1548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd041efa640 1 -- 172.21.15.111:0/3159668722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd03c158b00 msgr2=0x7fd03c0977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd041efa640 1 --2- 172.21.15.111:0/3159668722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd03c158b00 0x7fd03c0977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd041efa640 1 -- 172.21.15.111:0/3159668722 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c0094d0 con 0x7fd03c06c8a0 2024-09-13T17:57:35.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd041efa640 1 --2- 172.21.15.111:0/3159668722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c06c8a0 0x7fd03c157490 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd03800c400 tx=0x7fd03800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:35.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd0327fc640 1 -- 172.21.15.111:0/3159668722 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0380052d0 con 0x7fd03c06c8a0 2024-09-13T17:57:35.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd0327fc640 1 -- 172.21.15.111:0/3159668722 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd038005430 con 0x7fd03c06c8a0 2024-09-13T17:57:35.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd04415e640 1 -- 172.21.15.111:0/3159668722 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03c1592d0 con 0x7fd03c06c8a0 2024-09-13T17:57:35.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd0327fc640 1 -- 172.21.15.111:0/3159668722 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038003830 con 0x7fd03c06c8a0 2024-09-13T17:57:35.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.476+0000 7fd04415e640 1 -- 172.21.15.111:0/3159668722 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03c153a80 con 0x7fd03c06c8a0 2024-09-13T17:57:35.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.477+0000 7fd04415e640 1 -- 172.21.15.111:0/3159668722 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03c159130 con 0x7fd03c06c8a0 2024-09-13T17:57:35.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.478+0000 7fd0327fc640 1 -- 172.21.15.111:0/3159668722 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd038003990 con 0x7fd03c06c8a0 2024-09-13T17:57:35.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.478+0000 7fd0327fc640 1 --2- 172.21.15.111:0/3159668722 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd02006ac00 0x7fd02006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.478+0000 7fd0327fc640 1 -- 172.21.15.111:0/3159668722 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd038088bc0 con 0x7fd03c06c8a0 2024-09-13T17:57:35.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.478+0000 7fd0416f9640 1 --2- 172.21.15.111:0/3159668722 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd02006ac00 0x7fd02006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.479+0000 7fd0416f9640 1 --2- 172.21.15.111:0/3159668722 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd02006ac00 0x7fd02006d0c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd02c034000 tx=0x7fd02c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.482+0000 7fd0327fc640 1 -- 172.21.15.111:0/3159668722 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fd03c159130 con 0x7fd03c06c8a0 2024-09-13T17:57:35.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.641+0000 7fd04415e640 1 -- 172.21.15.111:0/3159668722 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd03c159130 con 0x7fd03c06c8a0 2024-09-13T17:57:35.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.642+0000 7fd0327fc640 1 -- 172.21.15.111:0/3159668722 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd03c159130 con 0x7fd03c06c8a0 2024-09-13T17:57:35.644 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:57:35.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.644+0000 7fd04415e640 1 -- 172.21.15.111:0/3159668722 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd02006ac00 msgr2=0x7fd02006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.644+0000 7fd04415e640 1 --2- 172.21.15.111:0/3159668722 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd02006ac00 0x7fd02006d0c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd02c034000 tx=0x7fd02c02e000 comp rx=0 tx=0).stop 2024-09-13T17:57:35.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.644+0000 7fd04415e640 1 -- 172.21.15.111:0/3159668722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c06c8a0 msgr2=0x7fd03c157490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.644+0000 7fd04415e640 1 --2- 172.21.15.111:0/3159668722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c06c8a0 0x7fd03c157490 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd03800c400 tx=0x7fd03800c8c0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.644+0000 7fd04415e640 1 -- 172.21.15.111:0/3159668722 shutdown_connections 2024-09-13T17:57:35.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.644+0000 7fd04415e640 1 --2- 172.21.15.111:0/3159668722 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd02006ac00 0x7fd02006d0c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.644+0000 7fd04415e640 1 --2- 172.21.15.111:0/3159668722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd03c158b00 0x7fd03c0977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.644+0000 7fd04415e640 1 --2- 172.21.15.111:0/3159668722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd03c071b20 0x7fd03c1548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.644+0000 7fd04415e640 1 --2- 172.21.15.111:0/3159668722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c06c8a0 0x7fd03c157490 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.644+0000 7fd04415e640 1 -- 172.21.15.111:0/3159668722 >> 172.21.15.111:0/3159668722 conn(0x7fd03c06a0b0 msgr2=0x7fd03c154fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:35.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.644+0000 7fd04415e640 1 -- 172.21.15.111:0/3159668722 shutdown_connections 2024-09-13T17:57:35.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.644+0000 7fd04415e640 1 -- 172.21.15.111:0/3159668722 wait complete. 2024-09-13T17:57:35.656 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:57:35.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.755+0000 7f7fb798a640 1 Processor -- start 2024-09-13T17:57:35.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.755+0000 7f7fb798a640 1 -- start start 2024-09-13T17:57:35.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fb798a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7fb006a5e0 0x7f7fb006ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fb798a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb00cbfd0 con 0x7f7fb015c5c0 2024-09-13T17:57:35.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fb798a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb00cc2b0 con 0x7f7fb00b2590 2024-09-13T17:57:35.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fb798a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb00cc590 con 0x7f7fb006a5e0 2024-09-13T17:57:35.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fb5726640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f7fb015c5c0 0x7f7fb0069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49174/0 (socket says 172.21.15.111:49174) 2024-09-13T17:57:35.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fb5726640 1 -- 172.21.15.111:0/1784051144 learned_addr learned my addr 172.21.15.111:0/1784051144 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:35.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fb4f25640 1 --2- 172.21.15.111:0/1784051144 >> v2:172.21.15.188:3300/0 conn(0x7f7fb006a5e0 0x7f7fb006ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fa7fff640 1 -- 172.21.15.111:0/1784051144 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 338734769 0 0) 0x7f7fb00cbfd0 con 0x7f7fb015c5c0 2024-09-13T17:57:35.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fa7fff640 1 -- 172.21.15.111:0/1784051144 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f9c003520 con 0x7f7fb015c5c0 2024-09-13T17:57:35.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fa7fff640 1 -- 172.21.15.111:0/1784051144 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 856904717 0 0) 0x7f7f9c003520 con 0x7f7fb015c5c0 2024-09-13T17:57:35.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fa7fff640 1 -- 172.21.15.111:0/1784051144 >> v2:172.21.15.188:3300/0 conn(0x7f7fb006a5e0 msgr2=0x7f7fb006ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:35.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fa7fff640 1 --2- 172.21.15.111:0/1784051144 >> v2:172.21.15.188:3300/0 conn(0x7f7fb006a5e0 0x7f7fb006ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fa7fff640 1 -- 172.21.15.111:0/1784051144 >> v1:172.21.15.188:6789/0 conn(0x7f7fb00b2590 legacy=0x7f7fb00b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fa7fff640 1 -- 172.21.15.111:0/1784051144 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb00cc870 con 0x7f7fb015c5c0 2024-09-13T17:57:35.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fa7fff640 1 -- 172.21.15.111:0/1784051144 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f7fac001670 con 0x7f7fb015c5c0 2024-09-13T17:57:35.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fa7fff640 1 -- 172.21.15.111:0/1784051144 >> v1:172.21.15.111:6789/0 conn(0x7f7fb015c5c0 legacy=0x7f7fb0069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fa7fff640 1 --2- 172.21.15.111:0/1784051144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f9c003970 0x7f7f9c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fa7fff640 1 --2- 172.21.15.111:0/1784051144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f9c0047c0 0x7f7f9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fa7fff640 1 --2- 172.21.15.111:0/1784051144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9c005180 0x7f7f9c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fa7fff640 1 -- 172.21.15.111:0/1784051144 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb00cc590 con 0x7f7f9c0047c0 2024-09-13T17:57:35.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fa7fff640 1 -- 172.21.15.111:0/1784051144 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c00c020 con 0x7f7f9c003970 2024-09-13T17:57:35.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.756+0000 7f7fa7fff640 1 -- 172.21.15.111:0/1784051144 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c00c1e0 con 0x7f7f9c005180 2024-09-13T17:57:35.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.757+0000 7f7fb4f25640 1 --2- 172.21.15.111:0/1784051144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f9c0047c0 0x7f7f9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.757+0000 7f7fb5726640 1 --2- 172.21.15.111:0/1784051144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f9c003970 0x7f7f9c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.757+0000 7f7fb5f27640 1 --2- 172.21.15.111:0/1784051144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9c005180 0x7f7f9c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.757+0000 7f7fb4f25640 1 -- 172.21.15.111:0/1784051144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9c005180 msgr2=0x7f7f9c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:35.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.757+0000 7f7fb4f25640 1 --2- 172.21.15.111:0/1784051144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9c005180 0x7f7f9c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.757+0000 7f7fb4f25640 1 -- 172.21.15.111:0/1784051144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f9c003970 msgr2=0x7f7f9c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:35.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.757+0000 7f7fb4f25640 1 --2- 172.21.15.111:0/1784051144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f9c003970 0x7f7f9c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.757+0000 7f7fb4f25640 1 -- 172.21.15.111:0/1784051144 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c003520 con 0x7f7f9c0047c0 2024-09-13T17:57:35.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.757+0000 7f7fb5726640 1 --2- 172.21.15.111:0/1784051144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f9c003970 0x7f7f9c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:35.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.757+0000 7f7fb5f27640 1 -- 172.21.15.111:0/1784051144 reap_dead start 2024-09-13T17:57:35.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.757+0000 7f7fb4f25640 1 --2- 172.21.15.111:0/1784051144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f9c0047c0 0x7f7f9c004c40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009ac0 tx=0x7f7f9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ad93b4fec62390 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:35.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.758+0000 7f7fa7fff640 1 -- 172.21.15.111:0/1784051144 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0031400 con 0x7f7f9c0047c0 2024-09-13T17:57:35.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.758+0000 7f7fa7fff640 1 -- 172.21.15.111:0/1784051144 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa002d050 con 0x7f7f9c0047c0 2024-09-13T17:57:35.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.758+0000 7f7fa7fff640 1 -- 172.21.15.111:0/1784051144 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0031780 con 0x7f7f9c0047c0 2024-09-13T17:57:35.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.758+0000 7f7fb798a640 1 -- 172.21.15.111:0/1784051144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f9c0047c0 msgr2=0x7f7f9c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.758+0000 7f7fb798a640 1 --2- 172.21.15.111:0/1784051144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f9c0047c0 0x7f7f9c004c40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009ac0 tx=0x7f7f9c0057f0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.758+0000 7f7fb798a640 1 -- 172.21.15.111:0/1784051144 shutdown_connections 2024-09-13T17:57:35.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.758+0000 7f7fb798a640 1 --2- 172.21.15.111:0/1784051144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f9c0047c0 0x7f7f9c004c40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.758+0000 7f7fb798a640 1 -- 172.21.15.111:0/1784051144 >> 172.21.15.111:0/1784051144 conn(0x7f7fb00717e0 msgr2=0x7f7fb00989d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:35.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.758+0000 7f7fb798a640 1 -- 172.21.15.111:0/1784051144 shutdown_connections 2024-09-13T17:57:35.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.758+0000 7f7fb798a640 1 -- 172.21.15.111:0/1784051144 wait complete. 2024-09-13T17:57:35.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.759+0000 7f7fb798a640 1 Processor -- start 2024-09-13T17:57:35.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.759+0000 7f7fb798a640 1 -- start start 2024-09-13T17:57:35.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.760+0000 7f7fb798a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9c0047c0 0x7f7fb0153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.760+0000 7f7fb798a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fb015e570 0x7f7fb00b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.760+0000 7f7fb798a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb015e9f0 0x7f7fb00b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.760+0000 7f7fb798a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb0153790 con 0x7f7fb015e570 2024-09-13T17:57:35.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.760+0000 7f7fb798a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb015cca0 con 0x7f7fb015e9f0 2024-09-13T17:57:35.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.760+0000 7f7fb798a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb015cf80 con 0x7f7f9c0047c0 2024-09-13T17:57:35.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.760+0000 7f7fb4f25640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fb015e570 0x7f7fb00b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.760+0000 7f7fb4f25640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fb015e570 0x7f7fb00b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38968/0 (socket says 172.21.15.111:38968) 2024-09-13T17:57:35.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.760+0000 7f7fb4f25640 1 -- 172.21.15.111:0/4168063227 learned_addr learned my addr 172.21.15.111:0/4168063227 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:35.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.760+0000 7f7fb5726640 1 --2- 172.21.15.111:0/4168063227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9c0047c0 0x7f7fb0153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.760+0000 7f7fb5f27640 1 --2- 172.21.15.111:0/4168063227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb015e9f0 0x7f7fb00b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.761+0000 7f7fb4f25640 1 -- 172.21.15.111:0/4168063227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9c0047c0 msgr2=0x7f7fb0153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.761+0000 7f7fb4f25640 1 --2- 172.21.15.111:0/4168063227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9c0047c0 0x7f7fb0153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.761+0000 7f7fb4f25640 1 -- 172.21.15.111:0/4168063227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb015e9f0 msgr2=0x7f7fb00b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.761+0000 7f7fb4f25640 1 --2- 172.21.15.111:0/4168063227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb015e9f0 0x7f7fb00b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.761+0000 7f7fb4f25640 1 -- 172.21.15.111:0/4168063227 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb015d260 con 0x7f7fb015e570 2024-09-13T17:57:35.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.761+0000 7f7fb5f27640 1 --2- 172.21.15.111:0/4168063227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb015e9f0 0x7f7fb00b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:35.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.761+0000 7f7fb4f25640 1 --2- 172.21.15.111:0/4168063227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fb015e570 0x7f7fb00b3b60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7fa002aae0 tx=0x7f7fa0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:35.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.761+0000 7f7fa5ffb640 1 -- 172.21.15.111:0/4168063227 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0003e90 con 0x7f7fb015e570 2024-09-13T17:57:35.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.761+0000 7f7fa5ffb640 1 -- 172.21.15.111:0/4168063227 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa003b400 con 0x7f7fb015e570 2024-09-13T17:57:35.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.761+0000 7f7fa5ffb640 1 -- 172.21.15.111:0/4168063227 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa003b790 con 0x7f7fb015e570 2024-09-13T17:57:35.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.761+0000 7f7fb798a640 1 -- 172.21.15.111:0/4168063227 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb015d540 con 0x7f7fb015e570 2024-09-13T17:57:35.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.761+0000 7f7fb798a640 1 -- 172.21.15.111:0/4168063227 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb00ae490 con 0x7f7fb015e570 2024-09-13T17:57:35.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.762+0000 7f7fb798a640 1 -- 172.21.15.111:0/4168063227 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb00d01f0 con 0x7f7fb015e570 2024-09-13T17:57:35.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.762+0000 7f7fa5ffb640 1 -- 172.21.15.111:0/4168063227 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7fa003b8f0 con 0x7f7fb015e570 2024-09-13T17:57:35.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.763+0000 7f7fa5ffb640 1 --2- 172.21.15.111:0/4168063227 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7f9406ac30 0x7f7f9406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:35.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.763+0000 7f7fb5726640 1 --2- 172.21.15.111:0/4168063227 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7f9406ac30 0x7f7f9406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:35.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.763+0000 7f7fa5ffb640 1 -- 172.21.15.111:0/4168063227 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7fa00a9cc0 con 0x7f7fb015e570 2024-09-13T17:57:35.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.763+0000 7f7fb5726640 1 --2- 172.21.15.111:0/4168063227 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7f9406ac30 0x7f7f9406d0f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7fac001c90 tx=0x7f7fac007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:35.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.767+0000 7f7fa5ffb640 1 -- 172.21.15.111:0/4168063227 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f7fa0074c60 con 0x7f7fb015e570 2024-09-13T17:57:35.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.930+0000 7f7fb798a640 1 -- 172.21.15.111:0/4168063227 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7fb0071580 con 0x7f7fb015e570 2024-09-13T17:57:35.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.931+0000 7f7fa5ffb640 1 -- 172.21.15.111:0/4168063227 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7fa00480a0 con 0x7f7fb015e570 2024-09-13T17:57:35.933 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:57:35.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.933+0000 7f7fb798a640 1 -- 172.21.15.111:0/4168063227 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7f9406ac30 msgr2=0x7f7f9406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.933+0000 7f7fb798a640 1 --2- 172.21.15.111:0/4168063227 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7f9406ac30 0x7f7f9406d0f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7fac001c90 tx=0x7f7fac007040 comp rx=0 tx=0).stop 2024-09-13T17:57:35.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.933+0000 7f7fb798a640 1 -- 172.21.15.111:0/4168063227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fb015e570 msgr2=0x7f7fb00b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:35.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.933+0000 7f7fb798a640 1 --2- 172.21.15.111:0/4168063227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fb015e570 0x7f7fb00b3b60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7fa002aae0 tx=0x7f7fa0004730 comp rx=0 tx=0).stop 2024-09-13T17:57:35.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.933+0000 7f7fb798a640 1 -- 172.21.15.111:0/4168063227 shutdown_connections 2024-09-13T17:57:35.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.933+0000 7f7fb798a640 1 --2- 172.21.15.111:0/4168063227 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7f9406ac30 0x7f7f9406d0f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.933+0000 7f7fb798a640 1 --2- 172.21.15.111:0/4168063227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb015e9f0 0x7f7fb00b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.933+0000 7f7fb798a640 1 --2- 172.21.15.111:0/4168063227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fb015e570 0x7f7fb00b3b60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.933+0000 7f7fb798a640 1 --2- 172.21.15.111:0/4168063227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9c0047c0 0x7f7fb0153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:35.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.933+0000 7f7fb798a640 1 -- 172.21.15.111:0/4168063227 >> 172.21.15.111:0/4168063227 conn(0x7f7fb00717e0 msgr2=0x7f7fb00cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:35.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.933+0000 7f7fb798a640 1 -- 172.21.15.111:0/4168063227 shutdown_connections 2024-09-13T17:57:35.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:35.933+0000 7f7fb798a640 1 -- 172.21.15.111:0/4168063227 wait complete. 2024-09-13T17:57:35.945 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:57:36.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.041+0000 7fabe6a41640 1 Processor -- start 2024-09-13T17:57:36.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.042+0000 7fabe6a41640 1 -- start start 2024-09-13T17:57:36.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.042+0000 7fabe6a41640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe015c8c0 0x7fabe00989d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.042+0000 7fabe6a41640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fabe00af8b0 0x7fabe0098f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.042+0000 7fabe6a41640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fabe00b2aa0 0x7fabe00ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.042+0000 7fabe6a41640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe00cfb10 con 0x7fabe00b2aa0 2024-09-13T17:57:36.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.042+0000 7fabe6a41640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe00caae0 con 0x7fabe00af8b0 2024-09-13T17:57:36.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.042+0000 7fabe6a41640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe015cca0 con 0x7fabe015c8c0 2024-09-13T17:57:36.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.042+0000 7fabe4fde640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fabe00b2aa0 0x7fabe00ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.043+0000 7fabe4fde640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fabe00b2aa0 0x7fabe00ce6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38980/0 (socket says 172.21.15.111:38980) 2024-09-13T17:57:36.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.043+0000 7fabe4fde640 1 -- 172.21.15.111:0/2791873113 learned_addr learned my addr 172.21.15.111:0/2791873113 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:36.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.043+0000 7fabdffff640 1 --2- 172.21.15.111:0/2791873113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe015c8c0 0x7fabe00989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.043+0000 7fabd7fff640 1 --2- 172.21.15.111:0/2791873113 >> v2:172.21.15.188:3300/0 conn(0x7fabe00af8b0 0x7fabe0098f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.043+0000 7fabe4fde640 1 -- 172.21.15.111:0/2791873113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe015c8c0 msgr2=0x7fabe00989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:36.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.043+0000 7fabe4fde640 1 --2- 172.21.15.111:0/2791873113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe015c8c0 0x7fabe00989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.043+0000 7fabe4fde640 1 -- 172.21.15.111:0/2791873113 >> v2:172.21.15.188:3300/0 conn(0x7fabe00af8b0 msgr2=0x7fabe0098f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:36.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.043+0000 7fabe4fde640 1 --2- 172.21.15.111:0/2791873113 >> v2:172.21.15.188:3300/0 conn(0x7fabe00af8b0 0x7fabe0098f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.043+0000 7fabe4fde640 1 -- 172.21.15.111:0/2791873113 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe00996c0 con 0x7fabe00b2aa0 2024-09-13T17:57:36.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.043+0000 7fabd7fff640 1 --2- 172.21.15.111:0/2791873113 >> v2:172.21.15.188:3300/0 conn(0x7fabe00af8b0 0x7fabe0098f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:36.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.044+0000 7fabe4fde640 1 --2- 172.21.15.111:0/2791873113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabe00b2aa0 0x7fabe00ce6d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fabe0071f30 tx=0x7fabd802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410bc875d4fd5bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:36.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.044+0000 7fabdf7fe640 1 -- 172.21.15.111:0/2791873113 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8031600 con 0x7fabe00b2aa0 2024-09-13T17:57:36.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.044+0000 7fabdf7fe640 1 -- 172.21.15.111:0/2791873113 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd802d050 con 0x7fabe00b2aa0 2024-09-13T17:57:36.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.044+0000 7fabdf7fe640 1 -- 172.21.15.111:0/2791873113 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8031980 con 0x7fabe00b2aa0 2024-09-13T17:57:36.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.044+0000 7fabe6a41640 1 -- 172.21.15.111:0/2791873113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabe00b2aa0 msgr2=0x7fabe00ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.044+0000 7fabe6a41640 1 --2- 172.21.15.111:0/2791873113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabe00b2aa0 0x7fabe00ce6d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fabe0071f30 tx=0x7fabd802a610 comp rx=0 tx=0).stop 2024-09-13T17:57:36.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.044+0000 7fabe6a41640 1 -- 172.21.15.111:0/2791873113 shutdown_connections 2024-09-13T17:57:36.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.044+0000 7fabe6a41640 1 --2- 172.21.15.111:0/2791873113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabe00b2aa0 0x7fabe00ce6d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.044+0000 7fabe6a41640 1 --2- 172.21.15.111:0/2791873113 >> v2:172.21.15.188:3300/0 conn(0x7fabe00af8b0 0x7fabe0098f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.044+0000 7fabe6a41640 1 --2- 172.21.15.111:0/2791873113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe015c8c0 0x7fabe00989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.044+0000 7fabe6a41640 1 -- 172.21.15.111:0/2791873113 >> 172.21.15.111:0/2791873113 conn(0x7fabe00b1f80 msgr2=0x7fabe00b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:36.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.044+0000 7fabe6a41640 1 -- 172.21.15.111:0/2791873113 shutdown_connections 2024-09-13T17:57:36.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.044+0000 7fabe6a41640 1 -- 172.21.15.111:0/2791873113 wait complete. 2024-09-13T17:57:36.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.046+0000 7fabe6a41640 1 Processor -- start 2024-09-13T17:57:36.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.046+0000 7fabe6a41640 1 -- start start 2024-09-13T17:57:36.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.046+0000 7fabe6a41640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe00af8b0 0x7fabe014f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.046+0000 7fabe6a41640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabe00b2aa0 0x7fabe00b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.046+0000 7fabe6a41640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabe015c8c0 0x7fabe00c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.046+0000 7fabe6a41640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe00c84c0 con 0x7fabe015c8c0 2024-09-13T17:57:36.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.046+0000 7fabe6a41640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe00c78b0 con 0x7fabe00b2aa0 2024-09-13T17:57:36.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.046+0000 7fabe6a41640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe00c7b90 con 0x7fabe00af8b0 2024-09-13T17:57:36.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.046+0000 7fabe4fde640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabe015c8c0 0x7fabe00c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.046+0000 7fabe4fde640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabe015c8c0 0x7fabe00c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38986/0 (socket says 172.21.15.111:38986) 2024-09-13T17:57:36.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.046+0000 7fabe4fde640 1 -- 172.21.15.111:0/2166277452 learned_addr learned my addr 172.21.15.111:0/2166277452 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:36.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.046+0000 7fabdffff640 1 --2- 172.21.15.111:0/2166277452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe00af8b0 0x7fabe014f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.046+0000 7fabd7fff640 1 --2- 172.21.15.111:0/2166277452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabe00b2aa0 0x7fabe00b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.047+0000 7fabe4fde640 1 -- 172.21.15.111:0/2166277452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe00af8b0 msgr2=0x7fabe014f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.047+0000 7fabe4fde640 1 --2- 172.21.15.111:0/2166277452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe00af8b0 0x7fabe014f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.047+0000 7fabe4fde640 1 -- 172.21.15.111:0/2166277452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabe00b2aa0 msgr2=0x7fabe00b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.047+0000 7fabe4fde640 1 --2- 172.21.15.111:0/2166277452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabe00b2aa0 0x7fabe00b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.047+0000 7fabe4fde640 1 -- 172.21.15.111:0/2166277452 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd8009b30 con 0x7fabe015c8c0 2024-09-13T17:57:36.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.047+0000 7fabe4fde640 1 --2- 172.21.15.111:0/2166277452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabe015c8c0 0x7fabe00c8950 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fabd8034040 tx=0x7fabd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:36.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.047+0000 7fabdd7fa640 1 -- 172.21.15.111:0/2166277452 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd803d860 con 0x7fabe015c8c0 2024-09-13T17:57:36.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.047+0000 7fabdd7fa640 1 -- 172.21.15.111:0/2166277452 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd803d9e0 con 0x7fabe015c8c0 2024-09-13T17:57:36.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.047+0000 7fabdd7fa640 1 -- 172.21.15.111:0/2166277452 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd803dd90 con 0x7fabe015c8c0 2024-09-13T17:57:36.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.047+0000 7fabe6a41640 1 -- 172.21.15.111:0/2166277452 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe00c7e70 con 0x7fabe015c8c0 2024-09-13T17:57:36.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.047+0000 7fabe6a41640 1 -- 172.21.15.111:0/2166277452 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe014fdf0 con 0x7fabe015c8c0 2024-09-13T17:57:36.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.048+0000 7fabe6a41640 1 -- 172.21.15.111:0/2166277452 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe00c8670 con 0x7fabe015c8c0 2024-09-13T17:57:36.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.048+0000 7fabdd7fa640 1 -- 172.21.15.111:0/2166277452 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fabd803b400 con 0x7fabe015c8c0 2024-09-13T17:57:36.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.049+0000 7fabdd7fa640 1 --2- 172.21.15.111:0/2166277452 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fabc006ac00 0x7fabc006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.049+0000 7fabdd7fa640 1 -- 172.21.15.111:0/2166277452 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fabd80aa070 con 0x7fabe015c8c0 2024-09-13T17:57:36.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.049+0000 7fabdffff640 1 --2- 172.21.15.111:0/2166277452 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fabc006ac00 0x7fabc006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.049+0000 7fabdffff640 1 --2- 172.21.15.111:0/2166277452 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fabc006ac00 0x7fabc006d0c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fabd00079a0 tx=0x7fabd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:36.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.053+0000 7fabdd7fa640 1 -- 172.21.15.111:0/2166277452 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fabd80750a0 con 0x7fabe015c8c0 2024-09-13T17:57:36.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.227+0000 7fabe6a41640 1 -- 172.21.15.111:0/2166277452 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fabe00989d0 con 0x7fabe015c8c0 2024-09-13T17:57:36.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.227+0000 7fabdd7fa640 1 -- 172.21.15.111:0/2166277452 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fabd8009510 con 0x7fabe015c8c0 2024-09-13T17:57:36.229 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2024-09-13T17:57:36.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.229+0000 7fabe6a41640 1 -- 172.21.15.111:0/2166277452 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fabc006ac00 msgr2=0x7fabc006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.229+0000 7fabe6a41640 1 --2- 172.21.15.111:0/2166277452 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fabc006ac00 0x7fabc006d0c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fabd00079a0 tx=0x7fabd0008040 comp rx=0 tx=0).stop 2024-09-13T17:57:36.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.229+0000 7fabe6a41640 1 -- 172.21.15.111:0/2166277452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabe015c8c0 msgr2=0x7fabe00c8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.229+0000 7fabe6a41640 1 --2- 172.21.15.111:0/2166277452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabe015c8c0 0x7fabe00c8950 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fabd8034040 tx=0x7fabd8004a40 comp rx=0 tx=0).stop 2024-09-13T17:57:36.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.229+0000 7fabe6a41640 1 -- 172.21.15.111:0/2166277452 shutdown_connections 2024-09-13T17:57:36.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.229+0000 7fabe6a41640 1 --2- 172.21.15.111:0/2166277452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabe015c8c0 0x7fabe00c8950 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.229+0000 7fabe6a41640 1 --2- 172.21.15.111:0/2166277452 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fabc006ac00 0x7fabc006d0c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.229+0000 7fabe6a41640 1 --2- 172.21.15.111:0/2166277452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabe00b2aa0 0x7fabe00b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.229+0000 7fabe6a41640 1 --2- 172.21.15.111:0/2166277452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe00af8b0 0x7fabe014f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.229+0000 7fabe6a41640 1 -- 172.21.15.111:0/2166277452 >> 172.21.15.111:0/2166277452 conn(0x7fabe00b1f80 msgr2=0x7fabe00b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:36.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.229+0000 7fabe6a41640 1 -- 172.21.15.111:0/2166277452 shutdown_connections 2024-09-13T17:57:36.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.229+0000 7fabe6a41640 1 -- 172.21.15.111:0/2166277452 wait complete. 2024-09-13T17:57:36.241 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:57:36.241 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:57:36.257 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:36.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.394+0000 7f8e041ce640 1 Processor -- start 2024-09-13T17:57:36.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.394+0000 7f8e041ce640 1 -- start start 2024-09-13T17:57:36.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.395+0000 7f8e041ce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8dfc071d30 0x7f8dfc070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.395+0000 7f8e041ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8dfc0c8920 0x7f8dfc0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.395+0000 7f8e041ce640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dfc0abdf0 con 0x7f8dfc072110 2024-09-13T17:57:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.395+0000 7f8e041ce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dfc0c9310 con 0x7f8dfc071d30 2024-09-13T17:57:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.395+0000 7f8e041ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc0b2280 con 0x7f8dfc0c8920 2024-09-13T17:57:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.395+0000 7f8e01f6a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8dfc071d30 0x7f8dfc070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.395+0000 7f8e0276b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8dfc0c8920 0x7f8dfc0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.395+0000 7f8e01769640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f8dfc072110 0x7f8dfc0b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:45356/0 (socket says 172.21.15.111:45356) 2024-09-13T17:57:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.395+0000 7f8e01769640 1 -- 172.21.15.111:0/2432815992 learned_addr learned my addr 172.21.15.111:0/2432815992 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.395+0000 7f8e01f6a640 1 -- 172.21.15.111:0/2432815992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8dfc0c8920 msgr2=0x7f8dfc0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.395+0000 7f8e01f6a640 1 --2- 172.21.15.111:0/2432815992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8dfc0c8920 0x7f8dfc0c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.395+0000 7f8e01f6a640 1 -- 172.21.15.111:0/2432815992 >> v1:172.21.15.188:6789/0 conn(0x7f8dfc072110 legacy=0x7f8dfc0b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.395+0000 7f8e01f6a640 1 -- 172.21.15.111:0/2432815992 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc0ab1e0 con 0x7f8dfc071d30 2024-09-13T17:57:36.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.396+0000 7f8e01f6a640 1 --2- 172.21.15.111:0/2432815992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dfc071d30 0x7f8dfc070190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8df80097a0 tx=0x7f8df802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a95334d68619c461 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:36.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.396+0000 7f8e00f68640 1 -- 172.21.15.111:0/2432815992 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8032440 con 0x7f8dfc071d30 2024-09-13T17:57:36.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.396+0000 7f8e00f68640 1 -- 172.21.15.111:0/2432815992 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df80325a0 con 0x7f8dfc071d30 2024-09-13T17:57:36.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.396+0000 7f8e00f68640 1 -- 172.21.15.111:0/2432815992 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8032930 con 0x7f8dfc071d30 2024-09-13T17:57:36.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.397+0000 7f8e041ce640 1 -- 172.21.15.111:0/2432815992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dfc071d30 msgr2=0x7f8dfc070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.397+0000 7f8e041ce640 1 --2- 172.21.15.111:0/2432815992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dfc071d30 0x7f8dfc070190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8df80097a0 tx=0x7f8df802a180 comp rx=0 tx=0).stop 2024-09-13T17:57:36.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.397+0000 7f8e041ce640 1 -- 172.21.15.111:0/2432815992 shutdown_connections 2024-09-13T17:57:36.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.397+0000 7f8e041ce640 1 --2- 172.21.15.111:0/2432815992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8dfc0c8920 0x7f8dfc0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.397+0000 7f8e041ce640 1 --2- 172.21.15.111:0/2432815992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dfc071d30 0x7f8dfc070190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.397+0000 7f8e041ce640 1 -- 172.21.15.111:0/2432815992 >> 172.21.15.111:0/2432815992 conn(0x7f8dfc069ed0 msgr2=0x7f8dfc06a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:36.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.397+0000 7f8e041ce640 1 -- 172.21.15.111:0/2432815992 shutdown_connections 2024-09-13T17:57:36.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.397+0000 7f8e041ce640 1 -- 172.21.15.111:0/2432815992 wait complete. 2024-09-13T17:57:36.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.398+0000 7f8e041ce640 1 Processor -- start 2024-09-13T17:57:36.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.398+0000 7f8e041ce640 1 -- start start 2024-09-13T17:57:36.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e041ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dfc071d30 0x7f8dfc06d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e041ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8dfc072110 0x7f8dfc15ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e041ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8dfc0c8920 0x7f8dfc15b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e041ce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc154a00 con 0x7f8dfc0c8920 2024-09-13T17:57:36.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e041ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc158220 con 0x7f8dfc071d30 2024-09-13T17:57:36.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e041ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc1f2340 con 0x7f8dfc072110 2024-09-13T17:57:36.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e0276b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8dfc0c8920 0x7f8dfc15b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e0276b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8dfc0c8920 0x7f8dfc15b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38990/0 (socket says 172.21.15.111:38990) 2024-09-13T17:57:36.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e0276b640 1 -- 172.21.15.111:0/2597539135 learned_addr learned my addr 172.21.15.111:0/2597539135 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:36.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e01f6a640 1 --2- 172.21.15.111:0/2597539135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dfc071d30 0x7f8dfc06d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e01769640 1 --2- 172.21.15.111:0/2597539135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8dfc072110 0x7f8dfc15ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e0276b640 1 -- 172.21.15.111:0/2597539135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8dfc072110 msgr2=0x7f8dfc15ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e0276b640 1 --2- 172.21.15.111:0/2597539135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8dfc072110 0x7f8dfc15ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e0276b640 1 -- 172.21.15.111:0/2597539135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dfc071d30 msgr2=0x7f8dfc06d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e0276b640 1 --2- 172.21.15.111:0/2597539135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dfc071d30 0x7f8dfc06d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e0276b640 1 -- 172.21.15.111:0/2597539135 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df8009400 con 0x7f8dfc0c8920 2024-09-13T17:57:36.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e01f6a640 1 --2- 172.21.15.111:0/2597539135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dfc071d30 0x7f8dfc06d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:36.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.399+0000 7f8e0276b640 1 --2- 172.21.15.111:0/2597539135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8dfc0c8920 0x7f8dfc15b070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8df400efc0 tx=0x7f8df400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:36.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.400+0000 7f8df27fc640 1 -- 172.21.15.111:0/2597539135 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4009ba0 con 0x7f8dfc0c8920 2024-09-13T17:57:36.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.400+0000 7f8df27fc640 1 -- 172.21.15.111:0/2597539135 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df400f040 con 0x7f8dfc0c8920 2024-09-13T17:57:36.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.400+0000 7f8e041ce640 1 -- 172.21.15.111:0/2597539135 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dfc1f25a0 con 0x7f8dfc0c8920 2024-09-13T17:57:36.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.400+0000 7f8df27fc640 1 -- 172.21.15.111:0/2597539135 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df40146c0 con 0x7f8dfc0c8920 2024-09-13T17:57:36.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.400+0000 7f8e041ce640 1 -- 172.21.15.111:0/2597539135 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dfc1f2b20 con 0x7f8dfc0c8920 2024-09-13T17:57:36.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.401+0000 7f8e041ce640 1 -- 172.21.15.111:0/2597539135 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dfc070190 con 0x7f8dfc0c8920 2024-09-13T17:57:36.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.401+0000 7f8df27fc640 1 -- 172.21.15.111:0/2597539135 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8df4014820 con 0x7f8dfc0c8920 2024-09-13T17:57:36.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.401+0000 7f8df27fc640 1 --2- 172.21.15.111:0/2597539135 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8de006ac00 0x7f8de006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.402+0000 7f8df27fc640 1 -- 172.21.15.111:0/2597539135 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8df408a1e0 con 0x7f8dfc0c8920 2024-09-13T17:57:36.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.402+0000 7f8e01f6a640 1 --2- 172.21.15.111:0/2597539135 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8de006ac00 0x7f8de006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.402+0000 7f8e01f6a640 1 --2- 172.21.15.111:0/2597539135 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8de006ac00 0x7f8de006d0c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8df8034000 tx=0x7f8df802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:36.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.406+0000 7f8df27fc640 1 -- 172.21.15.111:0/2597539135 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f8df4055040 con 0x7f8dfc0c8920 2024-09-13T17:57:36.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.567+0000 7f8e041ce640 1 -- 172.21.15.111:0/2597539135 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8dfc15cb20 con 0x7f8dfc0c8920 2024-09-13T17:57:36.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.569+0000 7f8df27fc640 1 -- 172.21.15.111:0/2597539135 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f8df4018090 con 0x7f8dfc0c8920 2024-09-13T17:57:36.571 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:36.571 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:30.065235+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:36.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.571+0000 7f8e041ce640 1 -- 172.21.15.111:0/2597539135 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8de006ac00 msgr2=0x7f8de006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.571+0000 7f8e041ce640 1 --2- 172.21.15.111:0/2597539135 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8de006ac00 0x7f8de006d0c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8df8034000 tx=0x7f8df802e000 comp rx=0 tx=0).stop 2024-09-13T17:57:36.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.571+0000 7f8e041ce640 1 -- 172.21.15.111:0/2597539135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8dfc0c8920 msgr2=0x7f8dfc15b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.571+0000 7f8e041ce640 1 --2- 172.21.15.111:0/2597539135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8dfc0c8920 0x7f8dfc15b070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8df400efc0 tx=0x7f8df400c490 comp rx=0 tx=0).stop 2024-09-13T17:57:36.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.571+0000 7f8e041ce640 1 -- 172.21.15.111:0/2597539135 shutdown_connections 2024-09-13T17:57:36.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.571+0000 7f8e041ce640 1 --2- 172.21.15.111:0/2597539135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8dfc0c8920 0x7f8dfc15b070 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.571+0000 7f8e041ce640 1 --2- 172.21.15.111:0/2597539135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8dfc072110 0x7f8dfc15ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.571+0000 7f8e041ce640 1 --2- 172.21.15.111:0/2597539135 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8de006ac00 0x7f8de006d0c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.571+0000 7f8e041ce640 1 --2- 172.21.15.111:0/2597539135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dfc071d30 0x7f8dfc06d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.571+0000 7f8e041ce640 1 -- 172.21.15.111:0/2597539135 >> 172.21.15.111:0/2597539135 conn(0x7f8dfc069ed0 msgr2=0x7f8dfc0aefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:36.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.571+0000 7f8e041ce640 1 -- 172.21.15.111:0/2597539135 shutdown_connections 2024-09-13T17:57:36.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.571+0000 7f8e041ce640 1 -- 172.21.15.111:0/2597539135 wait complete. 2024-09-13T17:57:36.583 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:57:36.584 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:57:36.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.682+0000 7f13b140d640 1 Processor -- start 2024-09-13T17:57:36.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.682+0000 7f13b140d640 1 -- start start 2024-09-13T17:57:36.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.683+0000 7f13b140d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f13ac15a7f0 0x7f13ac0ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.683+0000 7f13b140d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f13ac0989d0 0x7f13ac098e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.683+0000 7f13b140d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13ac0993a0 0x7f13ac0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.683+0000 7f13b140d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ac0cc300 con 0x7f13ac0989d0 2024-09-13T17:57:36.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.683+0000 7f13b140d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ac0b3d20 con 0x7f13ac15a7f0 2024-09-13T17:57:36.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.683+0000 7f13b140d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac099810 con 0x7f13ac0993a0 2024-09-13T17:57:36.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.683+0000 7f13aa7fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f13ac0989d0 0x7f13ac098e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.683+0000 7f13aa7fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f13ac0989d0 0x7f13ac098e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38994/0 (socket says 172.21.15.111:38994) 2024-09-13T17:57:36.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.683+0000 7f13aa7fc640 1 -- 172.21.15.111:0/468949167 learned_addr learned my addr 172.21.15.111:0/468949167 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:36.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.683+0000 7f13aaffd640 1 --2- 172.21.15.111:0/468949167 >> v2:172.21.15.188:3300/0 conn(0x7f13ac15a7f0 0x7f13ac0ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.683+0000 7f13ab7fe640 1 --2- 172.21.15.111:0/468949167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13ac0993a0 0x7f13ac0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.684+0000 7f13aa7fc640 1 -- 172.21.15.111:0/468949167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13ac0993a0 msgr2=0x7f13ac0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:36.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.684+0000 7f13aa7fc640 1 --2- 172.21.15.111:0/468949167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13ac0993a0 0x7f13ac0caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.684+0000 7f13aa7fc640 1 -- 172.21.15.111:0/468949167 >> v2:172.21.15.188:3300/0 conn(0x7f13ac15a7f0 msgr2=0x7f13ac0ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:36.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.684+0000 7f13aa7fc640 1 --2- 172.21.15.111:0/468949167 >> v2:172.21.15.188:3300/0 conn(0x7f13ac15a7f0 0x7f13ac0ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.684+0000 7f13aa7fc640 1 -- 172.21.15.111:0/468949167 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac0b33f0 con 0x7f13ac0989d0 2024-09-13T17:57:36.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.684+0000 7f13ab7fe640 1 --2- 172.21.15.111:0/468949167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13ac0993a0 0x7f13ac0caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:36.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.684+0000 7f13aa7fc640 1 --2- 172.21.15.111:0/468949167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13ac0989d0 0x7f13ac098e60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f13ac071930 tx=0x7f139402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2199693b373241d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:36.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.684+0000 7f13a9ffb640 1 -- 172.21.15.111:0/468949167 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139402ae00 con 0x7f13ac0989d0 2024-09-13T17:57:36.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.684+0000 7f13a9ffb640 1 -- 172.21.15.111:0/468949167 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1394032440 con 0x7f13ac0989d0 2024-09-13T17:57:36.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.684+0000 7f13a9ffb640 1 -- 172.21.15.111:0/468949167 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13940327d0 con 0x7f13ac0989d0 2024-09-13T17:57:36.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.685+0000 7f13b140d640 1 -- 172.21.15.111:0/468949167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13ac0989d0 msgr2=0x7f13ac098e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.685+0000 7f13b140d640 1 --2- 172.21.15.111:0/468949167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13ac0989d0 0x7f13ac098e60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f13ac071930 tx=0x7f139402a0b0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.685+0000 7f13b140d640 1 -- 172.21.15.111:0/468949167 shutdown_connections 2024-09-13T17:57:36.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.685+0000 7f13b140d640 1 --2- 172.21.15.111:0/468949167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13ac0993a0 0x7f13ac0caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.685+0000 7f13b140d640 1 --2- 172.21.15.111:0/468949167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13ac0989d0 0x7f13ac098e60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.685+0000 7f13b140d640 1 --2- 172.21.15.111:0/468949167 >> v2:172.21.15.188:3300/0 conn(0x7f13ac15a7f0 0x7f13ac0ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.685+0000 7f13b140d640 1 -- 172.21.15.111:0/468949167 >> 172.21.15.111:0/468949167 conn(0x7f13ac0b1f60 msgr2=0x7f13ac0b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:36.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.685+0000 7f13b140d640 1 -- 172.21.15.111:0/468949167 shutdown_connections 2024-09-13T17:57:36.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.685+0000 7f13b140d640 1 -- 172.21.15.111:0/468949167 wait complete. 2024-09-13T17:57:36.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.686+0000 7f13b140d640 1 Processor -- start 2024-09-13T17:57:36.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.686+0000 7f13b140d640 1 -- start start 2024-09-13T17:57:36.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.687+0000 7f13b140d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13ac0989d0 0x7f13ac15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.687+0000 7f13b140d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13ac0993a0 0x7f13ac1566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.687+0000 7f13b140d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13ac15a7f0 0x7f13ac15cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.687+0000 7f13b140d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac15b110 con 0x7f13ac0989d0 2024-09-13T17:57:36.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.687+0000 7f13b140d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac0c8270 con 0x7f13ac15a7f0 2024-09-13T17:57:36.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.687+0000 7f13b140d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac156c10 con 0x7f13ac0993a0 2024-09-13T17:57:36.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.687+0000 7f13aaffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13ac0989d0 0x7f13ac15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.687+0000 7f13aaffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13ac0989d0 0x7f13ac15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39010/0 (socket says 172.21.15.111:39010) 2024-09-13T17:57:36.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.687+0000 7f13aaffd640 1 -- 172.21.15.111:0/1387490979 learned_addr learned my addr 172.21.15.111:0/1387490979 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:36.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.687+0000 7f13aa7fc640 1 --2- 172.21.15.111:0/1387490979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13ac0993a0 0x7f13ac1566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.687+0000 7f13ab7fe640 1 --2- 172.21.15.111:0/1387490979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13ac15a7f0 0x7f13ac15cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.687+0000 7f13aaffd640 1 -- 172.21.15.111:0/1387490979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13ac0993a0 msgr2=0x7f13ac1566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.687+0000 7f13aaffd640 1 --2- 172.21.15.111:0/1387490979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13ac0993a0 0x7f13ac1566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.687+0000 7f13aaffd640 1 -- 172.21.15.111:0/1387490979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13ac15a7f0 msgr2=0x7f13ac15cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.687+0000 7f13aaffd640 1 --2- 172.21.15.111:0/1387490979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13ac15a7f0 0x7f13ac15cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.687+0000 7f13aaffd640 1 -- 172.21.15.111:0/1387490979 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13940094d0 con 0x7f13ac0989d0 2024-09-13T17:57:36.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.688+0000 7f13aaffd640 1 --2- 172.21.15.111:0/1387490979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13ac0989d0 0x7f13ac15abd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f139c007ef0 tx=0x7f139c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:36.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.688+0000 7f13a37fe640 1 -- 172.21.15.111:0/1387490979 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c014620 con 0x7f13ac0989d0 2024-09-13T17:57:36.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.688+0000 7f13a37fe640 1 -- 172.21.15.111:0/1387490979 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139c014780 con 0x7f13ac0989d0 2024-09-13T17:57:36.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.688+0000 7f13b140d640 1 -- 172.21.15.111:0/1387490979 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ac15b900 con 0x7f13ac0989d0 2024-09-13T17:57:36.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.688+0000 7f13a37fe640 1 -- 172.21.15.111:0/1387490979 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c014b30 con 0x7f13ac0989d0 2024-09-13T17:57:36.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.688+0000 7f13b140d640 1 -- 172.21.15.111:0/1387490979 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ac15b620 con 0x7f13ac0989d0 2024-09-13T17:57:36.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.689+0000 7f13b140d640 1 -- 172.21.15.111:0/1387490979 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ac098e60 con 0x7f13ac0989d0 2024-09-13T17:57:36.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.689+0000 7f13a37fe640 1 -- 172.21.15.111:0/1387490979 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f139c014c90 con 0x7f13ac0989d0 2024-09-13T17:57:36.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.690+0000 7f13a37fe640 1 --2- 172.21.15.111:0/1387490979 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f138c06ac00 0x7f138c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.690+0000 7f13a37fe640 1 -- 172.21.15.111:0/1387490979 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f139c089ec0 con 0x7f13ac0989d0 2024-09-13T17:57:36.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.690+0000 7f13aa7fc640 1 --2- 172.21.15.111:0/1387490979 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f138c06ac00 0x7f138c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.690+0000 7f13aa7fc640 1 --2- 172.21.15.111:0/1387490979 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f138c06ac00 0x7f138c06d0c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1394034000 tx=0x7f139402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:36.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.694+0000 7f13a37fe640 1 -- 172.21.15.111:0/1387490979 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f139c054dd0 con 0x7f13ac0989d0 2024-09-13T17:57:36.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.853+0000 7f13b140d640 1 -- 172.21.15.111:0/1387490979 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13ac0ca7a0 con 0x7f13ac0989d0 2024-09-13T17:57:36.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.854+0000 7f13a37fe640 1 -- 172.21.15.111:0/1387490979 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f139c0583f0 con 0x7f13ac0989d0 2024-09-13T17:57:36.856 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:57:36.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.856+0000 7f13b140d640 1 -- 172.21.15.111:0/1387490979 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f138c06ac00 msgr2=0x7f138c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.856+0000 7f13b140d640 1 --2- 172.21.15.111:0/1387490979 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f138c06ac00 0x7f138c06d0c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1394034000 tx=0x7f139402e000 comp rx=0 tx=0).stop 2024-09-13T17:57:36.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.856+0000 7f13b140d640 1 -- 172.21.15.111:0/1387490979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13ac0989d0 msgr2=0x7f13ac15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.856+0000 7f13b140d640 1 --2- 172.21.15.111:0/1387490979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13ac0989d0 0x7f13ac15abd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f139c007ef0 tx=0x7f139c00e4b0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.856+0000 7f13b140d640 1 -- 172.21.15.111:0/1387490979 shutdown_connections 2024-09-13T17:57:36.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.856+0000 7f13b140d640 1 --2- 172.21.15.111:0/1387490979 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f138c06ac00 0x7f138c06d0c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.856+0000 7f13b140d640 1 --2- 172.21.15.111:0/1387490979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13ac15a7f0 0x7f13ac15cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.856+0000 7f13b140d640 1 --2- 172.21.15.111:0/1387490979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13ac0993a0 0x7f13ac1566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.856+0000 7f13b140d640 1 --2- 172.21.15.111:0/1387490979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13ac0989d0 0x7f13ac15abd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.856+0000 7f13b140d640 1 -- 172.21.15.111:0/1387490979 >> 172.21.15.111:0/1387490979 conn(0x7f13ac0b1f60 msgr2=0x7f13ac0b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:36.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.856+0000 7f13b140d640 1 -- 172.21.15.111:0/1387490979 shutdown_connections 2024-09-13T17:57:36.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.856+0000 7f13b140d640 1 -- 172.21.15.111:0/1387490979 wait complete. 2024-09-13T17:57:36.868 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:57:36.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.967+0000 7fed9a4e9640 1 Processor -- start 2024-09-13T17:57:36.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.967+0000 7fed9a4e9640 1 -- start start 2024-09-13T17:57:36.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.968+0000 7fed9a4e9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fed9406c8a0 0x7fed940b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.968+0000 7fed9a4e9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed940b2d10 con 0x7fed94158b00 2024-09-13T17:57:36.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.968+0000 7fed9a4e9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed940b2ff0 con 0x7fed940b27b0 2024-09-13T17:57:36.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.968+0000 7fed9a4e9640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed940cec00 con 0x7fed9406c8a0 2024-09-13T17:57:36.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.968+0000 7fed8bfff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fed9406c8a0 0x7fed940b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.968+0000 7fed93fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fed94158b00 0x7fed9406c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49180/0 (socket says 172.21.15.111:49180) 2024-09-13T17:57:36.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.968+0000 7fed93fff640 1 -- 172.21.15.111:0/2363460905 learned_addr learned my addr 172.21.15.111:0/2363460905 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:36.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.968+0000 7fed8bfff640 1 -- 172.21.15.111:0/2363460905 >> v1:172.21.15.188:6789/0 conn(0x7fed940b27b0 legacy=0x7fed940ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:36.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.968+0000 7fed8bfff640 1 -- 172.21.15.111:0/2363460905 >> v1:172.21.15.111:6789/0 conn(0x7fed94158b00 legacy=0x7fed9406c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.968+0000 7fed8bfff640 1 -- 172.21.15.111:0/2363460905 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed94070310 con 0x7fed9406c8a0 2024-09-13T17:57:36.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.968+0000 7fed937fe640 1 -- 172.21.15.111:0/2363460905 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4138225083 0 0) 0x7fed940b2d10 con 0x7fed94158b00 2024-09-13T17:57:36.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.969+0000 7fed8bfff640 1 --2- 172.21.15.111:0/2363460905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed9406c8a0 0x7fed940b1160 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0097a0 tx=0x7fed7c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812ba217aa913155 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:36.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.969+0000 7fed937fe640 1 -- 172.21.15.111:0/2363460905 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c032440 con 0x7fed9406c8a0 2024-09-13T17:57:36.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.969+0000 7fed937fe640 1 -- 172.21.15.111:0/2363460905 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c0325a0 con 0x7fed9406c8a0 2024-09-13T17:57:36.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.969+0000 7fed937fe640 1 -- 172.21.15.111:0/2363460905 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c032930 con 0x7fed9406c8a0 2024-09-13T17:57:36.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.970+0000 7fed9a4e9640 1 -- 172.21.15.111:0/2363460905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed9406c8a0 msgr2=0x7fed940b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.970+0000 7fed9a4e9640 1 --2- 172.21.15.111:0/2363460905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed9406c8a0 0x7fed940b1160 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0097a0 tx=0x7fed7c02a1e0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.970+0000 7fed9a4e9640 1 -- 172.21.15.111:0/2363460905 shutdown_connections 2024-09-13T17:57:36.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.970+0000 7fed9a4e9640 1 --2- 172.21.15.111:0/2363460905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed9406c8a0 0x7fed940b1160 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.970+0000 7fed9a4e9640 1 -- 172.21.15.111:0/2363460905 >> 172.21.15.111:0/2363460905 conn(0x7fed9406a0b0 msgr2=0x7fed9406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:36.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.970+0000 7fed9a4e9640 1 -- 172.21.15.111:0/2363460905 shutdown_connections 2024-09-13T17:57:36.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.970+0000 7fed9a4e9640 1 -- 172.21.15.111:0/2363460905 wait complete. 2024-09-13T17:57:36.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.971+0000 7fed9a4e9640 1 Processor -- start 2024-09-13T17:57:36.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.971+0000 7fed9a4e9640 1 -- start start 2024-09-13T17:57:36.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.972+0000 7fed9a4e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed9406c8a0 0x7fed940c2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.972+0000 7fed9a4e9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed940b27b0 0x7fed940c2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.972+0000 7fed9a4e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed94158b00 0x7fed940c38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.972+0000 7fed9a4e9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed94157500 con 0x7fed940b27b0 2024-09-13T17:57:36.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.972+0000 7fed9a4e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed941efee0 con 0x7fed94158b00 2024-09-13T17:57:36.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.972+0000 7fed9a4e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed940c2f80 con 0x7fed9406c8a0 2024-09-13T17:57:36.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.972+0000 7fed8bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed940b27b0 0x7fed940c2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.972+0000 7fed8bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed940b27b0 0x7fed940c2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39024/0 (socket says 172.21.15.111:39024) 2024-09-13T17:57:36.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.972+0000 7fed8bfff640 1 -- 172.21.15.111:0/2232127381 learned_addr learned my addr 172.21.15.111:0/2232127381 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:36.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.972+0000 7fed98a86640 1 --2- 172.21.15.111:0/2232127381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed94158b00 0x7fed940c38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.972+0000 7fed93fff640 1 --2- 172.21.15.111:0/2232127381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed9406c8a0 0x7fed940c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.972+0000 7fed8bfff640 1 -- 172.21.15.111:0/2232127381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed9406c8a0 msgr2=0x7fed940c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.972+0000 7fed8bfff640 1 --2- 172.21.15.111:0/2232127381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed9406c8a0 0x7fed940c2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.972+0000 7fed8bfff640 1 -- 172.21.15.111:0/2232127381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed94158b00 msgr2=0x7fed940c38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:36.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.972+0000 7fed8bfff640 1 --2- 172.21.15.111:0/2232127381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed94158b00 0x7fed940c38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:36.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.972+0000 7fed8bfff640 1 -- 172.21.15.111:0/2232127381 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c009400 con 0x7fed940b27b0 2024-09-13T17:57:36.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.972+0000 7fed93fff640 1 --2- 172.21.15.111:0/2232127381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed9406c8a0 0x7fed940c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:36.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.973+0000 7fed8bfff640 1 --2- 172.21.15.111:0/2232127381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed940b27b0 0x7fed940c2a40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0098c0 tx=0x7fed7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:36.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.973+0000 7fed917fa640 1 -- 172.21.15.111:0/2232127381 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c004230 con 0x7fed940b27b0 2024-09-13T17:57:36.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.973+0000 7fed917fa640 1 -- 172.21.15.111:0/2232127381 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c004390 con 0x7fed940b27b0 2024-09-13T17:57:36.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.973+0000 7fed9a4e9640 1 -- 172.21.15.111:0/2232127381 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed941e3a90 con 0x7fed940b27b0 2024-09-13T17:57:36.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.973+0000 7fed917fa640 1 -- 172.21.15.111:0/2232127381 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c030bc0 con 0x7fed940b27b0 2024-09-13T17:57:36.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.973+0000 7fed9a4e9640 1 -- 172.21.15.111:0/2232127381 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed940c34a0 con 0x7fed940b27b0 2024-09-13T17:57:36.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.974+0000 7fed9a4e9640 1 -- 172.21.15.111:0/2232127381 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed940b1160 con 0x7fed940b27b0 2024-09-13T17:57:36.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.974+0000 7fed917fa640 1 -- 172.21.15.111:0/2232127381 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fed7c030d20 con 0x7fed940b27b0 2024-09-13T17:57:36.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.975+0000 7fed917fa640 1 --2- 172.21.15.111:0/2232127381 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fed7406ac00 0x7fed7406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:36.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.975+0000 7fed917fa640 1 -- 172.21.15.111:0/2232127381 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fed7c0aa6b0 con 0x7fed940b27b0 2024-09-13T17:57:36.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.975+0000 7fed93fff640 1 --2- 172.21.15.111:0/2232127381 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fed7406ac00 0x7fed7406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:36.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.975+0000 7fed93fff640 1 --2- 172.21.15.111:0/2232127381 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fed7406ac00 0x7fed7406d0c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fed84000f60 tx=0x7fed84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:36.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:36.979+0000 7fed917fa640 1 -- 172.21.15.111:0/2232127381 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fed7c075630 con 0x7fed940b27b0 2024-09-13T17:57:37.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.140+0000 7fed9a4e9640 1 -- 172.21.15.111:0/2232127381 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed940af0d0 con 0x7fed940b27b0 2024-09-13T17:57:37.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.140+0000 7fed917fa640 1 -- 172.21.15.111:0/2232127381 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed7c04b020 con 0x7fed940b27b0 2024-09-13T17:57:37.142 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:57:37.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.142+0000 7fed9a4e9640 1 -- 172.21.15.111:0/2232127381 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fed7406ac00 msgr2=0x7fed7406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.142+0000 7fed9a4e9640 1 --2- 172.21.15.111:0/2232127381 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fed7406ac00 0x7fed7406d0c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fed84000f60 tx=0x7fed84009040 comp rx=0 tx=0).stop 2024-09-13T17:57:37.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.142+0000 7fed9a4e9640 1 -- 172.21.15.111:0/2232127381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed940b27b0 msgr2=0x7fed940c2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.142+0000 7fed9a4e9640 1 --2- 172.21.15.111:0/2232127381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed940b27b0 0x7fed940c2a40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0098c0 tx=0x7fed7c0046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.142+0000 7fed9a4e9640 1 -- 172.21.15.111:0/2232127381 shutdown_connections 2024-09-13T17:57:37.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.142+0000 7fed9a4e9640 1 --2- 172.21.15.111:0/2232127381 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fed7406ac00 0x7fed7406d0c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.142+0000 7fed9a4e9640 1 --2- 172.21.15.111:0/2232127381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed94158b00 0x7fed940c38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.142+0000 7fed9a4e9640 1 --2- 172.21.15.111:0/2232127381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed940b27b0 0x7fed940c2a40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.142+0000 7fed9a4e9640 1 --2- 172.21.15.111:0/2232127381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed9406c8a0 0x7fed940c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.142+0000 7fed9a4e9640 1 -- 172.21.15.111:0/2232127381 >> 172.21.15.111:0/2232127381 conn(0x7fed9406a0b0 msgr2=0x7fed940b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:37.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.142+0000 7fed9a4e9640 1 -- 172.21.15.111:0/2232127381 shutdown_connections 2024-09-13T17:57:37.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.142+0000 7fed9a4e9640 1 -- 172.21.15.111:0/2232127381 wait complete. 2024-09-13T17:57:37.155 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:57:37.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.254+0000 7f1adb8e9640 1 Processor -- start 2024-09-13T17:57:37.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.254+0000 7f1adb8e9640 1 -- start start 2024-09-13T17:57:37.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.254+0000 7f1adb8e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ad4159310 0x7f1ad40ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.254+0000 7f1adb8e9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1ad40ce4f0 0x7f1ad40ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.255+0000 7f1adb8e9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad40ccbe0 con 0x7f1ad40b2b40 2024-09-13T17:57:37.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.255+0000 7f1adb8e9640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad40ad940 con 0x7f1ad40ce4f0 2024-09-13T17:57:37.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.255+0000 7f1adb8e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40cee80 con 0x7f1ad4159310 2024-09-13T17:57:37.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.255+0000 7f1ad9e86640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1ad40ce4f0 0x7f1ad40ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.255+0000 7f1ad9e86640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1ad40ce4f0 0x7f1ad40ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39026/0 (socket says 172.21.15.111:39026) 2024-09-13T17:57:37.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.255+0000 7f1ad9e86640 1 -- 172.21.15.111:0/616298801 learned_addr learned my addr 172.21.15.111:0/616298801 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:37.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.255+0000 7f1ad9685640 1 --2- 172.21.15.111:0/616298801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ad4159310 0x7f1ad40ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.255+0000 7f1ad9e86640 1 -- 172.21.15.111:0/616298801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ad4159310 msgr2=0x7f1ad40ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:37.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.255+0000 7f1ad9e86640 1 --2- 172.21.15.111:0/616298801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ad4159310 0x7f1ad40ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.255+0000 7f1ad9e86640 1 -- 172.21.15.111:0/616298801 >> v1:172.21.15.188:6789/0 conn(0x7f1ad40b2b40 legacy=0x7f1ad40ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:37.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.255+0000 7f1ad9e86640 1 -- 172.21.15.111:0/616298801 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad40b1810 con 0x7f1ad40ce4f0 2024-09-13T17:57:37.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.255+0000 7f1ad9685640 1 --2- 172.21.15.111:0/616298801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ad4159310 0x7f1ad40ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:37.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.256+0000 7f1ad9e86640 1 --2- 172.21.15.111:0/616298801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad40ce4f0 0x7f1ad40ce940 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00bb10 tx=0x7f1acc02c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9742c0e0e37356 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:37.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.256+0000 7f1acbfff640 1 -- 172.21.15.111:0/616298801 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc0326f0 con 0x7f1ad40ce4f0 2024-09-13T17:57:37.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.256+0000 7f1acbfff640 1 -- 172.21.15.111:0/616298801 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc032850 con 0x7f1ad40ce4f0 2024-09-13T17:57:37.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.256+0000 7f1acbfff640 1 -- 172.21.15.111:0/616298801 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc032be0 con 0x7f1ad40ce4f0 2024-09-13T17:57:37.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.256+0000 7f1adb8e9640 1 -- 172.21.15.111:0/616298801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad40ce4f0 msgr2=0x7f1ad40ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.256+0000 7f1adb8e9640 1 --2- 172.21.15.111:0/616298801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad40ce4f0 0x7f1ad40ce940 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00bb10 tx=0x7f1acc02c830 comp rx=0 tx=0).stop 2024-09-13T17:57:37.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.256+0000 7f1adb8e9640 1 -- 172.21.15.111:0/616298801 shutdown_connections 2024-09-13T17:57:37.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.256+0000 7f1adb8e9640 1 --2- 172.21.15.111:0/616298801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad40ce4f0 0x7f1ad40ce940 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.256+0000 7f1adb8e9640 1 --2- 172.21.15.111:0/616298801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ad4159310 0x7f1ad40ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.256+0000 7f1adb8e9640 1 -- 172.21.15.111:0/616298801 >> 172.21.15.111:0/616298801 conn(0x7f1ad406a0b0 msgr2=0x7f1ad406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:37.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.257+0000 7f1adb8e9640 1 -- 172.21.15.111:0/616298801 shutdown_connections 2024-09-13T17:57:37.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.257+0000 7f1adb8e9640 1 -- 172.21.15.111:0/616298801 wait complete. 2024-09-13T17:57:37.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.258+0000 7f1adb8e9640 1 Processor -- start 2024-09-13T17:57:37.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.258+0000 7f1adb8e9640 1 -- start start 2024-09-13T17:57:37.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.258+0000 7f1adb8e9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad40b2b40 0x7f1ad4097680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.258+0000 7f1adb8e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ad40ce4f0 0x7f1ad41eda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.258+0000 7f1adb8e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad4159310 0x7f1ad40c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.258+0000 7f1adb8e9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad41541e0 con 0x7f1ad40b2b40 2024-09-13T17:57:37.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.258+0000 7f1adb8e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad4154430 con 0x7f1ad4159310 2024-09-13T17:57:37.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.258+0000 7f1adb8e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad4156e80 con 0x7f1ad40ce4f0 2024-09-13T17:57:37.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.259+0000 7f1ad9685640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad40b2b40 0x7f1ad4097680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.259+0000 7f1ad9685640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad40b2b40 0x7f1ad4097680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39038/0 (socket says 172.21.15.111:39038) 2024-09-13T17:57:37.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.259+0000 7f1ad9685640 1 -- 172.21.15.111:0/2882808559 learned_addr learned my addr 172.21.15.111:0/2882808559 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:37.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.259+0000 7f1ad8e84640 1 --2- 172.21.15.111:0/2882808559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ad40ce4f0 0x7f1ad41eda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.259+0000 7f1ad9e86640 1 --2- 172.21.15.111:0/2882808559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad4159310 0x7f1ad40c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.259+0000 7f1ad9685640 1 -- 172.21.15.111:0/2882808559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ad40ce4f0 msgr2=0x7f1ad41eda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.259+0000 7f1ad9685640 1 --2- 172.21.15.111:0/2882808559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ad40ce4f0 0x7f1ad41eda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.259+0000 7f1ad9685640 1 -- 172.21.15.111:0/2882808559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad4159310 msgr2=0x7f1ad40c1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.259+0000 7f1ad9685640 1 --2- 172.21.15.111:0/2882808559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad4159310 0x7f1ad40c1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.259+0000 7f1ad9685640 1 -- 172.21.15.111:0/2882808559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1acc00b770 con 0x7f1ad40b2b40 2024-09-13T17:57:37.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.259+0000 7f1ad9685640 1 --2- 172.21.15.111:0/2882808559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad40b2b40 0x7f1ad4097680 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000ce00 tx=0x7f1ad000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:37.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.259+0000 7f1ac9ffb640 1 -- 172.21.15.111:0/2882808559 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad000f890 con 0x7f1ad40b2b40 2024-09-13T17:57:37.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.259+0000 7f1ac9ffb640 1 -- 172.21.15.111:0/2882808559 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad000f9f0 con 0x7f1ad40b2b40 2024-09-13T17:57:37.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.259+0000 7f1adb8e9640 1 -- 172.21.15.111:0/2882808559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad4157160 con 0x7f1ad40b2b40 2024-09-13T17:57:37.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.259+0000 7f1ac9ffb640 1 -- 172.21.15.111:0/2882808559 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad000fdc0 con 0x7f1ad40b2b40 2024-09-13T17:57:37.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.259+0000 7f1adb8e9640 1 -- 172.21.15.111:0/2882808559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad4154710 con 0x7f1ad40b2b40 2024-09-13T17:57:37.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.260+0000 7f1adb8e9640 1 -- 172.21.15.111:0/2882808559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad40cec70 con 0x7f1ad40b2b40 2024-09-13T17:57:37.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.261+0000 7f1ac9ffb640 1 -- 172.21.15.111:0/2882808559 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1ad000b040 con 0x7f1ad40b2b40 2024-09-13T17:57:37.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.261+0000 7f1ac9ffb640 1 --2- 172.21.15.111:0/2882808559 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ab806ac00 0x7f1ab806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.261+0000 7f1ac9ffb640 1 -- 172.21.15.111:0/2882808559 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1ad0089ed0 con 0x7f1ad40b2b40 2024-09-13T17:57:37.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.261+0000 7f1ad8e84640 1 --2- 172.21.15.111:0/2882808559 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ab806ac00 0x7f1ab806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.262+0000 7f1ad8e84640 1 --2- 172.21.15.111:0/2882808559 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ab806ac00 0x7f1ab806d0c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4004f60 tx=0x7f1ac4007e30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:37.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.266+0000 7f1ac9ffb640 1 -- 172.21.15.111:0/2882808559 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f1ad0054f00 con 0x7f1ad40b2b40 2024-09-13T17:57:37.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.429+0000 7f1adb8e9640 1 -- 172.21.15.111:0/2882808559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1ad406dfa0 con 0x7f1ad40b2b40 2024-09-13T17:57:37.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.429+0000 7f1ac9ffb640 1 -- 172.21.15.111:0/2882808559 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1ad0058520 con 0x7f1ad40b2b40 2024-09-13T17:57:37.433 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:57:37.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.431+0000 7f1adb8e9640 1 -- 172.21.15.111:0/2882808559 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ab806ac00 msgr2=0x7f1ab806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.431+0000 7f1adb8e9640 1 --2- 172.21.15.111:0/2882808559 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ab806ac00 0x7f1ab806d0c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4004f60 tx=0x7f1ac4007e30 comp rx=0 tx=0).stop 2024-09-13T17:57:37.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.431+0000 7f1adb8e9640 1 -- 172.21.15.111:0/2882808559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad40b2b40 msgr2=0x7f1ad4097680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.431+0000 7f1adb8e9640 1 --2- 172.21.15.111:0/2882808559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad40b2b40 0x7f1ad4097680 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000ce00 tx=0x7f1ad000a650 comp rx=0 tx=0).stop 2024-09-13T17:57:37.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.431+0000 7f1adb8e9640 1 -- 172.21.15.111:0/2882808559 shutdown_connections 2024-09-13T17:57:37.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.431+0000 7f1adb8e9640 1 --2- 172.21.15.111:0/2882808559 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ab806ac00 0x7f1ab806d0c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.431+0000 7f1adb8e9640 1 --2- 172.21.15.111:0/2882808559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad4159310 0x7f1ad40c1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.431+0000 7f1adb8e9640 1 --2- 172.21.15.111:0/2882808559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ad40ce4f0 0x7f1ad41eda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.431+0000 7f1adb8e9640 1 --2- 172.21.15.111:0/2882808559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad40b2b40 0x7f1ad4097680 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.431+0000 7f1adb8e9640 1 -- 172.21.15.111:0/2882808559 >> 172.21.15.111:0/2882808559 conn(0x7f1ad406a0b0 msgr2=0x7f1ad40ac350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:37.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.431+0000 7f1adb8e9640 1 -- 172.21.15.111:0/2882808559 shutdown_connections 2024-09-13T17:57:37.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.431+0000 7f1adb8e9640 1 -- 172.21.15.111:0/2882808559 wait complete. 2024-09-13T17:57:37.444 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:57:37.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.542+0000 7f701bfbc640 1 Processor -- start 2024-09-13T17:57:37.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.543+0000 7f701bfbc640 1 -- start start 2024-09-13T17:57:37.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.543+0000 7f701bfbc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f701415d9d0 0x7f70140c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.543+0000 7f701bfbc640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7014066400 con 0x7f701406ca30 2024-09-13T17:57:37.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.543+0000 7f701bfbc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7014066120 con 0x7f701406a5e0 2024-09-13T17:57:37.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.543+0000 7f701bfbc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70140c9220 con 0x7f701415d9d0 2024-09-13T17:57:37.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.543+0000 7f7019557640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f701406ca30 0x7f7014069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49182/0 (socket says 172.21.15.111:49182) 2024-09-13T17:57:37.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.543+0000 7f7019557640 1 -- 172.21.15.111:0/666871530 learned_addr learned my addr 172.21.15.111:0/666871530 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:37.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7019d58640 1 --2- 172.21.15.111:0/666871530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f701415d9d0 0x7f70140c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7018d56640 1 -- 172.21.15.111:0/666871530 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576332366 0 0) 0x7f7014066400 con 0x7f701406ca30 2024-09-13T17:57:37.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7018d56640 1 -- 172.21.15.111:0/666871530 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7000003520 con 0x7f701406ca30 2024-09-13T17:57:37.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7018d56640 1 -- 172.21.15.111:0/666871530 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067669908 0 0) 0x7f7014066120 con 0x7f701406a5e0 2024-09-13T17:57:37.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7018d56640 1 -- 172.21.15.111:0/666871530 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7014066400 con 0x7f701406a5e0 2024-09-13T17:57:37.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7018d56640 1 -- 172.21.15.111:0/666871530 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1470300055 0 0) 0x7f7000003520 con 0x7f701406ca30 2024-09-13T17:57:37.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7018d56640 1 -- 172.21.15.111:0/666871530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f701415d9d0 msgr2=0x7f70140c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:37.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7018d56640 1 --2- 172.21.15.111:0/666871530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f701415d9d0 0x7f70140c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7018d56640 1 -- 172.21.15.111:0/666871530 >> v1:172.21.15.188:6789/0 conn(0x7f701406a5e0 legacy=0x7f701406e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7018d56640 1 -- 172.21.15.111:0/666871530 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70140c9500 con 0x7f701406ca30 2024-09-13T17:57:37.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7018d56640 1 -- 172.21.15.111:0/666871530 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f7004001670 con 0x7f701406ca30 2024-09-13T17:57:37.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7018d56640 1 -- 172.21.15.111:0/666871530 >> v1:172.21.15.111:6789/0 conn(0x7f701406ca30 legacy=0x7f7014069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7018d56640 1 --2- 172.21.15.111:0/666871530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7000003b30 0x7f7000003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7018d56640 1 --2- 172.21.15.111:0/666871530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f701415d9d0 0x7f70000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7018d56640 1 --2- 172.21.15.111:0/666871530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7000004d00 0x7f7000005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7018d56640 1 -- 172.21.15.111:0/666871530 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70140c9220 con 0x7f7000003b30 2024-09-13T17:57:37.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7018d56640 1 -- 172.21.15.111:0/666871530 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700000ba30 con 0x7f7000004d00 2024-09-13T17:57:37.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7018d56640 1 -- 172.21.15.111:0/666871530 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700000bba0 con 0x7f701415d9d0 2024-09-13T17:57:37.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7019d58640 1 --2- 172.21.15.111:0/666871530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7000003b30 0x7f7000003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7019557640 1 --2- 172.21.15.111:0/666871530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f701415d9d0 0x7f70000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f701a559640 1 --2- 172.21.15.111:0/666871530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7000004d00 0x7f7000005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7019d58640 1 -- 172.21.15.111:0/666871530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f701415d9d0 msgr2=0x7f70000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:37.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7019d58640 1 --2- 172.21.15.111:0/666871530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f701415d9d0 0x7f70000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7019d58640 1 -- 172.21.15.111:0/666871530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7000004d00 msgr2=0x7f7000005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:37.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7019d58640 1 --2- 172.21.15.111:0/666871530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7000004d00 0x7f7000005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f7019d58640 1 -- 172.21.15.111:0/666871530 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7000003520 con 0x7f7000003b30 2024-09-13T17:57:37.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.544+0000 7f701a559640 1 --2- 172.21.15.111:0/666871530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7000004d00 0x7f7000005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:37.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.545+0000 7f7019d58640 1 --2- 172.21.15.111:0/666871530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7000003b30 0x7f7000003f30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7010009ac0 tx=0x7f70000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90da54dc188b7bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:37.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.545+0000 7f7018d56640 1 -- 172.21.15.111:0/666871530 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010031400 con 0x7f7000003b30 2024-09-13T17:57:37.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.545+0000 7f7018d56640 1 -- 172.21.15.111:0/666871530 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701002d050 con 0x7f7000003b30 2024-09-13T17:57:37.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.545+0000 7f7018d56640 1 -- 172.21.15.111:0/666871530 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010031780 con 0x7f7000003b30 2024-09-13T17:57:37.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.546+0000 7f701bfbc640 1 -- 172.21.15.111:0/666871530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7000003b30 msgr2=0x7f7000003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.546+0000 7f701bfbc640 1 --2- 172.21.15.111:0/666871530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7000003b30 0x7f7000003f30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7010009ac0 tx=0x7f70000057f0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.546+0000 7f701a559640 1 -- 172.21.15.111:0/666871530 reap_dead start 2024-09-13T17:57:37.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.546+0000 7f701bfbc640 1 -- 172.21.15.111:0/666871530 shutdown_connections 2024-09-13T17:57:37.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.546+0000 7f701bfbc640 1 -- 172.21.15.111:0/666871530 >> 172.21.15.111:0/666871530 conn(0x7f70140cb5b0 msgr2=0x7f70140cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:37.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.546+0000 7f701bfbc640 1 -- 172.21.15.111:0/666871530 shutdown_connections 2024-09-13T17:57:37.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.546+0000 7f701bfbc640 1 -- 172.21.15.111:0/666871530 wait complete. 2024-09-13T17:57:37.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.547+0000 7f701bfbc640 1 Processor -- start 2024-09-13T17:57:37.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.547+0000 7f701bfbc640 1 -- start start 2024-09-13T17:57:37.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f701bfbc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7000003b30 0x7f70140c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f701bfbc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70140c5d00 0x7f70141f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f701bfbc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70140c7010 0x7f70141f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f701bfbc640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70140c7500 con 0x7f70140c5d00 2024-09-13T17:57:37.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f701bfbc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70140c77e0 con 0x7f7000003b30 2024-09-13T17:57:37.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f701bfbc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70141f1270 con 0x7f70140c7010 2024-09-13T17:57:37.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f7019557640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70140c5d00 0x7f70141f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f7019557640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70140c5d00 0x7f70141f36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39070/0 (socket says 172.21.15.111:39070) 2024-09-13T17:57:37.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f7019557640 1 -- 172.21.15.111:0/4244901353 learned_addr learned my addr 172.21.15.111:0/4244901353 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:37.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f7019d58640 1 --2- 172.21.15.111:0/4244901353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7000003b30 0x7f70140c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f701a559640 1 --2- 172.21.15.111:0/4244901353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70140c7010 0x7f70141f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f7019557640 1 -- 172.21.15.111:0/4244901353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70140c7010 msgr2=0x7f70141f3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f7019557640 1 --2- 172.21.15.111:0/4244901353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70140c7010 0x7f70141f3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f7019557640 1 -- 172.21.15.111:0/4244901353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7000003b30 msgr2=0x7f70140c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f7019557640 1 --2- 172.21.15.111:0/4244901353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7000003b30 0x7f70140c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f7019557640 1 -- 172.21.15.111:0/4244901353 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70141f1550 con 0x7f70140c5d00 2024-09-13T17:57:37.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f7019557640 1 --2- 172.21.15.111:0/4244901353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70140c5d00 0x7f70141f36a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7004001050 tx=0x7f70040077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:37.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f700a7fc640 1 -- 172.21.15.111:0/4244901353 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700400f960 con 0x7f70140c5d00 2024-09-13T17:57:37.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f700a7fc640 1 -- 172.21.15.111:0/4244901353 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700400fac0 con 0x7f70140c5d00 2024-09-13T17:57:37.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.548+0000 7f701bfbc640 1 -- 172.21.15.111:0/4244901353 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70141f1830 con 0x7f70140c5d00 2024-09-13T17:57:37.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.549+0000 7f700a7fc640 1 -- 172.21.15.111:0/4244901353 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700400fe30 con 0x7f70140c5d00 2024-09-13T17:57:37.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.549+0000 7f701bfbc640 1 -- 172.21.15.111:0/4244901353 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70141552b0 con 0x7f70140c5d00 2024-09-13T17:57:37.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.549+0000 7f701bfbc640 1 -- 172.21.15.111:0/4244901353 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7014066120 con 0x7f70140c5d00 2024-09-13T17:57:37.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.550+0000 7f700a7fc640 1 -- 172.21.15.111:0/4244901353 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f700401d460 con 0x7f70140c5d00 2024-09-13T17:57:37.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.550+0000 7f700a7fc640 1 --2- 172.21.15.111:0/4244901353 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6ff806ac00 0x7f6ff806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.550+0000 7f700a7fc640 1 -- 172.21.15.111:0/4244901353 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7004089690 con 0x7f70140c5d00 2024-09-13T17:57:37.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.550+0000 7f7019d58640 1 --2- 172.21.15.111:0/4244901353 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6ff806ac00 0x7f6ff806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.551+0000 7f7019d58640 1 --2- 172.21.15.111:0/4244901353 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6ff806ac00 0x7f6ff806d0c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7010033000 tx=0x7f7010008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:37.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.554+0000 7f700a7fc640 1 -- 172.21.15.111:0/4244901353 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f7004054610 con 0x7f70140c5d00 2024-09-13T17:57:37.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.716+0000 7f701bfbc640 1 -- 172.21.15.111:0/4244901353 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f70140cc310 con 0x7f70140c5d00 2024-09-13T17:57:37.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.717+0000 7f700a7fc640 1 -- 172.21.15.111:0/4244901353 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7004057c30 con 0x7f70140c5d00 2024-09-13T17:57:37.719 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:57:37.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.719+0000 7f701bfbc640 1 -- 172.21.15.111:0/4244901353 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6ff806ac00 msgr2=0x7f6ff806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.719+0000 7f701bfbc640 1 --2- 172.21.15.111:0/4244901353 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6ff806ac00 0x7f6ff806d0c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7010033000 tx=0x7f7010008030 comp rx=0 tx=0).stop 2024-09-13T17:57:37.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.719+0000 7f701bfbc640 1 -- 172.21.15.111:0/4244901353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70140c5d00 msgr2=0x7f70141f36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.719+0000 7f701bfbc640 1 --2- 172.21.15.111:0/4244901353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70140c5d00 0x7f70141f36a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7004001050 tx=0x7f70040077a0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.719+0000 7f701bfbc640 1 -- 172.21.15.111:0/4244901353 shutdown_connections 2024-09-13T17:57:37.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.719+0000 7f701bfbc640 1 --2- 172.21.15.111:0/4244901353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70140c7010 0x7f70141f3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.719+0000 7f701bfbc640 1 --2- 172.21.15.111:0/4244901353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70140c5d00 0x7f70141f36a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.719+0000 7f701bfbc640 1 --2- 172.21.15.111:0/4244901353 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6ff806ac00 0x7f6ff806d0c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.719+0000 7f701bfbc640 1 --2- 172.21.15.111:0/4244901353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7000003b30 0x7f70140c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.719+0000 7f701bfbc640 1 -- 172.21.15.111:0/4244901353 >> 172.21.15.111:0/4244901353 conn(0x7f70140cb5b0 msgr2=0x7f701415c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:37.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.719+0000 7f701bfbc640 1 -- 172.21.15.111:0/4244901353 shutdown_connections 2024-09-13T17:57:37.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.719+0000 7f701bfbc640 1 -- 172.21.15.111:0/4244901353 wait complete. 2024-09-13T17:57:37.731 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:57:37.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.833+0000 7f171648b640 1 Processor -- start 2024-09-13T17:57:37.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.833+0000 7f171648b640 1 -- start start 2024-09-13T17:57:37.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.834+0000 7f171648b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1710159370 0x7f171015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.834+0000 7f171648b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f171015d480 0x7f171006dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.834+0000 7f171648b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171015a610 con 0x7f171006aee0 2024-09-13T17:57:37.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.834+0000 7f171648b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17100b3c70 con 0x7f171015d480 2024-09-13T17:57:37.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.834+0000 7f171648b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17100b2eb0 con 0x7f1710159370 2024-09-13T17:57:37.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.834+0000 7f1714a28640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f171015d480 0x7f171006dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.834+0000 7f170ffff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f171006aee0 0x7f171006b2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49190/0 (socket says 172.21.15.111:49190) 2024-09-13T17:57:37.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.834+0000 7f170ffff640 1 -- 172.21.15.111:0/4211428024 learned_addr learned my addr 172.21.15.111:0/4211428024 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:37.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.834+0000 7f1707fff640 1 --2- 172.21.15.111:0/4211428024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1710159370 0x7f171015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.835+0000 7f1714a28640 1 -- 172.21.15.111:0/4211428024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1710159370 msgr2=0x7f171015cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:37.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.835+0000 7f1714a28640 1 --2- 172.21.15.111:0/4211428024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1710159370 0x7f171015cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.835+0000 7f1714a28640 1 -- 172.21.15.111:0/4211428024 >> v1:172.21.15.111:6789/0 conn(0x7f171006aee0 legacy=0x7f171006b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.835+0000 7f170f7fe640 1 -- 172.21.15.111:0/4211428024 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910561371 0 0) 0x7f1700001110 con 0x7f171006aee0 2024-09-13T17:57:37.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.835+0000 7f1714a28640 1 -- 172.21.15.111:0/4211428024 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17100b3190 con 0x7f171015d480 2024-09-13T17:57:37.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.835+0000 7f1707fff640 1 --2- 172.21.15.111:0/4211428024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1710159370 0x7f171015cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:37.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.835+0000 7f1714a28640 1 --2- 172.21.15.111:0/4211428024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f171015d480 0x7f171006dc40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1708009d90 tx=0x7f170802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb114a55cc40cd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:37.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.835+0000 7f170f7fe640 1 -- 172.21.15.111:0/4211428024 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708031600 con 0x7f171015d480 2024-09-13T17:57:37.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.835+0000 7f170f7fe640 1 -- 172.21.15.111:0/4211428024 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170802d050 con 0x7f171015d480 2024-09-13T17:57:37.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.835+0000 7f170f7fe640 1 -- 172.21.15.111:0/4211428024 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708031980 con 0x7f171015d480 2024-09-13T17:57:37.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.836+0000 7f171648b640 1 -- 172.21.15.111:0/4211428024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f171015d480 msgr2=0x7f171006dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.836+0000 7f171648b640 1 --2- 172.21.15.111:0/4211428024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f171015d480 0x7f171006dc40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1708009d90 tx=0x7f170802a5d0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.836+0000 7f171648b640 1 -- 172.21.15.111:0/4211428024 shutdown_connections 2024-09-13T17:57:37.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.836+0000 7f171648b640 1 --2- 172.21.15.111:0/4211428024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f171015d480 0x7f171006dc40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.836+0000 7f171648b640 1 --2- 172.21.15.111:0/4211428024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1710159370 0x7f171015cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.836+0000 7f171648b640 1 -- 172.21.15.111:0/4211428024 >> 172.21.15.111:0/4211428024 conn(0x7f17100709f0 msgr2=0x7f1710070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:37.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.836+0000 7f171648b640 1 -- 172.21.15.111:0/4211428024 shutdown_connections 2024-09-13T17:57:37.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.836+0000 7f171648b640 1 -- 172.21.15.111:0/4211428024 wait complete. 2024-09-13T17:57:37.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.837+0000 7f171648b640 1 Processor -- start 2024-09-13T17:57:37.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.837+0000 7f171648b640 1 -- start start 2024-09-13T17:57:37.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.838+0000 7f171648b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f171006aee0 0x7f1710151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.838+0000 7f171648b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1710159370 0x7f17100a8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.838+0000 7f171648b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f171015d480 0x7f1710153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.838+0000 7f171648b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17100a8840 con 0x7f1710159370 2024-09-13T17:57:37.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.838+0000 7f171648b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1710155110 con 0x7f171015d480 2024-09-13T17:57:37.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.838+0000 7f171648b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17101542f0 con 0x7f171006aee0 2024-09-13T17:57:37.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.838+0000 7f1707fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1710159370 0x7f17100a8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.838+0000 7f1707fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1710159370 0x7f17100a8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39090/0 (socket says 172.21.15.111:39090) 2024-09-13T17:57:37.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.838+0000 7f1707fff640 1 -- 172.21.15.111:0/4194660139 learned_addr learned my addr 172.21.15.111:0/4194660139 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:37.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.838+0000 7f1714a28640 1 --2- 172.21.15.111:0/4194660139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f171015d480 0x7f1710153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.838+0000 7f170ffff640 1 --2- 172.21.15.111:0/4194660139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f171006aee0 0x7f1710151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.838+0000 7f1707fff640 1 -- 172.21.15.111:0/4194660139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f171006aee0 msgr2=0x7f1710151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.838+0000 7f1707fff640 1 --2- 172.21.15.111:0/4194660139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f171006aee0 0x7f1710151740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.838+0000 7f1707fff640 1 -- 172.21.15.111:0/4194660139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f171015d480 msgr2=0x7f1710153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:37.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.838+0000 7f1707fff640 1 --2- 172.21.15.111:0/4194660139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f171015d480 0x7f1710153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:37.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.838+0000 7f1707fff640 1 -- 172.21.15.111:0/4194660139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1708009a70 con 0x7f1710159370 2024-09-13T17:57:37.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.839+0000 7f1707fff640 1 --2- 172.21.15.111:0/4194660139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1710159370 0x7f17100a8300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f16f800e730 tx=0x7f16f800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:37.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.839+0000 7f170d7fa640 1 -- 172.21.15.111:0/4194660139 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f8004230 con 0x7f1710159370 2024-09-13T17:57:37.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.839+0000 7f170d7fa640 1 -- 172.21.15.111:0/4194660139 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f8004390 con 0x7f1710159370 2024-09-13T17:57:37.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.839+0000 7f171648b640 1 -- 172.21.15.111:0/4194660139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1710154590 con 0x7f1710159370 2024-09-13T17:57:37.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.839+0000 7f170d7fa640 1 -- 172.21.15.111:0/4194660139 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f800c8f0 con 0x7f1710159370 2024-09-13T17:57:37.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.839+0000 7f171648b640 1 -- 172.21.15.111:0/4194660139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1710154b50 con 0x7f1710159370 2024-09-13T17:57:37.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.840+0000 7f171648b640 1 -- 172.21.15.111:0/4194660139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171015cf40 con 0x7f1710159370 2024-09-13T17:57:37.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.840+0000 7f170d7fa640 1 -- 172.21.15.111:0/4194660139 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f16f800ca50 con 0x7f1710159370 2024-09-13T17:57:37.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.841+0000 7f170d7fa640 1 --2- 172.21.15.111:0/4194660139 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f16f006ac00 0x7f16f006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:37.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.841+0000 7f170d7fa640 1 -- 172.21.15.111:0/4194660139 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f16f8089270 con 0x7f1710159370 2024-09-13T17:57:37.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.841+0000 7f170ffff640 1 --2- 172.21.15.111:0/4194660139 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f16f006ac00 0x7f16f006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:37.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.841+0000 7f170ffff640 1 --2- 172.21.15.111:0/4194660139 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f16f006ac00 0x7f16f006d0c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1700000f60 tx=0x7f1700009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:37.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:37.845+0000 7f170d7fa640 1 -- 172.21.15.111:0/4194660139 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f16f8054160 con 0x7f1710159370 2024-09-13T17:57:38.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:38.016+0000 7f171648b640 1 -- 172.21.15.111:0/4194660139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f17101eeb90 con 0x7f1710159370 2024-09-13T17:57:38.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:38.017+0000 7f170d7fa640 1 -- 172.21.15.111:0/4194660139 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f16f8057780 con 0x7f1710159370 2024-09-13T17:57:38.019 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2024-09-13T17:57:38.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:38.019+0000 7f171648b640 1 -- 172.21.15.111:0/4194660139 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f16f006ac00 msgr2=0x7f16f006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:38.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:38.019+0000 7f171648b640 1 --2- 172.21.15.111:0/4194660139 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f16f006ac00 0x7f16f006d0c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1700000f60 tx=0x7f1700009040 comp rx=0 tx=0).stop 2024-09-13T17:57:38.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:38.019+0000 7f171648b640 1 -- 172.21.15.111:0/4194660139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1710159370 msgr2=0x7f17100a8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:38.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:38.019+0000 7f171648b640 1 --2- 172.21.15.111:0/4194660139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1710159370 0x7f17100a8300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f16f800e730 tx=0x7f16f800ebf0 comp rx=0 tx=0).stop 2024-09-13T17:57:38.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:38.019+0000 7f171648b640 1 -- 172.21.15.111:0/4194660139 shutdown_connections 2024-09-13T17:57:38.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:38.019+0000 7f171648b640 1 --2- 172.21.15.111:0/4194660139 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f16f006ac00 0x7f16f006d0c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:38.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:38.019+0000 7f171648b640 1 --2- 172.21.15.111:0/4194660139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f171015d480 0x7f1710153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:38.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:38.019+0000 7f171648b640 1 --2- 172.21.15.111:0/4194660139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1710159370 0x7f17100a8300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:38.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:38.019+0000 7f171648b640 1 --2- 172.21.15.111:0/4194660139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f171006aee0 0x7f1710151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:38.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:38.019+0000 7f171648b640 1 -- 172.21.15.111:0/4194660139 >> 172.21.15.111:0/4194660139 conn(0x7f17100709f0 msgr2=0x7f171015ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:38.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:38.019+0000 7f171648b640 1 -- 172.21.15.111:0/4194660139 shutdown_connections 2024-09-13T17:57:38.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:38.019+0000 7f171648b640 1 -- 172.21.15.111:0/4194660139 wait complete. 2024-09-13T17:57:38.031 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi111.front.sepia.ceph.com... 2024-09-13T17:57:38.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:38.032 DEBUG:teuthology.orchestra.run.smithi111:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:57:38.055 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi111.stderr:ceph-fuse[43474]: fuse finished with error 0 and tester_r 0 2024-09-13T17:57:38.057 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:57:44.159 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T17:57:44.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:44.160 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-13T17:57:44.238 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:44.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.339+0000 7ff085879640 1 Processor -- start 2024-09-13T17:57:44.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.339+0000 7ff085879640 1 -- start start 2024-09-13T17:57:44.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.340+0000 7ff085879640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff08015c4c0 0x7ff08006d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.340+0000 7ff085879640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff08015c8a0 0x7ff0800cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.340+0000 7ff085879640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff0800cd750 0x7ff0800cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.340+0000 7ff085879640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0800ce0f0 con 0x7ff0800cd750 2024-09-13T17:57:44.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.340+0000 7ff085879640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0801601d0 con 0x7ff08015c8a0 2024-09-13T17:57:44.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.340+0000 7ff085879640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08006de20 con 0x7ff08015c4c0 2024-09-13T17:57:44.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.340+0000 7ff07f7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff0800cd750 0x7ff0800cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.340+0000 7ff07f7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff0800cd750 0x7ff0800cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53778/0 (socket says 172.21.15.111:53778) 2024-09-13T17:57:44.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.340+0000 7ff07f7fe640 1 -- 172.21.15.111:0/3911663480 learned_addr learned my addr 172.21.15.111:0/3911663480 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:44.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.340+0000 7ff07effd640 1 --2- 172.21.15.111:0/3911663480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff08015c4c0 0x7ff08006d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.340+0000 7ff07e7fc640 1 --2- 172.21.15.111:0/3911663480 >> v2:172.21.15.188:3300/0 conn(0x7ff08015c8a0 0x7ff0800cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.340+0000 7ff07f7fe640 1 -- 172.21.15.111:0/3911663480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff08015c4c0 msgr2=0x7ff08006d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:44.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.340+0000 7ff07f7fe640 1 --2- 172.21.15.111:0/3911663480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff08015c4c0 0x7ff08006d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.340+0000 7ff07f7fe640 1 -- 172.21.15.111:0/3911663480 >> v2:172.21.15.188:3300/0 conn(0x7ff08015c8a0 msgr2=0x7ff0800cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:44.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.340+0000 7ff07f7fe640 1 --2- 172.21.15.111:0/3911663480 >> v2:172.21.15.188:3300/0 conn(0x7ff08015c8a0 0x7ff0800cc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.340+0000 7ff07f7fe640 1 -- 172.21.15.111:0/3911663480 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff080160420 con 0x7ff0800cd750 2024-09-13T17:57:44.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.341+0000 7ff07f7fe640 1 --2- 172.21.15.111:0/3911663480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0800cd750 0x7ff0800cdbb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff08006fd10 tx=0x7ff07802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a1b07c2c0f29cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:44.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.341+0000 7ff07dffb640 1 -- 172.21.15.111:0/3911663480 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078031600 con 0x7ff0800cd750 2024-09-13T17:57:44.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.341+0000 7ff07dffb640 1 -- 172.21.15.111:0/3911663480 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07802d050 con 0x7ff0800cd750 2024-09-13T17:57:44.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.341+0000 7ff07dffb640 1 -- 172.21.15.111:0/3911663480 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078031980 con 0x7ff0800cd750 2024-09-13T17:57:44.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.342+0000 7ff085879640 1 -- 172.21.15.111:0/3911663480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0800cd750 msgr2=0x7ff0800cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:44.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.342+0000 7ff085879640 1 --2- 172.21.15.111:0/3911663480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0800cd750 0x7ff0800cdbb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff08006fd10 tx=0x7ff07802a6a0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.342+0000 7ff085879640 1 -- 172.21.15.111:0/3911663480 shutdown_connections 2024-09-13T17:57:44.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.342+0000 7ff085879640 1 --2- 172.21.15.111:0/3911663480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0800cd750 0x7ff0800cdbb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.342+0000 7ff085879640 1 --2- 172.21.15.111:0/3911663480 >> v2:172.21.15.188:3300/0 conn(0x7ff08015c8a0 0x7ff0800cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.342+0000 7ff085879640 1 --2- 172.21.15.111:0/3911663480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff08015c4c0 0x7ff08006d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.342+0000 7ff085879640 1 -- 172.21.15.111:0/3911663480 >> 172.21.15.111:0/3911663480 conn(0x7ff0800989d0 msgr2=0x7ff0800b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:44.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.342+0000 7ff085879640 1 -- 172.21.15.111:0/3911663480 shutdown_connections 2024-09-13T17:57:44.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.342+0000 7ff085879640 1 -- 172.21.15.111:0/3911663480 wait complete. 2024-09-13T17:57:44.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.343+0000 7ff085879640 1 Processor -- start 2024-09-13T17:57:44.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.343+0000 7ff085879640 1 -- start start 2024-09-13T17:57:44.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff085879640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0800cd750 0x7ff080153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff085879640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff08015c4c0 0x7ff08015ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff085879640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff08015c8a0 0x7ff08015b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff085879640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08015e710 con 0x7ff0800cd750 2024-09-13T17:57:44.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff085879640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08015db00 con 0x7ff08015c8a0 2024-09-13T17:57:44.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff085879640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08015dde0 con 0x7ff08015c4c0 2024-09-13T17:57:44.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff07effd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0800cd750 0x7ff080153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff07effd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0800cd750 0x7ff080153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53782/0 (socket says 172.21.15.111:53782) 2024-09-13T17:57:44.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff07effd640 1 -- 172.21.15.111:0/3843705991 learned_addr learned my addr 172.21.15.111:0/3843705991 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:44.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff07e7fc640 1 --2- 172.21.15.111:0/3843705991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff08015c4c0 0x7ff08015ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff07f7fe640 1 --2- 172.21.15.111:0/3843705991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff08015c8a0 0x7ff08015b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff07effd640 1 -- 172.21.15.111:0/3843705991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff08015c4c0 msgr2=0x7ff08015ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:44.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff07effd640 1 --2- 172.21.15.111:0/3843705991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff08015c4c0 0x7ff08015ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff07effd640 1 -- 172.21.15.111:0/3843705991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff08015c8a0 msgr2=0x7ff08015b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:44.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff07effd640 1 --2- 172.21.15.111:0/3843705991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff08015c8a0 0x7ff08015b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff07effd640 1 -- 172.21.15.111:0/3843705991 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff078009b30 con 0x7ff0800cd750 2024-09-13T17:57:44.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff07effd640 1 --2- 172.21.15.111:0/3843705991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0800cd750 0x7ff080153980 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff07000e9b0 tx=0x7ff07000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:44.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff067fff640 1 -- 172.21.15.111:0/3843705991 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff070009670 con 0x7ff0800cd750 2024-09-13T17:57:44.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff067fff640 1 -- 172.21.15.111:0/3843705991 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07000c9a0 con 0x7ff0800cd750 2024-09-13T17:57:44.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff085879640 1 -- 172.21.15.111:0/3843705991 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08015e0c0 con 0x7ff0800cd750 2024-09-13T17:57:44.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff067fff640 1 -- 172.21.15.111:0/3843705991 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07000cd50 con 0x7ff0800cd750 2024-09-13T17:57:44.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.344+0000 7ff085879640 1 -- 172.21.15.111:0/3843705991 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08015eba0 con 0x7ff0800cd750 2024-09-13T17:57:44.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.346+0000 7ff085879640 1 -- 172.21.15.111:0/3843705991 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff080066f30 con 0x7ff0800cd750 2024-09-13T17:57:44.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.346+0000 7ff067fff640 1 -- 172.21.15.111:0/3843705991 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff070013470 con 0x7ff0800cd750 2024-09-13T17:57:44.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.346+0000 7ff067fff640 1 --2- 172.21.15.111:0/3843705991 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff06c06ac00 0x7ff06c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.346+0000 7ff067fff640 1 -- 172.21.15.111:0/3843705991 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff0700897a0 con 0x7ff0800cd750 2024-09-13T17:57:44.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.346+0000 7ff07e7fc640 1 --2- 172.21.15.111:0/3843705991 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff06c06ac00 0x7ff06c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.347+0000 7ff07e7fc640 1 --2- 172.21.15.111:0/3843705991 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff06c06ac00 0x7ff06c06d0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff080098fb0 tx=0x7ff074009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:44.353 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.352+0000 7ff067fff640 1 -- 172.21.15.111:0/3843705991 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff0700547d0 con 0x7ff0800cd750 2024-09-13T17:57:44.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.513+0000 7ff085879640 1 -- 172.21.15.111:0/3843705991 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0800b5240 con 0x7ff0800cd750 2024-09-13T17:57:44.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.514+0000 7ff067fff640 1 -- 172.21.15.111:0/3843705991 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7ff070057df0 con 0x7ff0800cd750 2024-09-13T17:57:44.517 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:44.517 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:30.065235+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:44.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.516+0000 7ff085879640 1 -- 172.21.15.111:0/3843705991 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff06c06ac00 msgr2=0x7ff06c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:44.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.516+0000 7ff085879640 1 --2- 172.21.15.111:0/3843705991 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff06c06ac00 0x7ff06c06d0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff080098fb0 tx=0x7ff074009190 comp rx=0 tx=0).stop 2024-09-13T17:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.516+0000 7ff085879640 1 -- 172.21.15.111:0/3843705991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0800cd750 msgr2=0x7ff080153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.516+0000 7ff085879640 1 --2- 172.21.15.111:0/3843705991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0800cd750 0x7ff080153980 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff07000e9b0 tx=0x7ff07000ee70 comp rx=0 tx=0).stop 2024-09-13T17:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.516+0000 7ff085879640 1 -- 172.21.15.111:0/3843705991 shutdown_connections 2024-09-13T17:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.516+0000 7ff085879640 1 --2- 172.21.15.111:0/3843705991 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff06c06ac00 0x7ff06c06d0c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.516+0000 7ff085879640 1 --2- 172.21.15.111:0/3843705991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff08015c8a0 0x7ff08015b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.516+0000 7ff085879640 1 --2- 172.21.15.111:0/3843705991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff08015c4c0 0x7ff08015ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.516+0000 7ff085879640 1 --2- 172.21.15.111:0/3843705991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0800cd750 0x7ff080153980 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.516+0000 7ff085879640 1 -- 172.21.15.111:0/3843705991 >> 172.21.15.111:0/3843705991 conn(0x7ff0800989d0 msgr2=0x7ff080069f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.516+0000 7ff085879640 1 -- 172.21.15.111:0/3843705991 shutdown_connections 2024-09-13T17:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.516+0000 7ff085879640 1 -- 172.21.15.111:0/3843705991 wait complete. 2024-09-13T17:57:44.530 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:44.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.624+0000 7f98f02d7640 1 Processor -- start 2024-09-13T17:57:44.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.624+0000 7f98f02d7640 1 -- start start 2024-09-13T17:57:44.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.625+0000 7f98f02d7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e8071d30 0x7f98e8070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.625+0000 7f98f02d7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f98e80c8920 0x7f98e80c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.625+0000 7f98f02d7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e80abdf0 con 0x7f98e8072110 2024-09-13T17:57:44.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.625+0000 7f98f02d7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e80c9310 con 0x7f98e80c8920 2024-09-13T17:57:44.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.625+0000 7f98f02d7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80b2280 con 0x7f98e8071d30 2024-09-13T17:57:44.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.625+0000 7f98ee874640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f98e80c8920 0x7f98e80c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.625+0000 7f98ee073640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e8071d30 0x7f98e8070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.625+0000 7f98ee874640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f98e80c8920 0x7f98e80c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:48840/0 (socket says 172.21.15.111:48840) 2024-09-13T17:57:44.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.625+0000 7f98ee874640 1 -- 172.21.15.111:0/401009724 learned_addr learned my addr 172.21.15.111:0/401009724 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:44.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.625+0000 7f98ed872640 1 --1- 172.21.15.111:0/401009724 >> v1:172.21.15.188:6789/0 conn(0x7f98e8072110 0x7f98e80b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:45594/0 (socket says 172.21.15.111:45594) 2024-09-13T17:57:44.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.625+0000 7f98ee073640 1 -- 172.21.15.111:0/401009724 >> v2:172.21.15.188:3300/0 conn(0x7f98e80c8920 msgr2=0x7f98e80c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:44.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.625+0000 7f98ee073640 1 --2- 172.21.15.111:0/401009724 >> v2:172.21.15.188:3300/0 conn(0x7f98e80c8920 0x7f98e80c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.625+0000 7f98ee073640 1 -- 172.21.15.111:0/401009724 >> v1:172.21.15.188:6789/0 conn(0x7f98e8072110 legacy=0x7f98e80b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:44.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.625+0000 7f98ee073640 1 -- 172.21.15.111:0/401009724 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e80ab1e0 con 0x7f98e8071d30 2024-09-13T17:57:44.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.625+0000 7f98ee874640 1 --2- 172.21.15.111:0/401009724 >> v2:172.21.15.188:3300/0 conn(0x7f98e80c8920 0x7f98e80c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:44.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.626+0000 7f98ee073640 1 --2- 172.21.15.111:0/401009724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e8071d30 0x7f98e8070190 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f98d80097d0 tx=0x7f98d802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6047ea95a1d5fab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:44.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.626+0000 7f98ed071640 1 -- 172.21.15.111:0/401009724 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8032440 con 0x7f98e8071d30 2024-09-13T17:57:44.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.626+0000 7f98ed071640 1 -- 172.21.15.111:0/401009724 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d80325a0 con 0x7f98e8071d30 2024-09-13T17:57:44.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.626+0000 7f98ed071640 1 -- 172.21.15.111:0/401009724 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8032930 con 0x7f98e8071d30 2024-09-13T17:57:44.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.627+0000 7f98f02d7640 1 -- 172.21.15.111:0/401009724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e8071d30 msgr2=0x7f98e8070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:44.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.627+0000 7f98f02d7640 1 --2- 172.21.15.111:0/401009724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e8071d30 0x7f98e8070190 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f98d80097d0 tx=0x7f98d802a1c0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.627+0000 7f98f02d7640 1 -- 172.21.15.111:0/401009724 shutdown_connections 2024-09-13T17:57:44.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.627+0000 7f98f02d7640 1 --2- 172.21.15.111:0/401009724 >> v2:172.21.15.188:3300/0 conn(0x7f98e80c8920 0x7f98e80c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.627+0000 7f98f02d7640 1 --2- 172.21.15.111:0/401009724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e8071d30 0x7f98e8070190 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.627+0000 7f98f02d7640 1 -- 172.21.15.111:0/401009724 >> 172.21.15.111:0/401009724 conn(0x7f98e8069ed0 msgr2=0x7f98e806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:44.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.627+0000 7f98f02d7640 1 -- 172.21.15.111:0/401009724 shutdown_connections 2024-09-13T17:57:44.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.627+0000 7f98f02d7640 1 -- 172.21.15.111:0/401009724 wait complete. 2024-09-13T17:57:44.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.628+0000 7f98f02d7640 1 Processor -- start 2024-09-13T17:57:44.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.628+0000 7f98f02d7640 1 -- start start 2024-09-13T17:57:44.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.629+0000 7f98f02d7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98e8071d30 0x7f98e80c4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.629+0000 7f98f02d7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98e8072110 0x7f98e8150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.629+0000 7f98f02d7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e80c8920 0x7f98e80c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.629+0000 7f98f02d7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80c2780 con 0x7f98e8072110 2024-09-13T17:57:44.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.629+0000 7f98f02d7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80c1bb0 con 0x7f98e8071d30 2024-09-13T17:57:44.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.629+0000 7f98f02d7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80c4a90 con 0x7f98e80c8920 2024-09-13T17:57:44.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.629+0000 7f98ed872640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98e8072110 0x7f98e8150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.629+0000 7f98ed872640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98e8072110 0x7f98e8150e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53786/0 (socket says 172.21.15.111:53786) 2024-09-13T17:57:44.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.629+0000 7f98ed872640 1 -- 172.21.15.111:0/2946952124 learned_addr learned my addr 172.21.15.111:0/2946952124 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:44.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.629+0000 7f98ee073640 1 --2- 172.21.15.111:0/2946952124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98e8071d30 0x7f98e80c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.629+0000 7f98ee874640 1 --2- 172.21.15.111:0/2946952124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e80c8920 0x7f98e80c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.629+0000 7f98ed872640 1 -- 172.21.15.111:0/2946952124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e80c8920 msgr2=0x7f98e80c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:44.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.629+0000 7f98ed872640 1 --2- 172.21.15.111:0/2946952124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e80c8920 0x7f98e80c2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.629+0000 7f98ed872640 1 -- 172.21.15.111:0/2946952124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98e8071d30 msgr2=0x7f98e80c4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:44.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.629+0000 7f98ed872640 1 --2- 172.21.15.111:0/2946952124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98e8071d30 0x7f98e80c4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.629+0000 7f98ed872640 1 -- 172.21.15.111:0/2946952124 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e000bd40 con 0x7f98e8072110 2024-09-13T17:57:44.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.630+0000 7f98ed872640 1 --2- 172.21.15.111:0/2946952124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98e8072110 0x7f98e8150e60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f98e400c420 tx=0x7f98e400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:44.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.630+0000 7f98deffd640 1 -- 172.21.15.111:0/2946952124 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e40074e0 con 0x7f98e8072110 2024-09-13T17:57:44.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.630+0000 7f98deffd640 1 -- 172.21.15.111:0/2946952124 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e4007640 con 0x7f98e8072110 2024-09-13T17:57:44.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.630+0000 7f98f02d7640 1 -- 172.21.15.111:0/2946952124 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d8009400 con 0x7f98e8072110 2024-09-13T17:57:44.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.630+0000 7f98deffd640 1 -- 172.21.15.111:0/2946952124 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e4004a20 con 0x7f98e8072110 2024-09-13T17:57:44.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.630+0000 7f98f02d7640 1 -- 172.21.15.111:0/2946952124 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e80c2130 con 0x7f98e8072110 2024-09-13T17:57:44.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.631+0000 7f98f02d7640 1 -- 172.21.15.111:0/2946952124 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e80c28f0 con 0x7f98e8072110 2024-09-13T17:57:44.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.631+0000 7f98deffd640 1 -- 172.21.15.111:0/2946952124 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f98e4004b80 con 0x7f98e8072110 2024-09-13T17:57:44.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.631+0000 7f98deffd640 1 --2- 172.21.15.111:0/2946952124 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f98cc06ac00 0x7f98cc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.632+0000 7f98deffd640 1 -- 172.21.15.111:0/2946952124 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f98e4088f20 con 0x7f98e8072110 2024-09-13T17:57:44.636 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.632+0000 7f98ee073640 1 --2- 172.21.15.111:0/2946952124 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f98cc06ac00 0x7f98cc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.636 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.632+0000 7f98ee073640 1 --2- 172.21.15.111:0/2946952124 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f98cc06ac00 0x7f98cc06d0c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f98d8034000 tx=0x7f98d802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:44.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.636+0000 7f98deffd640 1 -- 172.21.15.111:0/2946952124 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f98e4053ec0 con 0x7f98e8072110 2024-09-13T17:57:44.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.798+0000 7f98f02d7640 1 -- 172.21.15.111:0/2946952124 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98e8070190 con 0x7f98e8072110 2024-09-13T17:57:44.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.799+0000 7f98deffd640 1 -- 172.21.15.111:0/2946952124 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f98e40574e0 con 0x7f98e8072110 2024-09-13T17:57:44.804 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:44.805 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:30.065235+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:44.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.801+0000 7f98f02d7640 1 -- 172.21.15.111:0/2946952124 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f98cc06ac00 msgr2=0x7f98cc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:44.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.801+0000 7f98f02d7640 1 --2- 172.21.15.111:0/2946952124 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f98cc06ac00 0x7f98cc06d0c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f98d8034000 tx=0x7f98d802e000 comp rx=0 tx=0).stop 2024-09-13T17:57:44.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.801+0000 7f98f02d7640 1 -- 172.21.15.111:0/2946952124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98e8072110 msgr2=0x7f98e8150e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:44.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.801+0000 7f98f02d7640 1 --2- 172.21.15.111:0/2946952124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98e8072110 0x7f98e8150e60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f98e400c420 tx=0x7f98e400c8e0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.801+0000 7f98f02d7640 1 -- 172.21.15.111:0/2946952124 shutdown_connections 2024-09-13T17:57:44.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.801+0000 7f98f02d7640 1 --2- 172.21.15.111:0/2946952124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e80c8920 0x7f98e80c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.801+0000 7f98f02d7640 1 --2- 172.21.15.111:0/2946952124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98e8072110 0x7f98e8150e60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.801+0000 7f98f02d7640 1 --2- 172.21.15.111:0/2946952124 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f98cc06ac00 0x7f98cc06d0c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.801+0000 7f98f02d7640 1 --2- 172.21.15.111:0/2946952124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98e8071d30 0x7f98e80c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.801+0000 7f98f02d7640 1 -- 172.21.15.111:0/2946952124 >> 172.21.15.111:0/2946952124 conn(0x7f98e8069ed0 msgr2=0x7f98e80adcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:44.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.801+0000 7f98f02d7640 1 -- 172.21.15.111:0/2946952124 shutdown_connections 2024-09-13T17:57:44.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.802+0000 7f98f02d7640 1 -- 172.21.15.111:0/2946952124 wait complete. 2024-09-13T17:57:44.814 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T17:57:44.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.912+0000 7f1e9c02b640 1 Processor -- start 2024-09-13T17:57:44.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.912+0000 7f1e9c02b640 1 -- start start 2024-09-13T17:57:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.913+0000 7f1e9c02b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1e94159310 0x7f1e940ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.913+0000 7f1e9c02b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e940ccbe0 con 0x7f1e940ce4f0 2024-09-13T17:57:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.913+0000 7f1e9c02b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e940b1810 con 0x7f1e940b2b40 2024-09-13T17:57:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.913+0000 7f1e9c02b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e940ad940 con 0x7f1e94159310 2024-09-13T17:57:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.913+0000 7f1e99dc7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1e94159310 0x7f1e940ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.913+0000 7f1e9a5c8640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1e940ce4f0 0x7f1e940ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49626/0 (socket says 172.21.15.111:49626) 2024-09-13T17:57:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.913+0000 7f1e9a5c8640 1 -- 172.21.15.111:0/4023180083 learned_addr learned my addr 172.21.15.111:0/4023180083 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.913+0000 7f1e99dc7640 1 -- 172.21.15.111:0/4023180083 >> v1:172.21.15.188:6789/0 conn(0x7f1e940b2b40 legacy=0x7f1e940ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.913+0000 7f1e99dc7640 1 -- 172.21.15.111:0/4023180083 >> v1:172.21.15.111:6789/0 conn(0x7f1e940ce4f0 legacy=0x7f1e940ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.913+0000 7f1e99dc7640 1 -- 172.21.15.111:0/4023180083 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9406d9e0 con 0x7f1e94159310 2024-09-13T17:57:44.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.914+0000 7f1e99dc7640 1 --2- 172.21.15.111:0/4023180083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e94159310 0x7f1e940ad400 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1e900097a0 tx=0x7f1e9002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b1c2061bae8302 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:44.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.914+0000 7f1e98dc5640 1 -- 172.21.15.111:0/4023180083 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9002ae50 con 0x7f1e94159310 2024-09-13T17:57:44.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.914+0000 7f1e98dc5640 1 -- 172.21.15.111:0/4023180083 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e90032440 con 0x7f1e94159310 2024-09-13T17:57:44.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.914+0000 7f1e98dc5640 1 -- 172.21.15.111:0/4023180083 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e900327d0 con 0x7f1e94159310 2024-09-13T17:57:44.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.914+0000 7f1e9c02b640 1 -- 172.21.15.111:0/4023180083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e94159310 msgr2=0x7f1e940ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:44.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.914+0000 7f1e9c02b640 1 --2- 172.21.15.111:0/4023180083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e94159310 0x7f1e940ad400 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1e900097a0 tx=0x7f1e9002a140 comp rx=0 tx=0).stop 2024-09-13T17:57:44.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.914+0000 7f1e9c02b640 1 -- 172.21.15.111:0/4023180083 shutdown_connections 2024-09-13T17:57:44.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.914+0000 7f1e9c02b640 1 --2- 172.21.15.111:0/4023180083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e94159310 0x7f1e940ad400 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.914+0000 7f1e9c02b640 1 -- 172.21.15.111:0/4023180083 >> 172.21.15.111:0/4023180083 conn(0x7f1e9406a0b0 msgr2=0x7f1e9406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:44.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.915+0000 7f1e9c02b640 1 -- 172.21.15.111:0/4023180083 shutdown_connections 2024-09-13T17:57:44.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.915+0000 7f1e9c02b640 1 -- 172.21.15.111:0/4023180083 wait complete. 2024-09-13T17:57:44.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.916+0000 7f1e9c02b640 1 Processor -- start 2024-09-13T17:57:44.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.916+0000 7f1e9c02b640 1 -- start start 2024-09-13T17:57:44.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.916+0000 7f1e9c02b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e940b2b40 0x7f1e94157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.916+0000 7f1e9c02b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e940ce4f0 0x7f1e9409bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.916+0000 7f1e9c02b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e94159310 0x7f1e940cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.916+0000 7f1e9c02b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94158410 con 0x7f1e940b2b40 2024-09-13T17:57:44.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.916+0000 7f1e9c02b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94099640 con 0x7f1e94159310 2024-09-13T17:57:44.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.916+0000 7f1e9c02b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94099920 con 0x7f1e940ce4f0 2024-09-13T17:57:44.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.916+0000 7f1e99dc7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e940b2b40 0x7f1e94157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.917+0000 7f1e99dc7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e940b2b40 0x7f1e94157ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53802/0 (socket says 172.21.15.111:53802) 2024-09-13T17:57:44.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.917+0000 7f1e99dc7640 1 -- 172.21.15.111:0/3333317627 learned_addr learned my addr 172.21.15.111:0/3333317627 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:44.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.917+0000 7f1e9a5c8640 1 --2- 172.21.15.111:0/3333317627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e94159310 0x7f1e940cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.917+0000 7f1e995c6640 1 --2- 172.21.15.111:0/3333317627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e940ce4f0 0x7f1e9409bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.917+0000 7f1e99dc7640 1 -- 172.21.15.111:0/3333317627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e940ce4f0 msgr2=0x7f1e9409bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:44.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.917+0000 7f1e99dc7640 1 --2- 172.21.15.111:0/3333317627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e940ce4f0 0x7f1e9409bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.917+0000 7f1e99dc7640 1 -- 172.21.15.111:0/3333317627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e94159310 msgr2=0x7f1e940cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:44.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.917+0000 7f1e99dc7640 1 --2- 172.21.15.111:0/3333317627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e94159310 0x7f1e940cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:44.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.917+0000 7f1e99dc7640 1 -- 172.21.15.111:0/3333317627 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e90009400 con 0x7f1e940b2b40 2024-09-13T17:57:44.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.917+0000 7f1e99dc7640 1 --2- 172.21.15.111:0/3333317627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e940b2b40 0x7f1e94157ed0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1e90000c00 tx=0x7f1e900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:44.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.917+0000 7f1e8a7fc640 1 -- 172.21.15.111:0/3333317627 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9003d910 con 0x7f1e940b2b40 2024-09-13T17:57:44.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.917+0000 7f1e8a7fc640 1 -- 172.21.15.111:0/3333317627 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9003da70 con 0x7f1e940b2b40 2024-09-13T17:57:44.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.917+0000 7f1e8a7fc640 1 -- 172.21.15.111:0/3333317627 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9003de20 con 0x7f1e940b2b40 2024-09-13T17:57:44.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.917+0000 7f1e9c02b640 1 -- 172.21.15.111:0/3333317627 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e94099c00 con 0x7f1e940b2b40 2024-09-13T17:57:44.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.917+0000 7f1e9c02b640 1 -- 172.21.15.111:0/3333317627 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9409a1c0 con 0x7f1e940b2b40 2024-09-13T17:57:44.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.918+0000 7f1e9c02b640 1 -- 172.21.15.111:0/3333317627 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e940ad400 con 0x7f1e940b2b40 2024-09-13T17:57:44.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.919+0000 7f1e8a7fc640 1 -- 172.21.15.111:0/3333317627 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1e90039040 con 0x7f1e940b2b40 2024-09-13T17:57:44.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.919+0000 7f1e8a7fc640 1 --2- 172.21.15.111:0/3333317627 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1e7806ac00 0x7f1e7806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:44.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.919+0000 7f1e8a7fc640 1 -- 172.21.15.111:0/3333317627 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1e90032e40 con 0x7f1e940b2b40 2024-09-13T17:57:44.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.919+0000 7f1e995c6640 1 --2- 172.21.15.111:0/3333317627 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1e7806ac00 0x7f1e7806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:44.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.919+0000 7f1e995c6640 1 --2- 172.21.15.111:0/3333317627 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1e7806ac00 0x7f1e7806d0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1e940cbf10 tx=0x7f1e84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:44.925 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:44.923+0000 7f1e8a7fc640 1 -- 172.21.15.111:0/3333317627 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f1e90074ef0 con 0x7f1e940b2b40 2024-09-13T17:57:45.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.116+0000 7f1e9c02b640 1 -- 172.21.15.111:0/3333317627 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1e940cd420 con 0x7f1e940b2b40 2024-09-13T17:57:45.353 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.351+0000 7f1e8a7fc640 1 -- 172.21.15.111:0/3333317627 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1e900410b0 con 0x7f1e940b2b40 2024-09-13T17:57:45.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.353+0000 7f1e9c02b640 1 -- 172.21.15.111:0/3333317627 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1e7806ac00 msgr2=0x7f1e7806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:45.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.353+0000 7f1e9c02b640 1 --2- 172.21.15.111:0/3333317627 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1e7806ac00 0x7f1e7806d0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1e940cbf10 tx=0x7f1e84009040 comp rx=0 tx=0).stop 2024-09-13T17:57:45.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.354+0000 7f1e9c02b640 1 -- 172.21.15.111:0/3333317627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e940b2b40 msgr2=0x7f1e94157ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:45.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.354+0000 7f1e9c02b640 1 --2- 172.21.15.111:0/3333317627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e940b2b40 0x7f1e94157ed0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1e90000c00 tx=0x7f1e900046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:45.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.354+0000 7f1e9c02b640 1 -- 172.21.15.111:0/3333317627 shutdown_connections 2024-09-13T17:57:45.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.354+0000 7f1e9c02b640 1 --2- 172.21.15.111:0/3333317627 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1e7806ac00 0x7f1e7806d0c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:45.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.354+0000 7f1e9c02b640 1 --2- 172.21.15.111:0/3333317627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e94159310 0x7f1e940cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:45.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.354+0000 7f1e9c02b640 1 --2- 172.21.15.111:0/3333317627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e940ce4f0 0x7f1e9409bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:45.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.354+0000 7f1e9c02b640 1 --2- 172.21.15.111:0/3333317627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e940b2b40 0x7f1e94157ed0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:45.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.354+0000 7f1e9c02b640 1 -- 172.21.15.111:0/3333317627 >> 172.21.15.111:0/3333317627 conn(0x7f1e9406a0b0 msgr2=0x7f1e940aebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:45.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.354+0000 7f1e9c02b640 1 -- 172.21.15.111:0/3333317627 shutdown_connections 2024-09-13T17:57:45.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.354+0000 7f1e9c02b640 1 -- 172.21.15.111:0/3333317627 wait complete. 2024-09-13T17:57:45.366 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-09-13T17:57:45.367 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T17:57:45.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.467+0000 7f2211565640 1 Processor -- start 2024-09-13T17:57:45.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.467+0000 7f2211565640 1 -- start start 2024-09-13T17:57:45.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.468+0000 7f2211565640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f220c15fbc0 0x7f220c0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:45.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.468+0000 7f2211565640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f220c0cd6c0 0x7f220c0d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:45.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.468+0000 7f2211565640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f220c06b6c0 con 0x7f220c0d2810 2024-09-13T17:57:45.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.468+0000 7f2211565640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220c0ce500 con 0x7f220c0cd6c0 2024-09-13T17:57:45.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.468+0000 7f2211565640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220c0d2c80 con 0x7f220c15fbc0 2024-09-13T17:57:45.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.468+0000 7f220a7fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f220c0cd6c0 0x7f220c0d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:45.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.468+0000 7f220a7fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f220c0cd6c0 0x7f220c0d22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53806/0 (socket says 172.21.15.111:53806) 2024-09-13T17:57:45.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.468+0000 7f220a7fc640 1 -- 172.21.15.111:0/3510246130 learned_addr learned my addr 172.21.15.111:0/3510246130 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:45.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.468+0000 7f220affd640 1 --2- 172.21.15.111:0/3510246130 >> v2:172.21.15.188:3300/0 conn(0x7f220c15fbc0 0x7f220c0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:45.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.468+0000 7f220a7fc640 1 -- 172.21.15.111:0/3510246130 >> v2:172.21.15.188:3300/0 conn(0x7f220c15fbc0 msgr2=0x7f220c0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:45.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.468+0000 7f220a7fc640 1 --2- 172.21.15.111:0/3510246130 >> v2:172.21.15.188:3300/0 conn(0x7f220c15fbc0 0x7f220c0cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:45.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.468+0000 7f220a7fc640 1 -- 172.21.15.111:0/3510246130 >> v1:172.21.15.188:6789/0 conn(0x7f220c0d2810 legacy=0x7f220c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:45.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.468+0000 7f220a7fc640 1 -- 172.21.15.111:0/3510246130 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c0cdc60 con 0x7f220c0cd6c0 2024-09-13T17:57:45.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.469+0000 7f220a7fc640 1 --2- 172.21.15.111:0/3510246130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0cd6c0 0x7f220c0d22d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f220c08c4c0 tx=0x7f21f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e53197d0a395b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:45.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.469+0000 7f2209ffb640 1 -- 172.21.15.111:0/3510246130 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4004030 con 0x7f220c0cd6c0 2024-09-13T17:57:45.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.469+0000 7f2209ffb640 1 -- 172.21.15.111:0/3510246130 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f4004190 con 0x7f220c0cd6c0 2024-09-13T17:57:45.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.469+0000 7f2209ffb640 1 -- 172.21.15.111:0/3510246130 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4003610 con 0x7f220c0cd6c0 2024-09-13T17:57:45.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.470+0000 7f2211565640 1 -- 172.21.15.111:0/3510246130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0cd6c0 msgr2=0x7f220c0d22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:45.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.470+0000 7f2211565640 1 --2- 172.21.15.111:0/3510246130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0cd6c0 0x7f220c0d22d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f220c08c4c0 tx=0x7f21f402a270 comp rx=0 tx=0).stop 2024-09-13T17:57:45.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.470+0000 7f2211565640 1 -- 172.21.15.111:0/3510246130 shutdown_connections 2024-09-13T17:57:45.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.470+0000 7f2211565640 1 --2- 172.21.15.111:0/3510246130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0cd6c0 0x7f220c0d22d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:45.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.470+0000 7f2211565640 1 --2- 172.21.15.111:0/3510246130 >> v2:172.21.15.188:3300/0 conn(0x7f220c15fbc0 0x7f220c0cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:45.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.470+0000 7f2211565640 1 -- 172.21.15.111:0/3510246130 >> 172.21.15.111:0/3510246130 conn(0x7f220c09b830 msgr2=0x7f220c089f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:45.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.470+0000 7f2211565640 1 -- 172.21.15.111:0/3510246130 shutdown_connections 2024-09-13T17:57:45.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.470+0000 7f2211565640 1 -- 172.21.15.111:0/3510246130 wait complete. 2024-09-13T17:57:45.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.471+0000 7f2211565640 1 Processor -- start 2024-09-13T17:57:45.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.471+0000 7f2211565640 1 -- start start 2024-09-13T17:57:45.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.472+0000 7f2211565640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220c0cd6c0 0x7f220c15c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:45.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.472+0000 7f2211565640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0d2810 0x7f220c0b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:45.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.472+0000 7f2211565640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f220c15fbc0 0x7f220c159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:45.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.472+0000 7f2211565640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220c0b4a80 con 0x7f220c0d2810 2024-09-13T17:57:45.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.472+0000 7f2211565640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220c15c140 con 0x7f220c15fbc0 2024-09-13T17:57:45.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.472+0000 7f2211565640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220c15b810 con 0x7f220c0cd6c0 2024-09-13T17:57:45.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.472+0000 7f220a7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0d2810 0x7f220c0b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:45.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.472+0000 7f220a7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0d2810 0x7f220c0b4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53816/0 (socket says 172.21.15.111:53816) 2024-09-13T17:57:45.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.472+0000 7f220a7fc640 1 -- 172.21.15.111:0/3668844094 learned_addr learned my addr 172.21.15.111:0/3668844094 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:45.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.472+0000 7f220affd640 1 --2- 172.21.15.111:0/3668844094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220c0cd6c0 0x7f220c15c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:45.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.472+0000 7f220b7fe640 1 --2- 172.21.15.111:0/3668844094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f220c15fbc0 0x7f220c159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:45.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.472+0000 7f220a7fc640 1 -- 172.21.15.111:0/3668844094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220c0cd6c0 msgr2=0x7f220c15c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:45.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.472+0000 7f220a7fc640 1 --2- 172.21.15.111:0/3668844094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220c0cd6c0 0x7f220c15c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:45.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.472+0000 7f220a7fc640 1 -- 172.21.15.111:0/3668844094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f220c15fbc0 msgr2=0x7f220c159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:45.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.472+0000 7f220a7fc640 1 --2- 172.21.15.111:0/3668844094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f220c15fbc0 0x7f220c159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:45.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.472+0000 7f220a7fc640 1 -- 172.21.15.111:0/3668844094 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f4009460 con 0x7f220c0d2810 2024-09-13T17:57:45.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.473+0000 7f220a7fc640 1 --2- 172.21.15.111:0/3668844094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0d2810 0x7f220c0b4540 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f21f4034040 tx=0x7f21f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:45.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.473+0000 7f22037fe640 1 -- 172.21.15.111:0/3668844094 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4004030 con 0x7f220c0d2810 2024-09-13T17:57:45.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.473+0000 7f22037fe640 1 -- 172.21.15.111:0/3668844094 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f4003bd0 con 0x7f220c0d2810 2024-09-13T17:57:45.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.473+0000 7f2211565640 1 -- 172.21.15.111:0/3668844094 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f220c15baf0 con 0x7f220c0d2810 2024-09-13T17:57:45.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.473+0000 7f22037fe640 1 -- 172.21.15.111:0/3668844094 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4006040 con 0x7f220c0d2810 2024-09-13T17:57:45.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.473+0000 7f2211565640 1 -- 172.21.15.111:0/3668844094 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f220c15b430 con 0x7f220c0d2810 2024-09-13T17:57:45.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.474+0000 7f2211565640 1 -- 172.21.15.111:0/3668844094 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f220c0af090 con 0x7f220c0d2810 2024-09-13T17:57:45.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.474+0000 7f22037fe640 1 -- 172.21.15.111:0/3668844094 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f21f40061a0 con 0x7f220c0d2810 2024-09-13T17:57:45.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.475+0000 7f22037fe640 1 --2- 172.21.15.111:0/3668844094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f21ec06ac00 0x7f21ec06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:45.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.475+0000 7f220affd640 1 --2- 172.21.15.111:0/3668844094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f21ec06ac00 0x7f21ec06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:45.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.475+0000 7f22037fe640 1 -- 172.21.15.111:0/3668844094 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f21f40aa570 con 0x7f220c0d2810 2024-09-13T17:57:45.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.476+0000 7f220affd640 1 --2- 172.21.15.111:0/3668844094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f21ec06ac00 0x7f21ec06d0c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2204003b80 tx=0x7f22040098a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:45.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.479+0000 7f22037fe640 1 -- 172.21.15.111:0/3668844094 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f21f4075510 con 0x7f220c0d2810 2024-09-13T17:57:45.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:45.681+0000 7f2211565640 1 -- 172.21.15.111:0/3668844094 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f220c1590f0 con 0x7f220c0d2810 2024-09-13T17:57:46.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.355+0000 7f22037fe640 1 -- 172.21.15.111:0/3668844094 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f21f4078b30 con 0x7f220c0d2810 2024-09-13T17:57:46.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.357+0000 7f2211565640 1 -- 172.21.15.111:0/3668844094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f21ec06ac00 msgr2=0x7f21ec06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:46.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.357+0000 7f2211565640 1 --2- 172.21.15.111:0/3668844094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f21ec06ac00 0x7f21ec06d0c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2204003b80 tx=0x7f22040098a0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.357+0000 7f2211565640 1 -- 172.21.15.111:0/3668844094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0d2810 msgr2=0x7f220c0b4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:46.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.357+0000 7f2211565640 1 --2- 172.21.15.111:0/3668844094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0d2810 0x7f220c0b4540 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f21f4034040 tx=0x7f21f40046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.357+0000 7f2211565640 1 -- 172.21.15.111:0/3668844094 shutdown_connections 2024-09-13T17:57:46.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.357+0000 7f2211565640 1 --2- 172.21.15.111:0/3668844094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f21ec06ac00 0x7f21ec06d0c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.357+0000 7f2211565640 1 --2- 172.21.15.111:0/3668844094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f220c15fbc0 0x7f220c159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.357+0000 7f2211565640 1 --2- 172.21.15.111:0/3668844094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0d2810 0x7f220c0b4540 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.357+0000 7f2211565640 1 --2- 172.21.15.111:0/3668844094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220c0cd6c0 0x7f220c15c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.357+0000 7f2211565640 1 -- 172.21.15.111:0/3668844094 >> 172.21.15.111:0/3668844094 conn(0x7f220c09b830 msgr2=0x7f220c0d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:46.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.357+0000 7f2211565640 1 -- 172.21.15.111:0/3668844094 shutdown_connections 2024-09-13T17:57:46.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.357+0000 7f2211565640 1 -- 172.21.15.111:0/3668844094 wait complete. 2024-09-13T17:57:46.371 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T17:57:46.371 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:57:46.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.467+0000 7f7c5b3c7640 1 Processor -- start 2024-09-13T17:57:46.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.467+0000 7f7c5b3c7640 1 -- start start 2024-09-13T17:57:46.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.468+0000 7f7c5b3c7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7c5415a7a0 0x7f7c540ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:46.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.468+0000 7f7c5b3c7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7c540caf30 0x7f7c540cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:46.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.468+0000 7f7c5b3c7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c540ad660 0x7f7c540adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:46.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.468+0000 7f7c5b3c7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c540af8e0 con 0x7f7c5415a7a0 2024-09-13T17:57:46.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.468+0000 7f7c5b3c7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c540ad1d0 con 0x7f7c540caf30 2024-09-13T17:57:46.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.468+0000 7f7c5b3c7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c540ce6f0 con 0x7f7c540ad660 2024-09-13T17:57:46.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.468+0000 7f7c59163640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7c5415a7a0 0x7f7c540ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:46.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.468+0000 7f7c59163640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7c5415a7a0 0x7f7c540ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53824/0 (socket says 172.21.15.111:53824) 2024-09-13T17:57:46.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.468+0000 7f7c59163640 1 -- 172.21.15.111:0/1660091606 learned_addr learned my addr 172.21.15.111:0/1660091606 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:46.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.468+0000 7f7c58962640 1 --2- 172.21.15.111:0/1660091606 >> v2:172.21.15.188:3300/0 conn(0x7f7c540caf30 0x7f7c540cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:46.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.468+0000 7f7c59163640 1 -- 172.21.15.111:0/1660091606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c540ad660 msgr2=0x7f7c540adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:46.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.469+0000 7f7c59163640 1 --2- 172.21.15.111:0/1660091606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c540ad660 0x7f7c540adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.469+0000 7f7c59163640 1 -- 172.21.15.111:0/1660091606 >> v2:172.21.15.188:3300/0 conn(0x7f7c540caf30 msgr2=0x7f7c540cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:46.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.469+0000 7f7c59163640 1 --2- 172.21.15.111:0/1660091606 >> v2:172.21.15.188:3300/0 conn(0x7f7c540caf30 0x7f7c540cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.469+0000 7f7c59163640 1 -- 172.21.15.111:0/1660091606 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c540cbc00 con 0x7f7c5415a7a0 2024-09-13T17:57:46.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.469+0000 7f7c59163640 1 --2- 172.21.15.111:0/1660091606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c5415a7a0 0x7f7c540ce060 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7c5415e050 tx=0x7f7c5002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e659cb5e8b2b98 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:46.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.469+0000 7f7c47fff640 1 -- 172.21.15.111:0/1660091606 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50032440 con 0x7f7c5415a7a0 2024-09-13T17:57:46.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.469+0000 7f7c47fff640 1 -- 172.21.15.111:0/1660091606 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c500325a0 con 0x7f7c5415a7a0 2024-09-13T17:57:46.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.469+0000 7f7c47fff640 1 -- 172.21.15.111:0/1660091606 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50032930 con 0x7f7c5415a7a0 2024-09-13T17:57:46.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.470+0000 7f7c5b3c7640 1 -- 172.21.15.111:0/1660091606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c5415a7a0 msgr2=0x7f7c540ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:46.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.470+0000 7f7c5b3c7640 1 --2- 172.21.15.111:0/1660091606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c5415a7a0 0x7f7c540ce060 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7c5415e050 tx=0x7f7c5002a1e0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.470+0000 7f7c5b3c7640 1 -- 172.21.15.111:0/1660091606 shutdown_connections 2024-09-13T17:57:46.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.470+0000 7f7c5b3c7640 1 --2- 172.21.15.111:0/1660091606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c540ad660 0x7f7c540adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.470+0000 7f7c5b3c7640 1 --2- 172.21.15.111:0/1660091606 >> v2:172.21.15.188:3300/0 conn(0x7f7c540caf30 0x7f7c540cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.470+0000 7f7c5b3c7640 1 --2- 172.21.15.111:0/1660091606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c5415a7a0 0x7f7c540ce060 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.470+0000 7f7c5b3c7640 1 -- 172.21.15.111:0/1660091606 >> 172.21.15.111:0/1660091606 conn(0x7f7c540b1f80 msgr2=0x7f7c540b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:46.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.470+0000 7f7c5b3c7640 1 -- 172.21.15.111:0/1660091606 shutdown_connections 2024-09-13T17:57:46.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.470+0000 7f7c5b3c7640 1 -- 172.21.15.111:0/1660091606 wait complete. 2024-09-13T17:57:46.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.472+0000 7f7c5b3c7640 1 Processor -- start 2024-09-13T17:57:46.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.472+0000 7f7c5b3c7640 1 -- start start 2024-09-13T17:57:46.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.473+0000 7f7c5b3c7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c540ad660 0x7f7c540b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:46.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.473+0000 7f7c5b3c7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c540caf30 0x7f7c54150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:46.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.473+0000 7f7c5b3c7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c5415a7a0 0x7f7c541524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:46.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.473+0000 7f7c5b3c7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c540b4d50 con 0x7f7c5415a7a0 2024-09-13T17:57:46.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.473+0000 7f7c5b3c7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c540c9570 con 0x7f7c540ad660 2024-09-13T17:57:46.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.473+0000 7f7c5b3c7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c540b40d0 con 0x7f7c540caf30 2024-09-13T17:57:46.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.473+0000 7f7c59964640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c5415a7a0 0x7f7c541524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:46.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.473+0000 7f7c59964640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c5415a7a0 0x7f7c541524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53832/0 (socket says 172.21.15.111:53832) 2024-09-13T17:57:46.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.473+0000 7f7c59964640 1 -- 172.21.15.111:0/625493751 learned_addr learned my addr 172.21.15.111:0/625493751 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:46.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.474+0000 7f7c59163640 1 --2- 172.21.15.111:0/625493751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c540ad660 0x7f7c540b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:46.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.474+0000 7f7c58962640 1 --2- 172.21.15.111:0/625493751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c540caf30 0x7f7c54150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:46.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.474+0000 7f7c59964640 1 -- 172.21.15.111:0/625493751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c540caf30 msgr2=0x7f7c54150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:46.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.474+0000 7f7c59964640 1 --2- 172.21.15.111:0/625493751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c540caf30 0x7f7c54150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.474+0000 7f7c59964640 1 -- 172.21.15.111:0/625493751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c540ad660 msgr2=0x7f7c540b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:46.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.474+0000 7f7c59964640 1 --2- 172.21.15.111:0/625493751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c540ad660 0x7f7c540b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.474+0000 7f7c59964640 1 -- 172.21.15.111:0/625493751 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c500094d0 con 0x7f7c5415a7a0 2024-09-13T17:57:46.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.474+0000 7f7c59964640 1 --2- 172.21.15.111:0/625493751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c5415a7a0 0x7f7c541524a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00b970 tx=0x7f7c4c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:46.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.474+0000 7f7c45ffb640 1 -- 172.21.15.111:0/625493751 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c00cb40 con 0x7f7c5415a7a0 2024-09-13T17:57:46.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.474+0000 7f7c45ffb640 1 -- 172.21.15.111:0/625493751 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4c00ccc0 con 0x7f7c5415a7a0 2024-09-13T17:57:46.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.474+0000 7f7c5b3c7640 1 -- 172.21.15.111:0/625493751 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c540b43b0 con 0x7f7c5415a7a0 2024-09-13T17:57:46.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.474+0000 7f7c45ffb640 1 -- 172.21.15.111:0/625493751 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c014680 con 0x7f7c5415a7a0 2024-09-13T17:57:46.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.474+0000 7f7c5b3c7640 1 -- 172.21.15.111:0/625493751 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c540b4970 con 0x7f7c5415a7a0 2024-09-13T17:57:46.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.475+0000 7f7c5b3c7640 1 -- 172.21.15.111:0/625493751 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c540ccd10 con 0x7f7c5415a7a0 2024-09-13T17:57:46.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.476+0000 7f7c45ffb640 1 -- 172.21.15.111:0/625493751 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7c4c01d3b0 con 0x7f7c5415a7a0 2024-09-13T17:57:46.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.477+0000 7f7c45ffb640 1 --2- 172.21.15.111:0/625493751 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7c3806ac00 0x7f7c3806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:46.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.477+0000 7f7c45ffb640 1 -- 172.21.15.111:0/625493751 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7c4c0894f0 con 0x7f7c5415a7a0 2024-09-13T17:57:46.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.477+0000 7f7c59163640 1 --2- 172.21.15.111:0/625493751 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7c3806ac00 0x7f7c3806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:46.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.477+0000 7f7c59163640 1 --2- 172.21.15.111:0/625493751 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7c3806ac00 0x7f7c3806d0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7c50034000 tx=0x7f7c5002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:46.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.481+0000 7f7c45ffb640 1 -- 172.21.15.111:0/625493751 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f7c4c054490 con 0x7f7c5415a7a0 2024-09-13T17:57:46.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.680+0000 7f7c5b3c7640 1 -- 172.21.15.111:0/625493751 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c540b4e90 con 0x7f7c5415a7a0 2024-09-13T17:57:46.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.681+0000 7f7c45ffb640 1 -- 172.21.15.111:0/625493751 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4742 (secure 0 0 0) 0x7f7c4c057ab0 con 0x7f7c5415a7a0 2024-09-13T17:57:46.684 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:46.684 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4001406804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":4001406804},{"type":"v1","addr":"172.21.15.188:6835","nonce":4001406804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2565466103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2565466103},{"type":"v1","addr":"172.21.15.111:6837","nonce":2565466103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/1252690799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1252690799},{"type":"v1","addr":"172.21.15.111:6836","nonce":1252690799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:57:25.702997+0000","modified":"2024-09-13T17:57:46.351493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/1266310843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1266310843},{"type":"v1","addr":"172.21.15.188:6834","nonce":1266310843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:57:46.684 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-09-13T17:57:46.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.683+0000 7f7c5b3c7640 1 -- 172.21.15.111:0/625493751 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7c3806ac00 msgr2=0x7f7c3806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:46.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.683+0000 7f7c5b3c7640 1 --2- 172.21.15.111:0/625493751 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7c3806ac00 0x7f7c3806d0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7c50034000 tx=0x7f7c5002e000 comp rx=0 tx=0).stop 2024-09-13T17:57:46.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.683+0000 7f7c5b3c7640 1 -- 172.21.15.111:0/625493751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c5415a7a0 msgr2=0x7f7c541524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:46.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.683+0000 7f7c5b3c7640 1 --2- 172.21.15.111:0/625493751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c5415a7a0 0x7f7c541524a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00b970 tx=0x7f7c4c00be30 comp rx=0 tx=0).stop 2024-09-13T17:57:46.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.683+0000 7f7c5b3c7640 1 -- 172.21.15.111:0/625493751 shutdown_connections 2024-09-13T17:57:46.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.683+0000 7f7c5b3c7640 1 --2- 172.21.15.111:0/625493751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c5415a7a0 0x7f7c541524a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.683+0000 7f7c5b3c7640 1 --2- 172.21.15.111:0/625493751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c540caf30 0x7f7c54150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.683+0000 7f7c5b3c7640 1 --2- 172.21.15.111:0/625493751 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7c3806ac00 0x7f7c3806d0c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.683+0000 7f7c5b3c7640 1 --2- 172.21.15.111:0/625493751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c540ad660 0x7f7c540b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.683+0000 7f7c5b3c7640 1 -- 172.21.15.111:0/625493751 >> 172.21.15.111:0/625493751 conn(0x7f7c540b1f80 msgr2=0x7f7c540afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:46.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.683+0000 7f7c5b3c7640 1 -- 172.21.15.111:0/625493751 shutdown_connections 2024-09-13T17:57:46.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.683+0000 7f7c5b3c7640 1 -- 172.21.15.111:0/625493751 wait complete. 2024-09-13T17:57:46.697 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:46.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.791+0000 7f080650f640 1 Processor -- start 2024-09-13T17:57:46.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.791+0000 7f080650f640 1 -- start start 2024-09-13T17:57:46.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.791+0000 7f080650f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f080015c6e0 0x7f08000b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:46.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.791+0000 7f080650f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f08000ccb20 0x7f08000b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:46.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.791+0000 7f080650f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08000d1840 con 0x7f08000d07b0 2024-09-13T17:57:46.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.791+0000 7f080650f640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f080006bd20 con 0x7f08000ccb20 2024-09-13T17:57:46.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.791+0000 7f080650f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08000d0c30 con 0x7f080015c6e0 2024-09-13T17:57:46.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.792+0000 7f0804aac640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f08000ccb20 0x7f08000b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:46.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.792+0000 7f0804aac640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f08000ccb20 0x7f08000b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53836/0 (socket says 172.21.15.111:53836) 2024-09-13T17:57:46.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.792+0000 7f0804aac640 1 -- 172.21.15.111:0/2213077186 learned_addr learned my addr 172.21.15.111:0/2213077186 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:46.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.792+0000 7f07fffff640 1 --2- 172.21.15.111:0/2213077186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f080015c6e0 0x7f08000b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:46.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.792+0000 7f0804aac640 1 -- 172.21.15.111:0/2213077186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f080015c6e0 msgr2=0x7f08000b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:46.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.792+0000 7f0804aac640 1 --2- 172.21.15.111:0/2213077186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f080015c6e0 0x7f08000b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.792+0000 7f0804aac640 1 -- 172.21.15.111:0/2213077186 >> v1:172.21.15.188:6789/0 conn(0x7f08000d07b0 legacy=0x7f0800160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:46.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.792+0000 7f0804aac640 1 -- 172.21.15.111:0/2213077186 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08000d0f10 con 0x7f08000ccb20 2024-09-13T17:57:46.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.793+0000 7f0804aac640 1 --2- 172.21.15.111:0/2213077186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08000ccb20 0x7f08000b4490 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009e50 tx=0x7f07f802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fad033dd0da46 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:46.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.793+0000 7f07feffd640 1 -- 172.21.15.111:0/2213077186 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f802aea0 con 0x7f08000ccb20 2024-09-13T17:57:46.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.793+0000 7f07feffd640 1 -- 172.21.15.111:0/2213077186 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f802d050 con 0x7f08000ccb20 2024-09-13T17:57:46.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.793+0000 7f07feffd640 1 -- 172.21.15.111:0/2213077186 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8031820 con 0x7f08000ccb20 2024-09-13T17:57:46.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.793+0000 7f080650f640 1 -- 172.21.15.111:0/2213077186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08000ccb20 msgr2=0x7f08000b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:46.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.793+0000 7f080650f640 1 --2- 172.21.15.111:0/2213077186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08000ccb20 0x7f08000b4490 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009e50 tx=0x7f07f802a680 comp rx=0 tx=0).stop 2024-09-13T17:57:46.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.793+0000 7f080650f640 1 -- 172.21.15.111:0/2213077186 shutdown_connections 2024-09-13T17:57:46.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.793+0000 7f080650f640 1 --2- 172.21.15.111:0/2213077186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08000ccb20 0x7f08000b4490 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.793+0000 7f080650f640 1 --2- 172.21.15.111:0/2213077186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f080015c6e0 0x7f08000b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.793+0000 7f080650f640 1 -- 172.21.15.111:0/2213077186 >> 172.21.15.111:0/2213077186 conn(0x7f0800089f60 msgr2=0x7f08000d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:46.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.793+0000 7f080650f640 1 -- 172.21.15.111:0/2213077186 shutdown_connections 2024-09-13T17:57:46.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.793+0000 7f080650f640 1 -- 172.21.15.111:0/2213077186 wait complete. 2024-09-13T17:57:46.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.795+0000 7f080650f640 1 Processor -- start 2024-09-13T17:57:46.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.795+0000 7f080650f640 1 -- start start 2024-09-13T17:57:46.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.795+0000 7f080650f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08000ccb20 0x7f08000cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:46.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.795+0000 7f080650f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08000d07b0 0x7f0800154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:46.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.795+0000 7f080650f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f080015c6e0 0x7f08000c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:46.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.795+0000 7f080650f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08000cbac0 con 0x7f08000ccb20 2024-09-13T17:57:46.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.795+0000 7f080650f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080015e360 con 0x7f08000d07b0 2024-09-13T17:57:46.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.795+0000 7f080650f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080015e600 con 0x7f080015c6e0 2024-09-13T17:57:46.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.795+0000 7f07fffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08000ccb20 0x7f08000cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:46.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.796+0000 7f07fffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08000ccb20 0x7f08000cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53840/0 (socket says 172.21.15.111:53840) 2024-09-13T17:57:46.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.796+0000 7f07fffff640 1 -- 172.21.15.111:0/3417791035 learned_addr learned my addr 172.21.15.111:0/3417791035 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:46.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.796+0000 7f0804aac640 1 --2- 172.21.15.111:0/3417791035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f080015c6e0 0x7f08000c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:46.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.796+0000 7f07ff7fe640 1 --2- 172.21.15.111:0/3417791035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08000d07b0 0x7f0800154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:46.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.796+0000 7f07fffff640 1 -- 172.21.15.111:0/3417791035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f080015c6e0 msgr2=0x7f08000c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:46.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.796+0000 7f07fffff640 1 --2- 172.21.15.111:0/3417791035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f080015c6e0 0x7f08000c95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.796+0000 7f07fffff640 1 -- 172.21.15.111:0/3417791035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08000d07b0 msgr2=0x7f0800154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:46.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.796+0000 7f07fffff640 1 --2- 172.21.15.111:0/3417791035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08000d07b0 0x7f0800154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.796+0000 7f07fffff640 1 -- 172.21.15.111:0/3417791035 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f8009b30 con 0x7f08000ccb20 2024-09-13T17:57:46.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.796+0000 7f07fffff640 1 --2- 172.21.15.111:0/3417791035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08000ccb20 0x7f08000cc510 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f07f000d800 tx=0x7f07f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:46.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.796+0000 7f07fcff9640 1 -- 172.21.15.111:0/3417791035 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f000b6c0 con 0x7f08000ccb20 2024-09-13T17:57:46.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.796+0000 7f07fcff9640 1 -- 172.21.15.111:0/3417791035 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f000b840 con 0x7f08000ccb20 2024-09-13T17:57:46.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.796+0000 7f07fcff9640 1 -- 172.21.15.111:0/3417791035 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f000bbf0 con 0x7f08000ccb20 2024-09-13T17:57:46.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.796+0000 7f080650f640 1 -- 172.21.15.111:0/3417791035 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08000cbd10 con 0x7f08000ccb20 2024-09-13T17:57:46.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.796+0000 7f080650f640 1 -- 172.21.15.111:0/3417791035 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08000c9ae0 con 0x7f08000ccb20 2024-09-13T17:57:46.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.797+0000 7f080650f640 1 -- 172.21.15.111:0/3417791035 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08000b4c80 con 0x7f08000ccb20 2024-09-13T17:57:46.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.798+0000 7f07fcff9640 1 -- 172.21.15.111:0/3417791035 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f07f000bd50 con 0x7f08000ccb20 2024-09-13T17:57:46.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.798+0000 7f07fcff9640 1 --2- 172.21.15.111:0/3417791035 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f07e006ac00 0x7f07e006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:46.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.798+0000 7f07fcff9640 1 -- 172.21.15.111:0/3417791035 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f07f0089a70 con 0x7f08000ccb20 2024-09-13T17:57:46.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.798+0000 7f07ff7fe640 1 --2- 172.21.15.111:0/3417791035 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f07e006ac00 0x7f07e006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:46.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.798+0000 7f07ff7fe640 1 --2- 172.21.15.111:0/3417791035 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f07e006ac00 0x7f07e006d0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f08000b8070 tx=0x7f07e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:46.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.802+0000 7f07fcff9640 1 -- 172.21.15.111:0/3417791035 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f07f0054aa0 con 0x7f08000ccb20 2024-09-13T17:57:46.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.966+0000 7f080650f640 1 -- 172.21.15.111:0/3417791035 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f080008c610 con 0x7f08000ccb20 2024-09-13T17:57:46.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.967+0000 7f07fcff9640 1 -- 172.21.15.111:0/3417791035 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f07f00580c0 con 0x7f08000ccb20 2024-09-13T17:57:46.970 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:46.970 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:30.065235+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:46.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.969+0000 7f080650f640 1 -- 172.21.15.111:0/3417791035 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f07e006ac00 msgr2=0x7f07e006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:46.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.969+0000 7f080650f640 1 --2- 172.21.15.111:0/3417791035 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f07e006ac00 0x7f07e006d0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f08000b8070 tx=0x7f07e8009040 comp rx=0 tx=0).stop 2024-09-13T17:57:46.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.969+0000 7f080650f640 1 -- 172.21.15.111:0/3417791035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08000ccb20 msgr2=0x7f08000cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:46.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.969+0000 7f080650f640 1 --2- 172.21.15.111:0/3417791035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08000ccb20 0x7f08000cc510 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f07f000d800 tx=0x7f07f000dcc0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.969+0000 7f080650f640 1 -- 172.21.15.111:0/3417791035 shutdown_connections 2024-09-13T17:57:46.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.969+0000 7f080650f640 1 --2- 172.21.15.111:0/3417791035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f080015c6e0 0x7f08000c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.969+0000 7f080650f640 1 --2- 172.21.15.111:0/3417791035 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f07e006ac00 0x7f07e006d0c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.969+0000 7f080650f640 1 --2- 172.21.15.111:0/3417791035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08000d07b0 0x7f0800154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.969+0000 7f080650f640 1 --2- 172.21.15.111:0/3417791035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08000ccb20 0x7f08000cc510 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:46.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.969+0000 7f080650f640 1 -- 172.21.15.111:0/3417791035 >> 172.21.15.111:0/3417791035 conn(0x7f0800089f60 msgr2=0x7f080006ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:46.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.969+0000 7f080650f640 1 -- 172.21.15.111:0/3417791035 shutdown_connections 2024-09-13T17:57:46.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:46.969+0000 7f080650f640 1 -- 172.21.15.111:0/3417791035 wait complete. 2024-09-13T17:57:46.983 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T17:57:46.983 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:57:47.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.077+0000 7fe2e9a9c640 1 Processor -- start 2024-09-13T17:57:47.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.077+0000 7fe2e9a9c640 1 -- start start 2024-09-13T17:57:47.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.078+0000 7fe2e9a9c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe2e415a2e0 0x7fe2e40c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:47.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.078+0000 7fe2e9a9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e40c9f80 0x7fe2e40cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:47.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.078+0000 7fe2e9a9c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e40c8920 con 0x7fe2e40cb220 2024-09-13T17:57:47.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.078+0000 7fe2e9a9c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e40cb650 con 0x7fe2e415a2e0 2024-09-13T17:57:47.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.078+0000 7fe2e9a9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e415a6c0 con 0x7fe2e40c9f80 2024-09-13T17:57:47.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.078+0000 7fe2e37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe2e415a2e0 0x7fe2e40c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:47.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.078+0000 7fe2e3fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe2e40cb220 0x7fe2e40ce260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49642/0 (socket says 172.21.15.111:49642) 2024-09-13T17:57:47.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.078+0000 7fe2e3fff640 1 -- 172.21.15.111:0/4131915208 learned_addr learned my addr 172.21.15.111:0/4131915208 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:47.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.078+0000 7fe2e37fe640 1 -- 172.21.15.111:0/4131915208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e40c9f80 msgr2=0x7fe2e40cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:47.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.078+0000 7fe2e2ffd640 1 --2- 172.21.15.111:0/4131915208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e40c9f80 0x7fe2e40cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:47.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.078+0000 7fe2e37fe640 1 --2- 172.21.15.111:0/4131915208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e40c9f80 0x7fe2e40cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.078+0000 7fe2e37fe640 1 -- 172.21.15.111:0/4131915208 >> v1:172.21.15.111:6789/0 conn(0x7fe2e40cb220 legacy=0x7fe2e40ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.078+0000 7fe2e27fc640 1 -- 172.21.15.111:0/4131915208 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492437063 0 0) 0x7fe2e40c8920 con 0x7fe2e40cb220 2024-09-13T17:57:47.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.078+0000 7fe2e37fe640 1 -- 172.21.15.111:0/4131915208 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e40ca500 con 0x7fe2e415a2e0 2024-09-13T17:57:47.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.079+0000 7fe2e37fe640 1 --2- 172.21.15.111:0/4131915208 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2e415a2e0 0x7fe2e40c9a40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4009d90 tx=0x7fe2d402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a160a0c31b65f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:47.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.079+0000 7fe2e27fc640 1 -- 172.21.15.111:0/4131915208 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4007740 con 0x7fe2e415a2e0 2024-09-13T17:57:47.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.079+0000 7fe2e27fc640 1 -- 172.21.15.111:0/4131915208 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d402d050 con 0x7fe2e415a2e0 2024-09-13T17:57:47.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.079+0000 7fe2e27fc640 1 -- 172.21.15.111:0/4131915208 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4007ac0 con 0x7fe2e415a2e0 2024-09-13T17:57:47.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.080+0000 7fe2e9a9c640 1 -- 172.21.15.111:0/4131915208 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2e415a2e0 msgr2=0x7fe2e40c9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.080+0000 7fe2e9a9c640 1 --2- 172.21.15.111:0/4131915208 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2e415a2e0 0x7fe2e40c9a40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4009d90 tx=0x7fe2d402a660 comp rx=0 tx=0).stop 2024-09-13T17:57:47.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.080+0000 7fe2e9a9c640 1 -- 172.21.15.111:0/4131915208 shutdown_connections 2024-09-13T17:57:47.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.080+0000 7fe2e9a9c640 1 --2- 172.21.15.111:0/4131915208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e40c9f80 0x7fe2e40cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.080+0000 7fe2e9a9c640 1 --2- 172.21.15.111:0/4131915208 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2e415a2e0 0x7fe2e40c9a40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.080+0000 7fe2e9a9c640 1 -- 172.21.15.111:0/4131915208 >> 172.21.15.111:0/4131915208 conn(0x7fe2e4069ed0 msgr2=0x7fe2e406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:47.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.080+0000 7fe2e9a9c640 1 -- 172.21.15.111:0/4131915208 shutdown_connections 2024-09-13T17:57:47.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.080+0000 7fe2e9a9c640 1 -- 172.21.15.111:0/4131915208 wait complete. 2024-09-13T17:57:47.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.081+0000 7fe2e9a9c640 1 Processor -- start 2024-09-13T17:57:47.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.081+0000 7fe2e9a9c640 1 -- start start 2024-09-13T17:57:47.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.082+0000 7fe2e9a9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e40c9f80 0x7fe2e40c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:47.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.082+0000 7fe2e9a9c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2e40cb220 0x7fe2e41f0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:47.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.082+0000 7fe2e9a9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e415a2e0 0x7fe2e415b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:47.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.082+0000 7fe2e9a9c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e415c900 con 0x7fe2e40cb220 2024-09-13T17:57:47.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.082+0000 7fe2e9a9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e415bd90 con 0x7fe2e415a2e0 2024-09-13T17:57:47.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.082+0000 7fe2e9a9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e415c060 con 0x7fe2e40c9f80 2024-09-13T17:57:47.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.082+0000 7fe2e2ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2e40cb220 0x7fe2e41f0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:47.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.082+0000 7fe2e2ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2e40cb220 0x7fe2e41f0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53854/0 (socket says 172.21.15.111:53854) 2024-09-13T17:57:47.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.082+0000 7fe2e2ffd640 1 -- 172.21.15.111:0/4152616143 learned_addr learned my addr 172.21.15.111:0/4152616143 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:47.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.082+0000 7fe2e37fe640 1 --2- 172.21.15.111:0/4152616143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e40c9f80 0x7fe2e40c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:47.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.082+0000 7fe2e2ffd640 1 -- 172.21.15.111:0/4152616143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e40c9f80 msgr2=0x7fe2e40c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.082+0000 7fe2e2ffd640 1 --2- 172.21.15.111:0/4152616143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e40c9f80 0x7fe2e40c78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.082+0000 7fe2e2ffd640 1 -- 172.21.15.111:0/4152616143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e415a2e0 msgr2=0x7fe2e415b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.082+0000 7fe2e2ffd640 1 --2- 172.21.15.111:0/4152616143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e415a2e0 0x7fe2e415b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.082+0000 7fe2e2ffd640 1 -- 172.21.15.111:0/4152616143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d4009a70 con 0x7fe2e40cb220 2024-09-13T17:57:47.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.082+0000 7fe2e2ffd640 1 --2- 172.21.15.111:0/4152616143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2e40cb220 0x7fe2e41f0c50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800c7f0 tx=0x7fe2d800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:47.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.083+0000 7fe2cbfff640 1 -- 172.21.15.111:0/4152616143 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8004220 con 0x7fe2e40cb220 2024-09-13T17:57:47.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.083+0000 7fe2cbfff640 1 -- 172.21.15.111:0/4152616143 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d80043a0 con 0x7fe2e40cb220 2024-09-13T17:57:47.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.083+0000 7fe2e9a9c640 1 -- 172.21.15.111:0/4152616143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e415c340 con 0x7fe2e40cb220 2024-09-13T17:57:47.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.083+0000 7fe2cbfff640 1 -- 172.21.15.111:0/4152616143 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8007860 con 0x7fe2e40cb220 2024-09-13T17:57:47.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.083+0000 7fe2e9a9c640 1 -- 172.21.15.111:0/4152616143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e40c5520 con 0x7fe2e40cb220 2024-09-13T17:57:47.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.084+0000 7fe2e9a9c640 1 -- 172.21.15.111:0/4152616143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e40c5380 con 0x7fe2e40cb220 2024-09-13T17:57:47.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.084+0000 7fe2cbfff640 1 -- 172.21.15.111:0/4152616143 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe2d8007bd0 con 0x7fe2e40cb220 2024-09-13T17:57:47.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.084+0000 7fe2cbfff640 1 --2- 172.21.15.111:0/4152616143 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe2d006ac30 0x7fe2d006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:47.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.085+0000 7fe2cbfff640 1 -- 172.21.15.111:0/4152616143 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe2d8089600 con 0x7fe2e40cb220 2024-09-13T17:57:47.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.085+0000 7fe2e37fe640 1 --2- 172.21.15.111:0/4152616143 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe2d006ac30 0x7fe2d006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:47.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.085+0000 7fe2e37fe640 1 --2- 172.21.15.111:0/4152616143 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe2d006ac30 0x7fe2d006d0f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4034000 tx=0x7fe2d4033040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:47.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.089+0000 7fe2cbfff640 1 -- 172.21.15.111:0/4152616143 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fe2e40c5380 con 0x7fe2e40cb220 2024-09-13T17:57:47.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.247+0000 7fe2e9a9c640 1 -- 172.21.15.111:0/4152616143 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe2e40c9130 con 0x7fe2d006ac30 2024-09-13T17:57:47.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.256+0000 7fe2cbfff640 1 -- 172.21.15.111:0/4152616143 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345285 (secure 0 0 0) 0x7fe2d4007920 con 0x7fe2d006ac30 2024-09-13T17:57:47.269 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:47.273 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-09-13T17:57:47.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.260+0000 7fe2e9a9c640 1 -- 172.21.15.111:0/4152616143 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe2d006ac30 msgr2=0x7fe2d006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.260+0000 7fe2e9a9c640 1 --2- 172.21.15.111:0/4152616143 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe2d006ac30 0x7fe2d006d0f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4034000 tx=0x7fe2d4033040 comp rx=0 tx=0).stop 2024-09-13T17:57:47.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.260+0000 7fe2e9a9c640 1 -- 172.21.15.111:0/4152616143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2e40cb220 msgr2=0x7fe2e41f0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.260+0000 7fe2e9a9c640 1 --2- 172.21.15.111:0/4152616143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2e40cb220 0x7fe2e41f0c50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800c7f0 tx=0x7fe2d800ccb0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.260+0000 7fe2e9a9c640 1 -- 172.21.15.111:0/4152616143 shutdown_connections 2024-09-13T17:57:47.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.260+0000 7fe2e9a9c640 1 --2- 172.21.15.111:0/4152616143 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe2d006ac30 0x7fe2d006d0f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.260+0000 7fe2e9a9c640 1 --2- 172.21.15.111:0/4152616143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e415a2e0 0x7fe2e415b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.260+0000 7fe2e9a9c640 1 --2- 172.21.15.111:0/4152616143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2e40cb220 0x7fe2e41f0c50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.260+0000 7fe2e9a9c640 1 --2- 172.21.15.111:0/4152616143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e40c9f80 0x7fe2e40c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.260+0000 7fe2e9a9c640 1 -- 172.21.15.111:0/4152616143 >> 172.21.15.111:0/4152616143 conn(0x7fe2e4069ed0 msgr2=0x7fe2e40ad150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:47.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.260+0000 7fe2e9a9c640 1 -- 172.21.15.111:0/4152616143 shutdown_connections 2024-09-13T17:57:47.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.260+0000 7fe2e9a9c640 1 -- 172.21.15.111:0/4152616143 wait complete. 2024-09-13T17:57:47.278 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-13T17:57:46.270478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3204,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3280896,"data_stored":1987143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2231,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002692"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727037+0000","last_change":"2024-09-13T17:57:25.378960+0000","last_active":"2024-09-13T17:57:29.727037+0000","last_peered":"2024-09-13T17:57:29.727037+0000","last_clean":"2024-09-13T17:57:29.727037+0000","last_became_active":"2024-09-13T17:57:25.378681+0000","last_became_peered":"2024-09-13T17:57:25.378681+0000","last_unstale":"2024-09-13T17:57:29.727037+0000","last_undegraded":"2024-09-13T17:57:29.727037+0000","last_fullsized":"2024-09-13T17:57:29.727037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721168+0000","last_change":"2024-09-13T17:57:25.734706+0000","last_active":"2024-09-13T17:57:29.721168+0000","last_peered":"2024-09-13T17:57:29.721168+0000","last_clean":"2024-09-13T17:57:29.721168+0000","last_became_active":"2024-09-13T17:57:25.734432+0000","last_became_peered":"2024-09-13T17:57:25.734432+0000","last_unstale":"2024-09-13T17:57:29.721168+0000","last_undegraded":"2024-09-13T17:57:29.721168+0000","last_fullsized":"2024-09-13T17:57:29.721168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:12.960061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.730686+0000","last_change":"2024-09-13T17:57:29.729380+0000","last_active":"2024-09-13T17:57:29.730686+0000","last_peered":"2024-09-13T17:57:29.730686+0000","last_clean":"2024-09-13T17:57:29.730686+0000","last_became_active":"2024-09-13T17:57:29.729000+0000","last_became_peered":"2024-09-13T17:57:29.729000+0000","last_unstale":"2024-09-13T17:57:29.730686+0000","last_undegraded":"2024-09-13T17:57:29.730686+0000","last_fullsized":"2024-09-13T17:57:29.730686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:31.285328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721336+0000","last_change":"2024-09-13T17:57:25.382416+0000","last_active":"2024-09-13T17:57:29.721336+0000","last_peered":"2024-09-13T17:57:29.721336+0000","last_clean":"2024-09-13T17:57:29.721336+0000","last_became_active":"2024-09-13T17:57:25.382154+0000","last_became_peered":"2024-09-13T17:57:25.382154+0000","last_unstale":"2024-09-13T17:57:29.721336+0000","last_undegraded":"2024-09-13T17:57:29.721336+0000","last_fullsized":"2024-09-13T17:57:29.721336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725491+0000","last_change":"2024-09-13T17:57:25.731759+0000","last_active":"2024-09-13T17:57:29.725491+0000","last_peered":"2024-09-13T17:57:29.725491+0000","last_clean":"2024-09-13T17:57:29.725491+0000","last_became_active":"2024-09-13T17:57:25.731494+0000","last_became_peered":"2024-09-13T17:57:25.731494+0000","last_unstale":"2024-09-13T17:57:29.725491+0000","last_undegraded":"2024-09-13T17:57:29.725491+0000","last_fullsized":"2024-09-13T17:57:29.725491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.751270+0000","last_change":"2024-09-13T17:57:29.750278+0000","last_active":"2024-09-13T17:57:29.751270+0000","last_peered":"2024-09-13T17:57:29.751270+0000","last_clean":"2024-09-13T17:57:29.751270+0000","last_became_active":"2024-09-13T17:57:29.749874+0000","last_became_peered":"2024-09-13T17:57:29.749874+0000","last_unstale":"2024-09-13T17:57:29.751270+0000","last_undegraded":"2024-09-13T17:57:29.751270+0000","last_fullsized":"2024-09-13T17:57:29.751270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:03.749305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725223+0000","last_change":"2024-09-13T17:57:25.377660+0000","last_active":"2024-09-13T17:57:29.725223+0000","last_peered":"2024-09-13T17:57:29.725223+0000","last_clean":"2024-09-13T17:57:29.725223+0000","last_became_active":"2024-09-13T17:57:25.377386+0000","last_became_peered":"2024-09-13T17:57:25.377386+0000","last_unstale":"2024-09-13T17:57:29.725223+0000","last_undegraded":"2024-09-13T17:57:29.725223+0000","last_fullsized":"2024-09-13T17:57:29.725223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:38.038738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725142+0000","last_change":"2024-09-13T17:57:25.728949+0000","last_active":"2024-09-13T17:57:29.725142+0000","last_peered":"2024-09-13T17:57:29.725142+0000","last_clean":"2024-09-13T17:57:29.725142+0000","last_became_active":"2024-09-13T17:57:25.728675+0000","last_became_peered":"2024-09-13T17:57:25.728675+0000","last_unstale":"2024-09-13T17:57:29.725142+0000","last_undegraded":"2024-09-13T17:57:29.725142+0000","last_fullsized":"2024-09-13T17:57:29.725142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749006+0000","last_change":"2024-09-13T17:57:29.745555+0000","last_active":"2024-09-13T17:57:29.749006+0000","last_peered":"2024-09-13T17:57:29.749006+0000","last_clean":"2024-09-13T17:57:29.749006+0000","last_became_active":"2024-09-13T17:57:29.745172+0000","last_became_peered":"2024-09-13T17:57:29.745172+0000","last_unstale":"2024-09-13T17:57:29.749006+0000","last_undegraded":"2024-09-13T17:57:29.749006+0000","last_fullsized":"2024-09-13T17:57:29.749006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:37.101711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725546+0000","last_change":"2024-09-13T17:57:25.375351+0000","last_active":"2024-09-13T17:57:29.725546+0000","last_peered":"2024-09-13T17:57:29.725546+0000","last_clean":"2024-09-13T17:57:29.725546+0000","last_became_active":"2024-09-13T17:57:25.374550+0000","last_became_peered":"2024-09-13T17:57:25.374550+0000","last_unstale":"2024-09-13T17:57:29.725546+0000","last_undegraded":"2024-09-13T17:57:29.725546+0000","last_fullsized":"2024-09-13T17:57:29.725546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:11:57.892225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725579+0000","last_change":"2024-09-13T17:57:25.729527+0000","last_active":"2024-09-13T17:57:29.725579+0000","last_peered":"2024-09-13T17:57:29.725579+0000","last_clean":"2024-09-13T17:57:29.725579+0000","last_became_active":"2024-09-13T17:57:25.729225+0000","last_became_peered":"2024-09-13T17:57:25.729225+0000","last_unstale":"2024-09-13T17:57:29.725579+0000","last_undegraded":"2024-09-13T17:57:29.725579+0000","last_fullsized":"2024-09-13T17:57:29.725579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:27.969805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.748980+0000","last_change":"2024-09-13T17:57:29.745532+0000","last_active":"2024-09-13T17:57:29.748980+0000","last_peered":"2024-09-13T17:57:29.748980+0000","last_clean":"2024-09-13T17:57:29.748980+0000","last_became_active":"2024-09-13T17:57:29.745130+0000","last_became_peered":"2024-09-13T17:57:29.745130+0000","last_unstale":"2024-09-13T17:57:29.748980+0000","last_undegraded":"2024-09-13T17:57:29.748980+0000","last_fullsized":"2024-09-13T17:57:29.748980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:54.172556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728175+0000","last_change":"2024-09-13T17:57:25.735067+0000","last_active":"2024-09-13T17:57:29.728175+0000","last_peered":"2024-09-13T17:57:29.728175+0000","last_clean":"2024-09-13T17:57:29.728175+0000","last_became_active":"2024-09-13T17:57:25.734807+0000","last_became_peered":"2024-09-13T17:57:25.734807+0000","last_unstale":"2024-09-13T17:57:29.728175+0000","last_undegraded":"2024-09-13T17:57:29.728175+0000","last_fullsized":"2024-09-13T17:57:29.728175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:19.536367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725323+0000","last_change":"2024-09-13T17:57:25.374934+0000","last_active":"2024-09-13T17:57:29.725323+0000","last_peered":"2024-09-13T17:57:29.725323+0000","last_clean":"2024-09-13T17:57:29.725323+0000","last_became_active":"2024-09-13T17:57:25.374307+0000","last_became_peered":"2024-09-13T17:57:25.374307+0000","last_unstale":"2024-09-13T17:57:29.725323+0000","last_undegraded":"2024-09-13T17:57:29.725323+0000","last_fullsized":"2024-09-13T17:57:29.725323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:15.663292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.748917+0000","last_change":"2024-09-13T17:57:29.743757+0000","last_active":"2024-09-13T17:57:29.748917+0000","last_peered":"2024-09-13T17:57:29.748917+0000","last_clean":"2024-09-13T17:57:29.748917+0000","last_became_active":"2024-09-13T17:57:29.743362+0000","last_became_peered":"2024-09-13T17:57:29.743362+0000","last_unstale":"2024-09-13T17:57:29.748917+0000","last_undegraded":"2024-09-13T17:57:29.748917+0000","last_fullsized":"2024-09-13T17:57:29.748917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:31.285328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720769+0000","last_change":"2024-09-13T17:57:25.720127+0000","last_active":"2024-09-13T17:57:29.720769+0000","last_peered":"2024-09-13T17:57:29.720769+0000","last_clean":"2024-09-13T17:57:29.720769+0000","last_became_active":"2024-09-13T17:57:25.719766+0000","last_became_peered":"2024-09-13T17:57:25.719766+0000","last_unstale":"2024-09-13T17:57:29.720769+0000","last_undegraded":"2024-09-13T17:57:29.720769+0000","last_fullsized":"2024-09-13T17:57:29.720769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.730752+0000","last_change":"2024-09-13T17:57:25.381085+0000","last_active":"2024-09-13T17:57:29.730752+0000","last_peered":"2024-09-13T17:57:29.730752+0000","last_clean":"2024-09-13T17:57:29.730752+0000","last_became_active":"2024-09-13T17:57:25.380825+0000","last_became_peered":"2024-09-13T17:57:25.380825+0000","last_unstale":"2024-09-13T17:57:29.730752+0000","last_undegraded":"2024-09-13T17:57:29.730752+0000","last_fullsized":"2024-09-13T17:57:29.730752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:15.663292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749031+0000","last_change":"2024-09-13T17:57:29.746426+0000","last_active":"2024-09-13T17:57:29.749031+0000","last_peered":"2024-09-13T17:57:29.749031+0000","last_clean":"2024-09-13T17:57:29.749031+0000","last_became_active":"2024-09-13T17:57:29.745582+0000","last_became_peered":"2024-09-13T17:57:29.745582+0000","last_unstale":"2024-09-13T17:57:29.749031+0000","last_undegraded":"2024-09-13T17:57:29.749031+0000","last_fullsized":"2024-09-13T17:57:29.749031+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:53.991862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721690+0000","last_change":"2024-09-13T17:57:25.724108+0000","last_active":"2024-09-13T17:57:29.721690+0000","last_peered":"2024-09-13T17:57:29.721690+0000","last_clean":"2024-09-13T17:57:29.721690+0000","last_became_active":"2024-09-13T17:57:25.723857+0000","last_became_peered":"2024-09-13T17:57:25.723857+0000","last_unstale":"2024-09-13T17:57:29.721690+0000","last_undegraded":"2024-09-13T17:57:29.721690+0000","last_fullsized":"2024-09-13T17:57:29.721690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:19.536367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729185+0000","last_change":"2024-09-13T17:57:25.374954+0000","last_active":"2024-09-13T17:57:29.729185+0000","last_peered":"2024-09-13T17:57:29.729185+0000","last_clean":"2024-09-13T17:57:29.729185+0000","last_became_active":"2024-09-13T17:57:25.374721+0000","last_became_peered":"2024-09-13T17:57:25.374721+0000","last_unstale":"2024-09-13T17:57:29.729185+0000","last_undegraded":"2024-09-13T17:57:29.729185+0000","last_fullsized":"2024-09-13T17:57:29.729185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:34.109316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749541+0000","last_change":"2024-09-13T17:57:29.744520+0000","last_active":"2024-09-13T17:57:29.749541+0000","last_peered":"2024-09-13T17:57:29.749541+0000","last_clean":"2024-09-13T17:57:29.749541+0000","last_became_active":"2024-09-13T17:57:29.744143+0000","last_became_peered":"2024-09-13T17:57:29.744143+0000","last_unstale":"2024-09-13T17:57:29.749541+0000","last_undegraded":"2024-09-13T17:57:29.749541+0000","last_fullsized":"2024-09-13T17:57:29.749541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:18.526568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.733595+0000","last_change":"2024-09-13T17:57:25.718903+0000","last_active":"2024-09-13T17:57:29.733595+0000","last_peered":"2024-09-13T17:57:29.733595+0000","last_clean":"2024-09-13T17:57:29.733595+0000","last_became_active":"2024-09-13T17:57:25.718483+0000","last_became_peered":"2024-09-13T17:57:25.718483+0000","last_unstale":"2024-09-13T17:57:29.733595+0000","last_undegraded":"2024-09-13T17:57:29.733595+0000","last_fullsized":"2024-09-13T17:57:29.733595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:59.252453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725956+0000","last_change":"2024-09-13T17:57:25.378628+0000","last_active":"2024-09-13T17:57:29.725956+0000","last_peered":"2024-09-13T17:57:29.725956+0000","last_clean":"2024-09-13T17:57:29.725956+0000","last_became_active":"2024-09-13T17:57:25.378359+0000","last_became_peered":"2024-09-13T17:57:25.378359+0000","last_unstale":"2024-09-13T17:57:29.725956+0000","last_undegraded":"2024-09-13T17:57:29.725956+0000","last_fullsized":"2024-09-13T17:57:29.725956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:00.725292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749048+0000","last_change":"2024-09-13T17:57:29.746604+0000","last_active":"2024-09-13T17:57:29.749048+0000","last_peered":"2024-09-13T17:57:29.749048+0000","last_clean":"2024-09-13T17:57:29.749048+0000","last_became_active":"2024-09-13T17:57:29.746216+0000","last_became_peered":"2024-09-13T17:57:29.746216+0000","last_unstale":"2024-09-13T17:57:29.749048+0000","last_undegraded":"2024-09-13T17:57:29.749048+0000","last_fullsized":"2024-09-13T17:57:29.749048+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:18.526568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725404+0000","last_change":"2024-09-13T17:57:25.722540+0000","last_active":"2024-09-13T17:57:29.725404+0000","last_peered":"2024-09-13T17:57:29.725404+0000","last_clean":"2024-09-13T17:57:29.725404+0000","last_became_active":"2024-09-13T17:57:25.722301+0000","last_became_peered":"2024-09-13T17:57:25.722301+0000","last_unstale":"2024-09-13T17:57:29.725404+0000","last_undegraded":"2024-09-13T17:57:29.725404+0000","last_fullsized":"2024-09-13T17:57:29.725404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:34.117674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728850+0000","last_change":"2024-09-13T17:57:25.380087+0000","last_active":"2024-09-13T17:57:29.728850+0000","last_peered":"2024-09-13T17:57:29.728850+0000","last_clean":"2024-09-13T17:57:29.728850+0000","last_became_active":"2024-09-13T17:57:25.379808+0000","last_became_peered":"2024-09-13T17:57:25.379808+0000","last_unstale":"2024-09-13T17:57:29.728850+0000","last_undegraded":"2024-09-13T17:57:29.728850+0000","last_fullsized":"2024-09-13T17:57:29.728850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:38.038738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.743674+0000","last_change":"2024-09-13T17:57:29.740242+0000","last_active":"2024-09-13T17:57:29.743674+0000","last_peered":"2024-09-13T17:57:29.743674+0000","last_clean":"2024-09-13T17:57:29.743674+0000","last_became_active":"2024-09-13T17:57:29.739815+0000","last_became_peered":"2024-09-13T17:57:29.739815+0000","last_unstale":"2024-09-13T17:57:29.743674+0000","last_undegraded":"2024-09-13T17:57:29.743674+0000","last_fullsized":"2024-09-13T17:57:29.743674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:45.306949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720934+0000","last_change":"2024-09-13T17:57:25.718065+0000","last_active":"2024-09-13T17:57:29.720934+0000","last_peered":"2024-09-13T17:57:29.720934+0000","last_clean":"2024-09-13T17:57:29.720934+0000","last_became_active":"2024-09-13T17:57:25.717759+0000","last_became_peered":"2024-09-13T17:57:25.717759+0000","last_unstale":"2024-09-13T17:57:29.720934+0000","last_undegraded":"2024-09-13T17:57:29.720934+0000","last_fullsized":"2024-09-13T17:57:29.720934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.814140+0000","last_change":"2024-09-13T17:57:25.377272+0000","last_active":"2024-09-13T17:57:29.814140+0000","last_peered":"2024-09-13T17:57:29.814140+0000","last_clean":"2024-09-13T17:57:29.814140+0000","last_became_active":"2024-09-13T17:57:25.376938+0000","last_became_peered":"2024-09-13T17:57:25.376938+0000","last_unstale":"2024-09-13T17:57:29.814140+0000","last_undegraded":"2024-09-13T17:57:29.814140+0000","last_fullsized":"2024-09-13T17:57:29.814140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:00.725292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.746741+0000","last_change":"2024-09-13T17:57:29.743363+0000","last_active":"2024-09-13T17:57:29.746741+0000","last_peered":"2024-09-13T17:57:29.746741+0000","last_clean":"2024-09-13T17:57:29.746741+0000","last_became_active":"2024-09-13T17:57:29.738263+0000","last_became_peered":"2024-09-13T17:57:29.738263+0000","last_unstale":"2024-09-13T17:57:29.746741+0000","last_undegraded":"2024-09-13T17:57:29.746741+0000","last_fullsized":"2024-09-13T17:57:29.746741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:42:39.016490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729043+0000","last_change":"2024-09-13T17:57:25.717635+0000","last_active":"2024-09-13T17:57:29.729043+0000","last_peered":"2024-09-13T17:57:29.729043+0000","last_clean":"2024-09-13T17:57:29.729043+0000","last_became_active":"2024-09-13T17:57:25.717360+0000","last_became_peered":"2024-09-13T17:57:25.717360+0000","last_unstale":"2024-09-13T17:57:29.729043+0000","last_undegraded":"2024-09-13T17:57:29.729043+0000","last_fullsized":"2024-09-13T17:57:29.729043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725395+0000","last_change":"2024-09-13T17:57:25.378289+0000","last_active":"2024-09-13T17:57:29.725395+0000","last_peered":"2024-09-13T17:57:29.725395+0000","last_clean":"2024-09-13T17:57:29.725395+0000","last_became_active":"2024-09-13T17:57:25.378038+0000","last_became_peered":"2024-09-13T17:57:25.378038+0000","last_unstale":"2024-09-13T17:57:29.725395+0000","last_undegraded":"2024-09-13T17:57:29.725395+0000","last_fullsized":"2024-09-13T17:57:29.725395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:58.248434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.746870+0000","last_change":"2024-09-13T17:57:29.741347+0000","last_active":"2024-09-13T17:57:29.746870+0000","last_peered":"2024-09-13T17:57:29.746870+0000","last_clean":"2024-09-13T17:57:29.746870+0000","last_became_active":"2024-09-13T17:57:29.740980+0000","last_became_peered":"2024-09-13T17:57:29.740980+0000","last_unstale":"2024-09-13T17:57:29.746870+0000","last_undegraded":"2024-09-13T17:57:29.746870+0000","last_fullsized":"2024-09-13T17:57:29.746870+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:53.991862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725663+0000","last_change":"2024-09-13T17:57:25.728135+0000","last_active":"2024-09-13T17:57:29.725663+0000","last_peered":"2024-09-13T17:57:29.725663+0000","last_clean":"2024-09-13T17:57:29.725663+0000","last_became_active":"2024-09-13T17:57:25.727903+0000","last_became_peered":"2024-09-13T17:57:25.727903+0000","last_unstale":"2024-09-13T17:57:29.725663+0000","last_undegraded":"2024-09-13T17:57:29.725663+0000","last_fullsized":"2024-09-13T17:57:29.725663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:48.163930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729130+0000","last_change":"2024-09-13T17:57:25.375187+0000","last_active":"2024-09-13T17:57:29.729130+0000","last_peered":"2024-09-13T17:57:29.729130+0000","last_clean":"2024-09-13T17:57:29.729130+0000","last_became_active":"2024-09-13T17:57:25.374295+0000","last_became_peered":"2024-09-13T17:57:25.374295+0000","last_unstale":"2024-09-13T17:57:29.729130+0000","last_undegraded":"2024-09-13T17:57:29.729130+0000","last_fullsized":"2024-09-13T17:57:29.729130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:58.248434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742560+0000","last_change":"2024-09-13T17:57:29.740383+0000","last_active":"2024-09-13T17:57:29.742560+0000","last_peered":"2024-09-13T17:57:29.742560+0000","last_clean":"2024-09-13T17:57:29.742560+0000","last_became_active":"2024-09-13T17:57:29.739542+0000","last_became_peered":"2024-09-13T17:57:29.739542+0000","last_unstale":"2024-09-13T17:57:29.742560+0000","last_undegraded":"2024-09-13T17:57:29.742560+0000","last_fullsized":"2024-09-13T17:57:29.742560+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:13.633882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.734117+0000","last_change":"2024-09-13T17:57:25.717973+0000","last_active":"2024-09-13T17:57:29.734117+0000","last_peered":"2024-09-13T17:57:29.734117+0000","last_clean":"2024-09-13T17:57:29.734117+0000","last_became_active":"2024-09-13T17:57:25.717702+0000","last_became_peered":"2024-09-13T17:57:25.717702+0000","last_unstale":"2024-09-13T17:57:29.734117+0000","last_undegraded":"2024-09-13T17:57:29.734117+0000","last_fullsized":"2024-09-13T17:57:29.734117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:35.113334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725035+0000","last_change":"2024-09-13T17:57:25.373814+0000","last_active":"2024-09-13T17:57:29.725035+0000","last_peered":"2024-09-13T17:57:29.725035+0000","last_clean":"2024-09-13T17:57:29.725035+0000","last_became_active":"2024-09-13T17:57:25.373556+0000","last_became_peered":"2024-09-13T17:57:25.373556+0000","last_unstale":"2024-09-13T17:57:29.725035+0000","last_undegraded":"2024-09-13T17:57:29.725035+0000","last_fullsized":"2024-09-13T17:57:29.725035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:04.441662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742509+0000","last_change":"2024-09-13T17:57:29.739908+0000","last_active":"2024-09-13T17:57:29.742509+0000","last_peered":"2024-09-13T17:57:29.742509+0000","last_clean":"2024-09-13T17:57:29.742509+0000","last_became_active":"2024-09-13T17:57:29.739070+0000","last_became_peered":"2024-09-13T17:57:29.739070+0000","last_unstale":"2024-09-13T17:57:29.742509+0000","last_undegraded":"2024-09-13T17:57:29.742509+0000","last_fullsized":"2024-09-13T17:57:29.742509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:37.101711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729849+0000","last_change":"2024-09-13T17:57:25.718810+0000","last_active":"2024-09-13T17:57:29.729849+0000","last_peered":"2024-09-13T17:57:29.729849+0000","last_clean":"2024-09-13T17:57:29.729849+0000","last_became_active":"2024-09-13T17:57:25.718576+0000","last_became_peered":"2024-09-13T17:57:25.718576+0000","last_unstale":"2024-09-13T17:57:29.729849+0000","last_undegraded":"2024-09-13T17:57:29.729849+0000","last_fullsized":"2024-09-13T17:57:29.729849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:13.197645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732373+0000","last_change":"2024-09-13T17:57:25.380336+0000","last_active":"2024-09-13T17:57:29.732373+0000","last_peered":"2024-09-13T17:57:29.732373+0000","last_clean":"2024-09-13T17:57:29.732373+0000","last_became_active":"2024-09-13T17:57:25.380044+0000","last_became_peered":"2024-09-13T17:57:25.380044+0000","last_unstale":"2024-09-13T17:57:29.732373+0000","last_undegraded":"2024-09-13T17:57:29.732373+0000","last_fullsized":"2024-09-13T17:57:29.732373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:41.189680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.746550+0000","last_change":"2024-09-13T17:57:29.741637+0000","last_active":"2024-09-13T17:57:29.746550+0000","last_peered":"2024-09-13T17:57:29.746550+0000","last_clean":"2024-09-13T17:57:29.746550+0000","last_became_active":"2024-09-13T17:57:29.741260+0000","last_became_peered":"2024-09-13T17:57:29.741260+0000","last_unstale":"2024-09-13T17:57:29.746550+0000","last_undegraded":"2024-09-13T17:57:29.746550+0000","last_fullsized":"2024-09-13T17:57:29.746550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:53.991862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724599+0000","last_change":"2024-09-13T17:57:25.718276+0000","last_active":"2024-09-13T17:57:29.724599+0000","last_peered":"2024-09-13T17:57:29.724599+0000","last_clean":"2024-09-13T17:57:29.724599+0000","last_became_active":"2024-09-13T17:57:25.718000+0000","last_became_peered":"2024-09-13T17:57:25.718000+0000","last_unstale":"2024-09-13T17:57:29.724599+0000","last_undegraded":"2024-09-13T17:57:29.724599+0000","last_fullsized":"2024-09-13T17:57:29.724599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:05.710729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724849+0000","last_change":"2024-09-13T17:57:25.375143+0000","last_active":"2024-09-13T17:57:29.724849+0000","last_peered":"2024-09-13T17:57:29.724849+0000","last_clean":"2024-09-13T17:57:29.724849+0000","last_became_active":"2024-09-13T17:57:25.374871+0000","last_became_peered":"2024-09-13T17:57:25.374871+0000","last_unstale":"2024-09-13T17:57:29.724849+0000","last_undegraded":"2024-09-13T17:57:29.724849+0000","last_fullsized":"2024-09-13T17:57:29.724849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:34.109316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.738027+0000","last_change":"2024-09-13T17:57:29.733428+0000","last_active":"2024-09-13T17:57:29.738027+0000","last_peered":"2024-09-13T17:57:29.738027+0000","last_clean":"2024-09-13T17:57:29.738027+0000","last_became_active":"2024-09-13T17:57:29.732952+0000","last_became_peered":"2024-09-13T17:57:29.732952+0000","last_unstale":"2024-09-13T17:57:29.738027+0000","last_undegraded":"2024-09-13T17:57:29.738027+0000","last_fullsized":"2024-09-13T17:57:29.738027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:56:53.973670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720518+0000","last_change":"2024-09-13T17:57:25.732656+0000","last_active":"2024-09-13T17:57:29.720518+0000","last_peered":"2024-09-13T17:57:29.720518+0000","last_clean":"2024-09-13T17:57:29.720518+0000","last_became_active":"2024-09-13T17:57:25.732308+0000","last_became_peered":"2024-09-13T17:57:25.732308+0000","last_unstale":"2024-09-13T17:57:29.720518+0000","last_undegraded":"2024-09-13T17:57:29.720518+0000","last_fullsized":"2024-09-13T17:57:29.720518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:34.797142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724840+0000","last_change":"2024-09-13T17:57:25.374372+0000","last_active":"2024-09-13T17:57:29.724840+0000","last_peered":"2024-09-13T17:57:29.724840+0000","last_clean":"2024-09-13T17:57:29.724840+0000","last_became_active":"2024-09-13T17:57:25.374110+0000","last_became_peered":"2024-09-13T17:57:25.374110+0000","last_unstale":"2024-09-13T17:57:29.724840+0000","last_undegraded":"2024-09-13T17:57:29.724840+0000","last_fullsized":"2024-09-13T17:57:29.724840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:38.038738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724941+0000","last_change":"2024-09-13T17:57:15.560506+0000","last_active":"2024-09-13T17:57:29.724941+0000","last_peered":"2024-09-13T17:57:29.724941+0000","last_clean":"2024-09-13T17:57:29.724941+0000","last_became_active":"2024-09-13T17:57:15.559612+0000","last_became_peered":"2024-09-13T17:57:15.559612+0000","last_unstale":"2024-09-13T17:57:29.724941+0000","last_undegraded":"2024-09-13T17:57:29.724941+0000","last_fullsized":"2024-09-13T17:57:29.724941+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_clean_scrub_stamp":"2024-09-13T17:57:14.317378+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:16:27.663312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.743992+0000","last_change":"2024-09-13T17:57:29.739891+0000","last_active":"2024-09-13T17:57:29.743992+0000","last_peered":"2024-09-13T17:57:29.743992+0000","last_clean":"2024-09-13T17:57:29.743992+0000","last_became_active":"2024-09-13T17:57:29.739331+0000","last_became_peered":"2024-09-13T17:57:29.739331+0000","last_unstale":"2024-09-13T17:57:29.743992+0000","last_undegraded":"2024-09-13T17:57:29.743992+0000","last_fullsized":"2024-09-13T17:57:29.743992+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:45.306949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725127+0000","last_change":"2024-09-13T17:57:25.725987+0000","last_active":"2024-09-13T17:57:29.725127+0000","last_peered":"2024-09-13T17:57:29.725127+0000","last_clean":"2024-09-13T17:57:29.725127+0000","last_became_active":"2024-09-13T17:57:25.725695+0000","last_became_peered":"2024-09-13T17:57:25.725695+0000","last_unstale":"2024-09-13T17:57:29.725127+0000","last_undegraded":"2024-09-13T17:57:29.725127+0000","last_fullsized":"2024-09-13T17:57:29.725127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:48.163930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732467+0000","last_change":"2024-09-13T17:57:25.376170+0000","last_active":"2024-09-13T17:57:29.732467+0000","last_peered":"2024-09-13T17:57:29.732467+0000","last_clean":"2024-09-13T17:57:29.732467+0000","last_became_active":"2024-09-13T17:57:25.375798+0000","last_became_peered":"2024-09-13T17:57:25.375798+0000","last_unstale":"2024-09-13T17:57:29.732467+0000","last_undegraded":"2024-09-13T17:57:29.732467+0000","last_fullsized":"2024-09-13T17:57:29.732467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:08.374958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744268+0000","last_change":"2024-09-13T17:57:29.741819+0000","last_active":"2024-09-13T17:57:29.744268+0000","last_peered":"2024-09-13T17:57:29.744268+0000","last_clean":"2024-09-13T17:57:29.744268+0000","last_became_active":"2024-09-13T17:57:29.741410+0000","last_became_peered":"2024-09-13T17:57:29.741410+0000","last_unstale":"2024-09-13T17:57:29.744268+0000","last_undegraded":"2024-09-13T17:57:29.744268+0000","last_fullsized":"2024-09-13T17:57:29.744268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:15.299874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725178+0000","last_change":"2024-09-13T17:57:25.729870+0000","last_active":"2024-09-13T17:57:29.725178+0000","last_peered":"2024-09-13T17:57:29.725178+0000","last_clean":"2024-09-13T17:57:29.725178+0000","last_became_active":"2024-09-13T17:57:25.728383+0000","last_became_peered":"2024-09-13T17:57:25.728383+0000","last_unstale":"2024-09-13T17:57:29.725178+0000","last_undegraded":"2024-09-13T17:57:29.725178+0000","last_fullsized":"2024-09-13T17:57:29.725178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:51.804478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.730112+0000","last_change":"2024-09-13T17:57:25.373843+0000","last_active":"2024-09-13T17:57:29.730112+0000","last_peered":"2024-09-13T17:57:29.730112+0000","last_clean":"2024-09-13T17:57:29.730112+0000","last_became_active":"2024-09-13T17:57:25.373498+0000","last_became_peered":"2024-09-13T17:57:25.373498+0000","last_unstale":"2024-09-13T17:57:29.730112+0000","last_undegraded":"2024-09-13T17:57:29.730112+0000","last_fullsized":"2024-09-13T17:57:29.730112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:11:57.892225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742650+0000","last_change":"2024-09-13T17:57:29.735791+0000","last_active":"2024-09-13T17:57:29.742650+0000","last_peered":"2024-09-13T17:57:29.742650+0000","last_clean":"2024-09-13T17:57:29.742650+0000","last_became_active":"2024-09-13T17:57:29.735444+0000","last_became_peered":"2024-09-13T17:57:29.735444+0000","last_unstale":"2024-09-13T17:57:29.742650+0000","last_undegraded":"2024-09-13T17:57:29.742650+0000","last_fullsized":"2024-09-13T17:57:29.742650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:13.633882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725552+0000","last_change":"2024-09-13T17:57:25.724498+0000","last_active":"2024-09-13T17:57:29.725552+0000","last_peered":"2024-09-13T17:57:29.725552+0000","last_clean":"2024-09-13T17:57:29.725552+0000","last_became_active":"2024-09-13T17:57:25.724159+0000","last_became_peered":"2024-09-13T17:57:25.724159+0000","last_unstale":"2024-09-13T17:57:29.725552+0000","last_undegraded":"2024-09-13T17:57:29.725552+0000","last_fullsized":"2024-09-13T17:57:29.725552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:34.797142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732295+0000","last_change":"2024-09-13T17:57:25.380132+0000","last_active":"2024-09-13T17:57:29.732295+0000","last_peered":"2024-09-13T17:57:29.732295+0000","last_clean":"2024-09-13T17:57:29.732295+0000","last_became_active":"2024-09-13T17:57:25.379862+0000","last_became_peered":"2024-09-13T17:57:25.379862+0000","last_unstale":"2024-09-13T17:57:29.732295+0000","last_undegraded":"2024-09-13T17:57:29.732295+0000","last_fullsized":"2024-09-13T17:57:29.732295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:43:20.708183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.740512+0000","last_change":"2024-09-13T17:57:29.738502+0000","last_active":"2024-09-13T17:57:29.740512+0000","last_peered":"2024-09-13T17:57:29.740512+0000","last_clean":"2024-09-13T17:57:29.740512+0000","last_became_active":"2024-09-13T17:57:29.737664+0000","last_became_peered":"2024-09-13T17:57:29.737664+0000","last_unstale":"2024-09-13T17:57:29.740512+0000","last_undegraded":"2024-09-13T17:57:29.740512+0000","last_fullsized":"2024-09-13T17:57:29.740512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:13.633882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725043+0000","last_change":"2024-09-13T17:57:25.716646+0000","last_active":"2024-09-13T17:57:29.725043+0000","last_peered":"2024-09-13T17:57:29.725043+0000","last_clean":"2024-09-13T17:57:29.725043+0000","last_became_active":"2024-09-13T17:57:25.716005+0000","last_became_peered":"2024-09-13T17:57:25.716005+0000","last_unstale":"2024-09-13T17:57:29.725043+0000","last_undegraded":"2024-09-13T17:57:29.725043+0000","last_fullsized":"2024-09-13T17:57:29.725043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:43.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.730970+0000","last_change":"2024-09-13T17:57:25.374986+0000","last_active":"2024-09-13T17:57:29.730970+0000","last_peered":"2024-09-13T17:57:29.730970+0000","last_clean":"2024-09-13T17:57:29.730970+0000","last_became_active":"2024-09-13T17:57:25.374609+0000","last_became_peered":"2024-09-13T17:57:25.374609+0000","last_unstale":"2024-09-13T17:57:29.730970+0000","last_undegraded":"2024-09-13T17:57:29.730970+0000","last_fullsized":"2024-09-13T17:57:29.730970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:43:20.708183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749009+0000","last_change":"2024-09-13T17:57:29.745707+0000","last_active":"2024-09-13T17:57:29.749009+0000","last_peered":"2024-09-13T17:57:29.749009+0000","last_clean":"2024-09-13T17:57:29.749009+0000","last_became_active":"2024-09-13T17:57:29.745303+0000","last_became_peered":"2024-09-13T17:57:29.745303+0000","last_unstale":"2024-09-13T17:57:29.749009+0000","last_undegraded":"2024-09-13T17:57:29.749009+0000","last_fullsized":"2024-09-13T17:57:29.749009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:06.371464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725823+0000","last_change":"2024-09-13T17:57:25.717385+0000","last_active":"2024-09-13T17:57:29.725823+0000","last_peered":"2024-09-13T17:57:29.725823+0000","last_clean":"2024-09-13T17:57:29.725823+0000","last_became_active":"2024-09-13T17:57:25.717042+0000","last_became_peered":"2024-09-13T17:57:25.717042+0000","last_unstale":"2024-09-13T17:57:29.725823+0000","last_undegraded":"2024-09-13T17:57:29.725823+0000","last_fullsized":"2024-09-13T17:57:29.725823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:49.905668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732190+0000","last_change":"2024-09-13T17:57:25.369802+0000","last_active":"2024-09-13T17:57:29.732190+0000","last_peered":"2024-09-13T17:57:29.732190+0000","last_clean":"2024-09-13T17:57:29.732190+0000","last_became_active":"2024-09-13T17:57:25.369534+0000","last_became_peered":"2024-09-13T17:57:25.369534+0000","last_unstale":"2024-09-13T17:57:29.732190+0000","last_undegraded":"2024-09-13T17:57:29.732190+0000","last_fullsized":"2024-09-13T17:57:29.732190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742623+0000","last_change":"2024-09-13T17:57:29.737826+0000","last_active":"2024-09-13T17:57:29.742623+0000","last_peered":"2024-09-13T17:57:29.742623+0000","last_clean":"2024-09-13T17:57:29.742623+0000","last_became_active":"2024-09-13T17:57:29.737181+0000","last_became_peered":"2024-09-13T17:57:29.737181+0000","last_unstale":"2024-09-13T17:57:29.742623+0000","last_undegraded":"2024-09-13T17:57:29.742623+0000","last_fullsized":"2024-09-13T17:57:29.742623+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:06.225969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728642+0000","last_change":"2024-09-13T17:57:25.734250+0000","last_active":"2024-09-13T17:57:29.728642+0000","last_peered":"2024-09-13T17:57:29.728642+0000","last_clean":"2024-09-13T17:57:29.728642+0000","last_became_active":"2024-09-13T17:57:25.733836+0000","last_became_peered":"2024-09-13T17:57:25.733836+0000","last_unstale":"2024-09-13T17:57:29.728642+0000","last_undegraded":"2024-09-13T17:57:29.728642+0000","last_fullsized":"2024-09-13T17:57:29.728642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:05.710729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728072+0000","last_change":"2024-09-13T17:57:25.374153+0000","last_active":"2024-09-13T17:57:29.728072+0000","last_peered":"2024-09-13T17:57:29.728072+0000","last_clean":"2024-09-13T17:57:29.728072+0000","last_became_active":"2024-09-13T17:57:25.373891+0000","last_became_peered":"2024-09-13T17:57:25.373891+0000","last_unstale":"2024-09-13T17:57:29.728072+0000","last_undegraded":"2024-09-13T17:57:29.728072+0000","last_fullsized":"2024-09-13T17:57:29.728072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744947+0000","last_change":"2024-09-13T17:57:29.741648+0000","last_active":"2024-09-13T17:57:29.744947+0000","last_peered":"2024-09-13T17:57:29.744947+0000","last_clean":"2024-09-13T17:57:29.744947+0000","last_became_active":"2024-09-13T17:57:29.741315+0000","last_became_peered":"2024-09-13T17:57:29.741315+0000","last_unstale":"2024-09-13T17:57:29.744947+0000","last_undegraded":"2024-09-13T17:57:29.744947+0000","last_fullsized":"2024-09-13T17:57:29.744947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:18.432911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725228+0000","last_change":"2024-09-13T17:57:25.716312+0000","last_active":"2024-09-13T17:57:29.725228+0000","last_peered":"2024-09-13T17:57:29.725228+0000","last_clean":"2024-09-13T17:57:29.725228+0000","last_became_active":"2024-09-13T17:57:25.716067+0000","last_became_peered":"2024-09-13T17:57:25.716067+0000","last_unstale":"2024-09-13T17:57:29.725228+0000","last_undegraded":"2024-09-13T17:57:29.725228+0000","last_fullsized":"2024-09-13T17:57:29.725228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:33.971487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724948+0000","last_change":"2024-09-13T17:57:25.380277+0000","last_active":"2024-09-13T17:57:29.724948+0000","last_peered":"2024-09-13T17:57:29.724948+0000","last_clean":"2024-09-13T17:57:29.724948+0000","last_became_active":"2024-09-13T17:57:25.379988+0000","last_became_peered":"2024-09-13T17:57:25.379988+0000","last_unstale":"2024-09-13T17:57:29.724948+0000","last_undegraded":"2024-09-13T17:57:29.724948+0000","last_fullsized":"2024-09-13T17:57:29.724948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744773+0000","last_change":"2024-09-13T17:57:29.733138+0000","last_active":"2024-09-13T17:57:29.744773+0000","last_peered":"2024-09-13T17:57:29.744773+0000","last_clean":"2024-09-13T17:57:29.744773+0000","last_became_active":"2024-09-13T17:57:29.732804+0000","last_became_peered":"2024-09-13T17:57:29.732804+0000","last_unstale":"2024-09-13T17:57:29.744773+0000","last_undegraded":"2024-09-13T17:57:29.744773+0000","last_fullsized":"2024-09-13T17:57:29.744773+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:08.695591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724921+0000","last_change":"2024-09-13T17:57:25.714463+0000","last_active":"2024-09-13T17:57:29.724921+0000","last_peered":"2024-09-13T17:57:29.724921+0000","last_clean":"2024-09-13T17:57:29.724921+0000","last_became_active":"2024-09-13T17:57:25.714209+0000","last_became_peered":"2024-09-13T17:57:25.714209+0000","last_unstale":"2024-09-13T17:57:29.724921+0000","last_undegraded":"2024-09-13T17:57:29.724921+0000","last_fullsized":"2024-09-13T17:57:29.724921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727847+0000","last_change":"2024-09-13T17:57:25.374886+0000","last_active":"2024-09-13T17:57:29.727847+0000","last_peered":"2024-09-13T17:57:29.727847+0000","last_clean":"2024-09-13T17:57:29.727847+0000","last_became_active":"2024-09-13T17:57:25.374025+0000","last_became_peered":"2024-09-13T17:57:25.374025+0000","last_unstale":"2024-09-13T17:57:29.727847+0000","last_undegraded":"2024-09-13T17:57:29.727847+0000","last_fullsized":"2024-09-13T17:57:29.727847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:41.189680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.739379+0000","last_change":"2024-09-13T17:57:29.735654+0000","last_active":"2024-09-13T17:57:29.739379+0000","last_peered":"2024-09-13T17:57:29.739379+0000","last_clean":"2024-09-13T17:57:29.739379+0000","last_became_active":"2024-09-13T17:57:29.735267+0000","last_became_peered":"2024-09-13T17:57:29.735267+0000","last_unstale":"2024-09-13T17:57:29.739379+0000","last_undegraded":"2024-09-13T17:57:29.739379+0000","last_fullsized":"2024-09-13T17:57:29.739379+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:59:14.990893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725041+0000","last_change":"2024-09-13T17:57:25.722101+0000","last_active":"2024-09-13T17:57:29.725041+0000","last_peered":"2024-09-13T17:57:29.725041+0000","last_clean":"2024-09-13T17:57:29.725041+0000","last_became_active":"2024-09-13T17:57:25.721845+0000","last_became_peered":"2024-09-13T17:57:25.721845+0000","last_unstale":"2024-09-13T17:57:29.725041+0000","last_undegraded":"2024-09-13T17:57:29.725041+0000","last_fullsized":"2024-09-13T17:57:29.725041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:11.996031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:57:34.418487+0000","last_change":"2024-09-13T17:57:25.378020+0000","last_active":"2024-09-13T17:57:34.418487+0000","last_peered":"2024-09-13T17:57:34.418487+0000","last_clean":"2024-09-13T17:57:34.418487+0000","last_became_active":"2024-09-13T17:57:25.377746+0000","last_became_peered":"2024-09-13T17:57:25.377746+0000","last_unstale":"2024-09-13T17:57:34.418487+0000","last_undegraded":"2024-09-13T17:57:34.418487+0000","last_fullsized":"2024-09-13T17:57:34.418487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:58.248434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742293+0000","last_change":"2024-09-13T17:57:29.738264+0000","last_active":"2024-09-13T17:57:29.742293+0000","last_peered":"2024-09-13T17:57:29.742293+0000","last_clean":"2024-09-13T17:57:29.742293+0000","last_became_active":"2024-09-13T17:57:29.737897+0000","last_became_peered":"2024-09-13T17:57:29.737897+0000","last_unstale":"2024-09-13T17:57:29.742293+0000","last_undegraded":"2024-09-13T17:57:29.742293+0000","last_fullsized":"2024-09-13T17:57:29.742293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:59:14.990893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.735362+0000","last_change":"2024-09-13T17:57:25.720986+0000","last_active":"2024-09-13T17:57:29.735362+0000","last_peered":"2024-09-13T17:57:29.735362+0000","last_clean":"2024-09-13T17:57:29.735362+0000","last_became_active":"2024-09-13T17:57:25.720696+0000","last_became_peered":"2024-09-13T17:57:25.720696+0000","last_unstale":"2024-09-13T17:57:29.735362+0000","last_undegraded":"2024-09-13T17:57:29.735362+0000","last_fullsized":"2024-09-13T17:57:29.735362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:01.729310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729997+0000","last_change":"2024-09-13T17:57:25.376707+0000","last_active":"2024-09-13T17:57:29.729997+0000","last_peered":"2024-09-13T17:57:29.729997+0000","last_clean":"2024-09-13T17:57:29.729997+0000","last_became_active":"2024-09-13T17:57:25.376080+0000","last_became_peered":"2024-09-13T17:57:25.376080+0000","last_unstale":"2024-09-13T17:57:29.729997+0000","last_undegraded":"2024-09-13T17:57:29.729997+0000","last_fullsized":"2024-09-13T17:57:29.729997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:28.985664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744127+0000","last_change":"2024-09-13T17:57:29.741053+0000","last_active":"2024-09-13T17:57:29.744127+0000","last_peered":"2024-09-13T17:57:29.744127+0000","last_clean":"2024-09-13T17:57:29.744127+0000","last_became_active":"2024-09-13T17:57:29.740751+0000","last_became_peered":"2024-09-13T17:57:29.740751+0000","last_unstale":"2024-09-13T17:57:29.744127+0000","last_undegraded":"2024-09-13T17:57:29.744127+0000","last_fullsized":"2024-09-13T17:57:29.744127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:56:53.973670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728126+0000","last_change":"2024-09-13T17:57:25.725767+0000","last_active":"2024-09-13T17:57:29.728126+0000","last_peered":"2024-09-13T17:57:29.728126+0000","last_clean":"2024-09-13T17:57:29.728126+0000","last_became_active":"2024-09-13T17:57:25.725565+0000","last_became_peered":"2024-09-13T17:57:25.725565+0000","last_unstale":"2024-09-13T17:57:29.728126+0000","last_undegraded":"2024-09-13T17:57:29.728126+0000","last_fullsized":"2024-09-13T17:57:29.728126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:19.536367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725379+0000","last_change":"2024-09-13T17:57:25.380232+0000","last_active":"2024-09-13T17:57:29.725379+0000","last_peered":"2024-09-13T17:57:29.725379+0000","last_clean":"2024-09-13T17:57:29.725379+0000","last_became_active":"2024-09-13T17:57:25.379910+0000","last_became_peered":"2024-09-13T17:57:25.379910+0000","last_unstale":"2024-09-13T17:57:29.725379+0000","last_undegraded":"2024-09-13T17:57:29.725379+0000","last_fullsized":"2024-09-13T17:57:29.725379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:28.958967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749501+0000","last_change":"2024-09-13T17:57:29.745584+0000","last_active":"2024-09-13T17:57:29.749501+0000","last_peered":"2024-09-13T17:57:29.749501+0000","last_clean":"2024-09-13T17:57:29.749501+0000","last_became_active":"2024-09-13T17:57:29.745250+0000","last_became_peered":"2024-09-13T17:57:29.745250+0000","last_unstale":"2024-09-13T17:57:29.749501+0000","last_undegraded":"2024-09-13T17:57:29.749501+0000","last_fullsized":"2024-09-13T17:57:29.749501+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:12.564762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725293+0000","last_change":"2024-09-13T17:57:25.724239+0000","last_active":"2024-09-13T17:57:29.725293+0000","last_peered":"2024-09-13T17:57:29.725293+0000","last_clean":"2024-09-13T17:57:29.725293+0000","last_became_active":"2024-09-13T17:57:25.723999+0000","last_became_peered":"2024-09-13T17:57:25.723999+0000","last_unstale":"2024-09-13T17:57:29.725293+0000","last_undegraded":"2024-09-13T17:57:29.725293+0000","last_fullsized":"2024-09-13T17:57:29.725293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728742+0000","last_change":"2024-09-13T17:57:25.376498+0000","last_active":"2024-09-13T17:57:29.728742+0000","last_peered":"2024-09-13T17:57:29.728742+0000","last_clean":"2024-09-13T17:57:29.728742+0000","last_became_active":"2024-09-13T17:57:25.376171+0000","last_became_peered":"2024-09-13T17:57:25.376171+0000","last_unstale":"2024-09-13T17:57:29.728742+0000","last_undegraded":"2024-09-13T17:57:29.728742+0000","last_fullsized":"2024-09-13T17:57:29.728742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:00.725292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744624+0000","last_change":"2024-09-13T17:57:29.737219+0000","last_active":"2024-09-13T17:57:29.744624+0000","last_peered":"2024-09-13T17:57:29.744624+0000","last_clean":"2024-09-13T17:57:29.744624+0000","last_became_active":"2024-09-13T17:57:29.736853+0000","last_became_peered":"2024-09-13T17:57:29.736853+0000","last_unstale":"2024-09-13T17:57:29.744624+0000","last_undegraded":"2024-09-13T17:57:29.744624+0000","last_fullsized":"2024-09-13T17:57:29.744624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:18.526568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.738047+0000","last_change":"2024-09-13T17:57:29.733906+0000","last_active":"2024-09-13T17:57:29.738047+0000","last_peered":"2024-09-13T17:57:29.738047+0000","last_clean":"2024-09-13T17:57:29.738047+0000","last_became_active":"2024-09-13T17:57:29.733543+0000","last_became_peered":"2024-09-13T17:57:29.733543+0000","last_unstale":"2024-09-13T17:57:29.738047+0000","last_undegraded":"2024-09-13T17:57:29.738047+0000","last_fullsized":"2024-09-13T17:57:29.738047+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:37.101711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729014+0000","last_change":"2024-09-13T17:57:25.721000+0000","last_active":"2024-09-13T17:57:29.729014+0000","last_peered":"2024-09-13T17:57:29.729014+0000","last_clean":"2024-09-13T17:57:29.729014+0000","last_became_active":"2024-09-13T17:57:25.720696+0000","last_became_peered":"2024-09-13T17:57:25.720696+0000","last_unstale":"2024-09-13T17:57:29.729014+0000","last_undegraded":"2024-09-13T17:57:29.729014+0000","last_fullsized":"2024-09-13T17:57:29.729014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:34.797142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724465+0000","last_change":"2024-09-13T17:57:25.374530+0000","last_active":"2024-09-13T17:57:29.724465+0000","last_peered":"2024-09-13T17:57:29.724465+0000","last_clean":"2024-09-13T17:57:29.724465+0000","last_became_active":"2024-09-13T17:57:25.374296+0000","last_became_peered":"2024-09-13T17:57:25.374296+0000","last_unstale":"2024-09-13T17:57:29.724465+0000","last_undegraded":"2024-09-13T17:57:29.724465+0000","last_fullsized":"2024-09-13T17:57:29.724465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:15.663292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744054+0000","last_change":"2024-09-13T17:57:29.740208+0000","last_active":"2024-09-13T17:57:29.744054+0000","last_peered":"2024-09-13T17:57:29.744054+0000","last_clean":"2024-09-13T17:57:29.744054+0000","last_became_active":"2024-09-13T17:57:29.739839+0000","last_became_peered":"2024-09-13T17:57:29.739839+0000","last_unstale":"2024-09-13T17:57:29.744054+0000","last_undegraded":"2024-09-13T17:57:29.744054+0000","last_fullsized":"2024-09-13T17:57:29.744054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:32.682495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721617+0000","last_change":"2024-09-13T17:57:25.720286+0000","last_active":"2024-09-13T17:57:29.721617+0000","last_peered":"2024-09-13T17:57:29.721617+0000","last_clean":"2024-09-13T17:57:29.721617+0000","last_became_active":"2024-09-13T17:57:25.719918+0000","last_became_peered":"2024-09-13T17:57:25.719918+0000","last_unstale":"2024-09-13T17:57:29.721617+0000","last_undegraded":"2024-09-13T17:57:29.721617+0000","last_fullsized":"2024-09-13T17:57:29.721617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:13.197645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721496+0000","last_change":"2024-09-13T17:57:25.380282+0000","last_active":"2024-09-13T17:57:29.721496+0000","last_peered":"2024-09-13T17:57:29.721496+0000","last_clean":"2024-09-13T17:57:29.721496+0000","last_became_active":"2024-09-13T17:57:25.379992+0000","last_became_peered":"2024-09-13T17:57:25.379992+0000","last_unstale":"2024-09-13T17:57:29.721496+0000","last_undegraded":"2024-09-13T17:57:29.721496+0000","last_fullsized":"2024-09-13T17:57:29.721496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:04.441662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728265+0000","last_change":"2024-09-13T17:57:25.373463+0000","last_active":"2024-09-13T17:57:29.728265+0000","last_peered":"2024-09-13T17:57:29.728265+0000","last_clean":"2024-09-13T17:57:29.728265+0000","last_became_active":"2024-09-13T17:57:25.373127+0000","last_became_peered":"2024-09-13T17:57:25.373127+0000","last_unstale":"2024-09-13T17:57:29.728265+0000","last_undegraded":"2024-09-13T17:57:29.728265+0000","last_fullsized":"2024-09-13T17:57:29.728265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:15.663292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727753+0000","last_change":"2024-09-13T17:57:25.719108+0000","last_active":"2024-09-13T17:57:29.727753+0000","last_peered":"2024-09-13T17:57:29.727753+0000","last_clean":"2024-09-13T17:57:29.727753+0000","last_became_active":"2024-09-13T17:57:25.718674+0000","last_became_peered":"2024-09-13T17:57:25.718674+0000","last_unstale":"2024-09-13T17:57:29.727753+0000","last_undegraded":"2024-09-13T17:57:29.727753+0000","last_fullsized":"2024-09-13T17:57:29.727753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:49.905668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.740987+0000","last_change":"2024-09-13T17:57:29.736695+0000","last_active":"2024-09-13T17:57:29.740987+0000","last_peered":"2024-09-13T17:57:29.740987+0000","last_clean":"2024-09-13T17:57:29.740987+0000","last_became_active":"2024-09-13T17:57:29.735361+0000","last_became_peered":"2024-09-13T17:57:29.735361+0000","last_unstale":"2024-09-13T17:57:29.740987+0000","last_undegraded":"2024-09-13T17:57:29.740987+0000","last_fullsized":"2024-09-13T17:57:29.740987+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:50.823708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720788+0000","last_change":"2024-09-13T17:57:25.378442+0000","last_active":"2024-09-13T17:57:29.720788+0000","last_peered":"2024-09-13T17:57:29.720788+0000","last_clean":"2024-09-13T17:57:29.720788+0000","last_became_active":"2024-09-13T17:57:25.378092+0000","last_became_peered":"2024-09-13T17:57:25.378092+0000","last_unstale":"2024-09-13T17:57:29.720788+0000","last_undegraded":"2024-09-13T17:57:29.720788+0000","last_fullsized":"2024-09-13T17:57:29.720788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:41.189680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.733727+0000","last_change":"2024-09-13T17:57:25.714970+0000","last_active":"2024-09-13T17:57:29.733727+0000","last_peered":"2024-09-13T17:57:29.733727+0000","last_clean":"2024-09-13T17:57:29.733727+0000","last_became_active":"2024-09-13T17:57:25.714338+0000","last_became_peered":"2024-09-13T17:57:25.714338+0000","last_unstale":"2024-09-13T17:57:29.733727+0000","last_undegraded":"2024-09-13T17:57:29.733727+0000","last_fullsized":"2024-09-13T17:57:29.733727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:11:58.896243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.745004+0000","last_change":"2024-09-13T17:57:29.741838+0000","last_active":"2024-09-13T17:57:29.745004+0000","last_peered":"2024-09-13T17:57:29.745004+0000","last_clean":"2024-09-13T17:57:29.745004+0000","last_became_active":"2024-09-13T17:57:29.741478+0000","last_became_peered":"2024-09-13T17:57:29.741478+0000","last_unstale":"2024-09-13T17:57:29.745004+0000","last_undegraded":"2024-09-13T17:57:29.745004+0000","last_fullsized":"2024-09-13T17:57:29.745004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:15.815856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728583+0000","last_change":"2024-09-13T17:57:25.734081+0000","last_active":"2024-09-13T17:57:29.728583+0000","last_peered":"2024-09-13T17:57:29.728583+0000","last_clean":"2024-09-13T17:57:29.728583+0000","last_became_active":"2024-09-13T17:57:25.733489+0000","last_became_peered":"2024-09-13T17:57:25.733489+0000","last_unstale":"2024-09-13T17:57:29.728583+0000","last_undegraded":"2024-09-13T17:57:29.728583+0000","last_fullsized":"2024-09-13T17:57:29.728583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:49.905668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.731942+0000","last_change":"2024-09-13T17:57:25.721413+0000","last_active":"2024-09-13T17:57:29.731942+0000","last_peered":"2024-09-13T17:57:29.731942+0000","last_clean":"2024-09-13T17:57:29.731942+0000","last_became_active":"2024-09-13T17:57:25.721167+0000","last_became_peered":"2024-09-13T17:57:25.721167+0000","last_unstale":"2024-09-13T17:57:29.731942+0000","last_undegraded":"2024-09-13T17:57:29.731942+0000","last_fullsized":"2024-09-13T17:57:29.731942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:39.042757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724511+0000","last_change":"2024-09-13T17:57:25.728836+0000","last_active":"2024-09-13T17:57:29.724511+0000","last_peered":"2024-09-13T17:57:29.724511+0000","last_clean":"2024-09-13T17:57:29.724511+0000","last_became_active":"2024-09-13T17:57:25.728566+0000","last_became_peered":"2024-09-13T17:57:25.728566+0000","last_unstale":"2024-09-13T17:57:29.724511+0000","last_undegraded":"2024-09-13T17:57:29.724511+0000","last_fullsized":"2024-09-13T17:57:29.724511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:50.812435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727862+0000","last_change":"2024-09-13T17:57:25.719773+0000","last_active":"2024-09-13T17:57:29.727862+0000","last_peered":"2024-09-13T17:57:29.727862+0000","last_clean":"2024-09-13T17:57:29.727862+0000","last_became_active":"2024-09-13T17:57:25.719057+0000","last_became_peered":"2024-09-13T17:57:25.719057+0000","last_unstale":"2024-09-13T17:57:29.727862+0000","last_undegraded":"2024-09-13T17:57:29.727862+0000","last_fullsized":"2024-09-13T17:57:29.727862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:13.197645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721032+0000","last_change":"2024-09-13T17:57:25.719998+0000","last_active":"2024-09-13T17:57:29.721032+0000","last_peered":"2024-09-13T17:57:29.721032+0000","last_clean":"2024-09-13T17:57:29.721032+0000","last_became_active":"2024-09-13T17:57:25.719739+0000","last_became_peered":"2024-09-13T17:57:25.719739+0000","last_unstale":"2024-09-13T17:57:29.721032+0000","last_undegraded":"2024-09-13T17:57:29.721032+0000","last_fullsized":"2024-09-13T17:57:29.721032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:47.463588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729592+0000","last_change":"2024-09-13T17:57:25.720277+0000","last_active":"2024-09-13T17:57:29.729592+0000","last_peered":"2024-09-13T17:57:29.729592+0000","last_clean":"2024-09-13T17:57:29.729592+0000","last_became_active":"2024-09-13T17:57:25.720034+0000","last_became_peered":"2024-09-13T17:57:25.720034+0000","last_unstale":"2024-09-13T17:57:29.729592+0000","last_undegraded":"2024-09-13T17:57:29.729592+0000","last_fullsized":"2024-09-13T17:57:29.729592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728536+0000","last_change":"2024-09-13T17:57:25.733794+0000","last_active":"2024-09-13T17:57:29.728536+0000","last_peered":"2024-09-13T17:57:29.728536+0000","last_clean":"2024-09-13T17:57:29.728536+0000","last_became_active":"2024-09-13T17:57:25.733512+0000","last_became_peered":"2024-09-13T17:57:25.733512+0000","last_unstale":"2024-09-13T17:57:29.728536+0000","last_undegraded":"2024-09-13T17:57:29.728536+0000","last_fullsized":"2024-09-13T17:57:29.728536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720839+0000","last_change":"2024-09-13T17:57:25.723300+0000","last_active":"2024-09-13T17:57:29.720839+0000","last_peered":"2024-09-13T17:57:29.720839+0000","last_clean":"2024-09-13T17:57:29.720839+0000","last_became_active":"2024-09-13T17:57:25.720475+0000","last_became_peered":"2024-09-13T17:57:25.720475+0000","last_unstale":"2024-09-13T17:57:29.720839+0000","last_undegraded":"2024-09-13T17:57:29.720839+0000","last_fullsized":"2024-09-13T17:57:29.720839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:37:23.859769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724381+0000","last_change":"2024-09-13T17:57:25.721201+0000","last_active":"2024-09-13T17:57:29.724381+0000","last_peered":"2024-09-13T17:57:29.724381+0000","last_clean":"2024-09-13T17:57:29.724381+0000","last_became_active":"2024-09-13T17:57:25.719998+0000","last_became_peered":"2024-09-13T17:57:25.719998+0000","last_unstale":"2024-09-13T17:57:29.724381+0000","last_undegraded":"2024-09-13T17:57:29.724381+0000","last_fullsized":"2024-09-13T17:57:29.724381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:33.971487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721180+0000","last_change":"2024-09-13T17:57:25.715726+0000","last_active":"2024-09-13T17:57:29.721180+0000","last_peered":"2024-09-13T17:57:29.721180+0000","last_clean":"2024-09-13T17:57:29.721180+0000","last_became_active":"2024-09-13T17:57:25.715459+0000","last_became_peered":"2024-09-13T17:57:25.715459+0000","last_unstale":"2024-09-13T17:57:29.721180+0000","last_undegraded":"2024-09-13T17:57:29.721180+0000","last_fullsized":"2024-09-13T17:57:29.721180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:49.905668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724413+0000","last_change":"2024-09-13T17:57:25.730059+0000","last_active":"2024-09-13T17:57:29.724413+0000","last_peered":"2024-09-13T17:57:29.724413+0000","last_clean":"2024-09-13T17:57:29.724413+0000","last_became_active":"2024-09-13T17:57:25.729083+0000","last_became_peered":"2024-09-13T17:57:25.729083+0000","last_unstale":"2024-09-13T17:57:29.724413+0000","last_undegraded":"2024-09-13T17:57:29.724413+0000","last_fullsized":"2024-09-13T17:57:29.724413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:12.960061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727633+0000","last_change":"2024-09-13T17:57:25.713791+0000","last_active":"2024-09-13T17:57:29.727633+0000","last_peered":"2024-09-13T17:57:29.727633+0000","last_clean":"2024-09-13T17:57:29.727633+0000","last_became_active":"2024-09-13T17:57:25.713540+0000","last_became_peered":"2024-09-13T17:57:25.713540+0000","last_unstale":"2024-09-13T17:57:29.727633+0000","last_undegraded":"2024-09-13T17:57:29.727633+0000","last_fullsized":"2024-09-13T17:57:29.727633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728046+0000","last_change":"2024-09-13T17:57:25.734509+0000","last_active":"2024-09-13T17:57:29.728046+0000","last_peered":"2024-09-13T17:57:29.728046+0000","last_clean":"2024-09-13T17:57:29.728046+0000","last_became_active":"2024-09-13T17:57:25.734233+0000","last_became_peered":"2024-09-13T17:57:25.734233+0000","last_unstale":"2024-09-13T17:57:29.728046+0000","last_undegraded":"2024-09-13T17:57:29.728046+0000","last_fullsized":"2024-09-13T17:57:29.728046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725310+0000","last_change":"2024-09-13T17:57:25.718661+0000","last_active":"2024-09-13T17:57:29.725310+0000","last_peered":"2024-09-13T17:57:29.725310+0000","last_clean":"2024-09-13T17:57:29.725310+0000","last_became_active":"2024-09-13T17:57:25.718366+0000","last_became_peered":"2024-09-13T17:57:25.718366+0000","last_unstale":"2024-09-13T17:57:29.725310+0000","last_undegraded":"2024-09-13T17:57:29.725310+0000","last_fullsized":"2024-09-13T17:57:29.725310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:12.960061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732206+0000","last_change":"2024-09-13T17:57:25.734683+0000","last_active":"2024-09-13T17:57:29.732206+0000","last_peered":"2024-09-13T17:57:29.732206+0000","last_clean":"2024-09-13T17:57:29.732206+0000","last_became_active":"2024-09-13T17:57:25.734409+0000","last_became_peered":"2024-09-13T17:57:25.734409+0000","last_unstale":"2024-09-13T17:57:29.732206+0000","last_undegraded":"2024-09-13T17:57:29.732206+0000","last_fullsized":"2024-09-13T17:57:29.732206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:59.252453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725864+0000","last_change":"2024-09-13T17:57:25.729680+0000","last_active":"2024-09-13T17:57:29.725864+0000","last_peered":"2024-09-13T17:57:29.725864+0000","last_clean":"2024-09-13T17:57:29.725864+0000","last_became_active":"2024-09-13T17:57:25.729402+0000","last_became_peered":"2024-09-13T17:57:25.729402+0000","last_unstale":"2024-09-13T17:57:29.725864+0000","last_undegraded":"2024-09-13T17:57:29.725864+0000","last_fullsized":"2024-09-13T17:57:29.725864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:43.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724296+0000","last_change":"2024-09-13T17:57:25.728126+0000","last_active":"2024-09-13T17:57:29.724296+0000","last_peered":"2024-09-13T17:57:29.724296+0000","last_clean":"2024-09-13T17:57:29.724296+0000","last_became_active":"2024-09-13T17:57:25.727793+0000","last_became_peered":"2024-09-13T17:57:25.727793+0000","last_unstale":"2024-09-13T17:57:29.724296+0000","last_undegraded":"2024-09-13T17:57:29.724296+0000","last_fullsized":"2024-09-13T17:57:29.724296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:34.797142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729099+0000","last_change":"2024-09-13T17:57:25.716918+0000","last_active":"2024-09-13T17:57:29.729099+0000","last_peered":"2024-09-13T17:57:29.729099+0000","last_clean":"2024-09-13T17:57:29.729099+0000","last_became_active":"2024-09-13T17:57:25.716550+0000","last_became_peered":"2024-09-13T17:57:25.716550+0000","last_unstale":"2024-09-13T17:57:29.729099+0000","last_undegraded":"2024-09-13T17:57:29.729099+0000","last_fullsized":"2024-09-13T17:57:29.729099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:19.536367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724836+0000","last_change":"2024-09-13T17:57:25.717079+0000","last_active":"2024-09-13T17:57:29.724836+0000","last_peered":"2024-09-13T17:57:29.724836+0000","last_clean":"2024-09-13T17:57:29.724836+0000","last_became_active":"2024-09-13T17:57:25.716782+0000","last_became_peered":"2024-09-13T17:57:25.716782+0000","last_unstale":"2024-09-13T17:57:29.724836+0000","last_undegraded":"2024-09-13T17:57:29.724836+0000","last_fullsized":"2024-09-13T17:57:29.724836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:26:19.049804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720610+0000","last_change":"2024-09-13T17:57:25.731733+0000","last_active":"2024-09-13T17:57:29.720610+0000","last_peered":"2024-09-13T17:57:29.720610+0000","last_clean":"2024-09-13T17:57:29.720610+0000","last_became_active":"2024-09-13T17:57:25.731460+0000","last_became_peered":"2024-09-13T17:57:25.731460+0000","last_unstale":"2024-09-13T17:57:29.720610+0000","last_undegraded":"2024-09-13T17:57:29.720610+0000","last_fullsized":"2024-09-13T17:57:29.720610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:43.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.735275+0000","last_change":"2024-09-13T17:57:25.734897+0000","last_active":"2024-09-13T17:57:29.735275+0000","last_peered":"2024-09-13T17:57:29.735275+0000","last_clean":"2024-09-13T17:57:29.735275+0000","last_became_active":"2024-09-13T17:57:25.734639+0000","last_became_peered":"2024-09-13T17:57:25.734639+0000","last_unstale":"2024-09-13T17:57:29.735275+0000","last_undegraded":"2024-09-13T17:57:29.735275+0000","last_fullsized":"2024-09-13T17:57:29.735275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:29.989683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728450+0000","last_change":"2024-09-13T17:57:25.736997+0000","last_active":"2024-09-13T17:57:29.728450+0000","last_peered":"2024-09-13T17:57:29.728450+0000","last_clean":"2024-09-13T17:57:29.728450+0000","last_became_active":"2024-09-13T17:57:25.736735+0000","last_became_peered":"2024-09-13T17:57:25.736735+0000","last_unstale":"2024-09-13T17:57:29.728450+0000","last_undegraded":"2024-09-13T17:57:29.728450+0000","last_fullsized":"2024-09-13T17:57:29.728450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728451+0000","last_change":"2024-09-13T17:57:25.736821+0000","last_active":"2024-09-13T17:57:29.728451+0000","last_peered":"2024-09-13T17:57:29.728451+0000","last_clean":"2024-09-13T17:57:29.728451+0000","last_became_active":"2024-09-13T17:57:25.736572+0000","last_became_peered":"2024-09-13T17:57:25.736572+0000","last_unstale":"2024-09-13T17:57:29.728451+0000","last_undegraded":"2024-09-13T17:57:29.728451+0000","last_fullsized":"2024-09-13T17:57:29.728451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:13.197645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725470+0000","last_change":"2024-09-13T17:57:25.725989+0000","last_active":"2024-09-13T17:57:29.725470+0000","last_peered":"2024-09-13T17:57:29.725470+0000","last_clean":"2024-09-13T17:57:29.725470+0000","last_became_active":"2024-09-13T17:57:25.725725+0000","last_became_peered":"2024-09-13T17:57:25.725725+0000","last_unstale":"2024-09-13T17:57:29.725470+0000","last_undegraded":"2024-09-13T17:57:29.725470+0000","last_fullsized":"2024-09-13T17:57:29.725470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:50.812435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720423+0000","last_change":"2024-09-13T17:57:25.723786+0000","last_active":"2024-09-13T17:57:29.720423+0000","last_peered":"2024-09-13T17:57:29.720423+0000","last_clean":"2024-09-13T17:57:29.720423+0000","last_became_active":"2024-09-13T17:57:25.723510+0000","last_became_peered":"2024-09-13T17:57:25.723510+0000","last_unstale":"2024-09-13T17:57:29.720423+0000","last_undegraded":"2024-09-13T17:57:29.720423+0000","last_fullsized":"2024-09-13T17:57:29.720423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725741+0000","last_change":"2024-09-13T17:57:25.721799+0000","last_active":"2024-09-13T17:57:29.725741+0000","last_peered":"2024-09-13T17:57:29.725741+0000","last_clean":"2024-09-13T17:57:29.725741+0000","last_became_active":"2024-09-13T17:57:25.721309+0000","last_became_peered":"2024-09-13T17:57:25.721309+0000","last_unstale":"2024-09-13T17:57:29.725741+0000","last_undegraded":"2024-09-13T17:57:29.725741+0000","last_fullsized":"2024-09-13T17:57:29.725741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:27.969805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724305+0000","last_change":"2024-09-13T17:57:25.727329+0000","last_active":"2024-09-13T17:57:29.724305+0000","last_peered":"2024-09-13T17:57:29.724305+0000","last_clean":"2024-09-13T17:57:29.724305+0000","last_became_active":"2024-09-13T17:57:25.727064+0000","last_became_peered":"2024-09-13T17:57:25.727064+0000","last_unstale":"2024-09-13T17:57:29.724305+0000","last_undegraded":"2024-09-13T17:57:29.724305+0000","last_fullsized":"2024-09-13T17:57:29.724305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725134+0000","last_change":"2024-09-13T17:57:25.714493+0000","last_active":"2024-09-13T17:57:29.725134+0000","last_peered":"2024-09-13T17:57:29.725134+0000","last_clean":"2024-09-13T17:57:29.725134+0000","last_became_active":"2024-09-13T17:57:25.714091+0000","last_became_peered":"2024-09-13T17:57:25.714091+0000","last_unstale":"2024-09-13T17:57:29.725134+0000","last_undegraded":"2024-09-13T17:57:29.725134+0000","last_fullsized":"2024-09-13T17:57:29.725134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:51.804478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725105+0000","last_change":"2024-09-13T17:57:25.730275+0000","last_active":"2024-09-13T17:57:29.725105+0000","last_peered":"2024-09-13T17:57:29.725105+0000","last_clean":"2024-09-13T17:57:29.725105+0000","last_became_active":"2024-09-13T17:57:25.729845+0000","last_became_peered":"2024-09-13T17:57:25.729845+0000","last_unstale":"2024-09-13T17:57:29.725105+0000","last_undegraded":"2024-09-13T17:57:29.725105+0000","last_fullsized":"2024-09-13T17:57:29.725105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:11.996031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725148+0000","last_change":"2024-09-13T17:57:25.714547+0000","last_active":"2024-09-13T17:57:29.725148+0000","last_peered":"2024-09-13T17:57:29.725148+0000","last_clean":"2024-09-13T17:57:29.725148+0000","last_became_active":"2024-09-13T17:57:25.714288+0000","last_became_peered":"2024-09-13T17:57:25.714288+0000","last_unstale":"2024-09-13T17:57:29.725148+0000","last_undegraded":"2024-09-13T17:57:29.725148+0000","last_fullsized":"2024-09-13T17:57:29.725148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:37:23.859769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729416+0000","last_change":"2024-09-13T17:57:25.727117+0000","last_active":"2024-09-13T17:57:29.729416+0000","last_peered":"2024-09-13T17:57:29.729416+0000","last_clean":"2024-09-13T17:57:29.729416+0000","last_became_active":"2024-09-13T17:57:25.726864+0000","last_became_peered":"2024-09-13T17:57:25.726864+0000","last_unstale":"2024-09-13T17:57:29.729416+0000","last_undegraded":"2024-09-13T17:57:29.729416+0000","last_fullsized":"2024-09-13T17:57:29.729416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725395+0000","last_change":"2024-09-13T17:57:25.734566+0000","last_active":"2024-09-13T17:57:29.725395+0000","last_peered":"2024-09-13T17:57:29.725395+0000","last_clean":"2024-09-13T17:57:29.725395+0000","last_became_active":"2024-09-13T17:57:25.733963+0000","last_became_peered":"2024-09-13T17:57:25.733963+0000","last_unstale":"2024-09-13T17:57:29.725395+0000","last_undegraded":"2024-09-13T17:57:29.725395+0000","last_fullsized":"2024-09-13T17:57:29.725395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:44.248672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":690730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":99870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":99906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":99780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":99940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":689767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:57:47.286 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:57:47.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.412+0000 7fa44a2d0640 1 Processor -- start 2024-09-13T17:57:47.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.412+0000 7fa44a2d0640 1 -- start start 2024-09-13T17:57:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.412+0000 7fa44a2d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa444071690 0x7fa4440ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.413+0000 7fa44a2d0640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44406f2b0 con 0x7fa4440b3e40 2024-09-13T17:57:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.413+0000 7fa44a2d0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44406e2f0 con 0x7fa444069ed0 2024-09-13T17:57:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.413+0000 7fa44a2d0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa444071b00 con 0x7fa444071690 2024-09-13T17:57:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.413+0000 7fa44886d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa4440b3e40 0x7fa4441671d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49648/0 (socket says 172.21.15.111:49648) 2024-09-13T17:57:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.413+0000 7fa44886d640 1 -- 172.21.15.111:0/2851953132 learned_addr learned my addr 172.21.15.111:0/2851953132 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.413+0000 7fa4437fe640 1 --2- 172.21.15.111:0/2851953132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa444071690 0x7fa4440ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.413+0000 7fa442ffd640 1 -- 172.21.15.111:0/2851953132 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 543080600 0 0) 0x7fa44406f2b0 con 0x7fa4440b3e40 2024-09-13T17:57:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.413+0000 7fa442ffd640 1 -- 172.21.15.111:0/2851953132 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa42c003520 con 0x7fa4440b3e40 2024-09-13T17:57:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.413+0000 7fa442ffd640 1 -- 172.21.15.111:0/2851953132 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659135240 0 0) 0x7fa44406e2f0 con 0x7fa444069ed0 2024-09-13T17:57:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.413+0000 7fa442ffd640 1 -- 172.21.15.111:0/2851953132 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa44406f2b0 con 0x7fa444069ed0 2024-09-13T17:57:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.413+0000 7fa4437fe640 1 -- 172.21.15.111:0/2851953132 >> v1:172.21.15.188:6789/0 conn(0x7fa444069ed0 legacy=0x7fa44406a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.413+0000 7fa4437fe640 1 -- 172.21.15.111:0/2851953132 >> v1:172.21.15.111:6789/0 conn(0x7fa4440b3e40 legacy=0x7fa4441671d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.413+0000 7fa4437fe640 1 -- 172.21.15.111:0/2851953132 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4440b28c0 con 0x7fa444071690 2024-09-13T17:57:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.414+0000 7fa4437fe640 1 --2- 172.21.15.111:0/2851953132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa444071690 0x7fa4440ac1e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa4380097d0 tx=0x7fa43802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d8045682a99f037 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.414+0000 7fa442ffd640 1 -- 172.21.15.111:0/2851953132 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa438032440 con 0x7fa444071690 2024-09-13T17:57:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.414+0000 7fa442ffd640 1 -- 172.21.15.111:0/2851953132 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4380325a0 con 0x7fa444071690 2024-09-13T17:57:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.414+0000 7fa442ffd640 1 -- 172.21.15.111:0/2851953132 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa438032930 con 0x7fa444071690 2024-09-13T17:57:47.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.414+0000 7fa44a2d0640 1 -- 172.21.15.111:0/2851953132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa444071690 msgr2=0x7fa4440ac1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.414+0000 7fa44a2d0640 1 --2- 172.21.15.111:0/2851953132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa444071690 0x7fa4440ac1e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa4380097d0 tx=0x7fa43802a180 comp rx=0 tx=0).stop 2024-09-13T17:57:47.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.415+0000 7fa44a2d0640 1 -- 172.21.15.111:0/2851953132 shutdown_connections 2024-09-13T17:57:47.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.415+0000 7fa44a2d0640 1 --2- 172.21.15.111:0/2851953132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa444071690 0x7fa4440ac1e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.415+0000 7fa44a2d0640 1 -- 172.21.15.111:0/2851953132 >> 172.21.15.111:0/2851953132 conn(0x7fa4440b49b0 msgr2=0x7fa4440b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:47.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.415+0000 7fa44a2d0640 1 -- 172.21.15.111:0/2851953132 shutdown_connections 2024-09-13T17:57:47.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.415+0000 7fa44a2d0640 1 -- 172.21.15.111:0/2851953132 wait complete. 2024-09-13T17:57:47.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.416+0000 7fa44a2d0640 1 Processor -- start 2024-09-13T17:57:47.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.416+0000 7fa44a2d0640 1 -- start start 2024-09-13T17:57:47.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa44a2d0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa444069ed0 0x7fa444155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:47.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa44a2d0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa444071690 0x7fa4441f3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:47.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa44a2d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4440b3e40 0x7fa44415faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:47.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa44a2d0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4441558c0 con 0x7fa444071690 2024-09-13T17:57:47.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa44a2d0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa444148560 con 0x7fa444069ed0 2024-09-13T17:57:47.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa44a2d0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4441f38d0 con 0x7fa4440b3e40 2024-09-13T17:57:47.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa4437fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa444071690 0x7fa4441f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:47.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa4437fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa444071690 0x7fa4441f3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53870/0 (socket says 172.21.15.111:53870) 2024-09-13T17:57:47.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa4437fe640 1 -- 172.21.15.111:0/3140807494 learned_addr learned my addr 172.21.15.111:0/3140807494 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:47.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa443fff640 1 --2- 172.21.15.111:0/3140807494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa444069ed0 0x7fa444155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:47.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa44886d640 1 --2- 172.21.15.111:0/3140807494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4440b3e40 0x7fa44415faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:47.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa4437fe640 1 -- 172.21.15.111:0/3140807494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4440b3e40 msgr2=0x7fa44415faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa4437fe640 1 --2- 172.21.15.111:0/3140807494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4440b3e40 0x7fa44415faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa4437fe640 1 -- 172.21.15.111:0/3140807494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa444069ed0 msgr2=0x7fa444155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa4437fe640 1 --2- 172.21.15.111:0/3140807494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa444069ed0 0x7fa444155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa4437fe640 1 -- 172.21.15.111:0/3140807494 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa438009400 con 0x7fa444071690 2024-09-13T17:57:47.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa4437fe640 1 --2- 172.21.15.111:0/3140807494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa444071690 0x7fa4441f3390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa4380098f0 tx=0x7fa4380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:47.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa440ff9640 1 -- 172.21.15.111:0/3140807494 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43803d8f0 con 0x7fa444071690 2024-09-13T17:57:47.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa440ff9640 1 -- 172.21.15.111:0/3140807494 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43803da70 con 0x7fa444071690 2024-09-13T17:57:47.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa44a2d0640 1 -- 172.21.15.111:0/3140807494 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4440ac510 con 0x7fa444071690 2024-09-13T17:57:47.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa440ff9640 1 -- 172.21.15.111:0/3140807494 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43803de20 con 0x7fa444071690 2024-09-13T17:57:47.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.417+0000 7fa44a2d0640 1 -- 172.21.15.111:0/3140807494 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4440acad0 con 0x7fa444071690 2024-09-13T17:57:47.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.418+0000 7fa44a2d0640 1 -- 172.21.15.111:0/3140807494 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44406e890 con 0x7fa444071690 2024-09-13T17:57:47.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.419+0000 7fa440ff9640 1 -- 172.21.15.111:0/3140807494 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa438039040 con 0x7fa444071690 2024-09-13T17:57:47.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.419+0000 7fa440ff9640 1 --2- 172.21.15.111:0/3140807494 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa43006ac00 0x7fa43006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:47.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.419+0000 7fa440ff9640 1 -- 172.21.15.111:0/3140807494 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa4380aa0c0 con 0x7fa444071690 2024-09-13T17:57:47.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.419+0000 7fa443fff640 1 --2- 172.21.15.111:0/3140807494 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa43006ac00 0x7fa43006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:47.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.420+0000 7fa443fff640 1 --2- 172.21.15.111:0/3140807494 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa43006ac00 0x7fa43006d0c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa434000f60 tx=0x7fa434009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:47.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.424+0000 7fa440ff9640 1 -- 172.21.15.111:0/3140807494 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fa438074fb0 con 0x7fa444071690 2024-09-13T17:57:47.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.580+0000 7fa44a2d0640 1 -- 172.21.15.111:0/3140807494 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa44415a420 con 0x7fa43006ac00 2024-09-13T17:57:47.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.588+0000 7fa440ff9640 1 -- 172.21.15.111:0/3140807494 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345285 (secure 0 0 0) 0x7fa44415a420 con 0x7fa43006ac00 2024-09-13T17:57:47.595 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:47.597 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-09-13T17:57:47.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.592+0000 7fa44a2d0640 1 -- 172.21.15.111:0/3140807494 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa43006ac00 msgr2=0x7fa43006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.592+0000 7fa44a2d0640 1 --2- 172.21.15.111:0/3140807494 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa43006ac00 0x7fa43006d0c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa434000f60 tx=0x7fa434009040 comp rx=0 tx=0).stop 2024-09-13T17:57:47.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.592+0000 7fa44a2d0640 1 -- 172.21.15.111:0/3140807494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa444071690 msgr2=0x7fa4441f3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.592+0000 7fa44a2d0640 1 --2- 172.21.15.111:0/3140807494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa444071690 0x7fa4441f3390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa4380098f0 tx=0x7fa4380046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.592+0000 7fa44a2d0640 1 -- 172.21.15.111:0/3140807494 shutdown_connections 2024-09-13T17:57:47.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.592+0000 7fa44a2d0640 1 --2- 172.21.15.111:0/3140807494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4440b3e40 0x7fa44415faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.592+0000 7fa44a2d0640 1 --2- 172.21.15.111:0/3140807494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa444071690 0x7fa4441f3390 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.592+0000 7fa44a2d0640 1 --2- 172.21.15.111:0/3140807494 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa43006ac00 0x7fa43006d0c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.592+0000 7fa44a2d0640 1 --2- 172.21.15.111:0/3140807494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa444069ed0 0x7fa444155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.592+0000 7fa44a2d0640 1 -- 172.21.15.111:0/3140807494 >> 172.21.15.111:0/3140807494 conn(0x7fa4440b49b0 msgr2=0x7fa44406d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:47.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.592+0000 7fa44a2d0640 1 -- 172.21.15.111:0/3140807494 shutdown_connections 2024-09-13T17:57:47.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.592+0000 7fa44a2d0640 1 -- 172.21.15.111:0/3140807494 wait complete. 2024-09-13T17:57:47.605 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-13T17:57:46.270478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3204,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3280896,"data_stored":1987143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2231,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002692"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727037+0000","last_change":"2024-09-13T17:57:25.378960+0000","last_active":"2024-09-13T17:57:29.727037+0000","last_peered":"2024-09-13T17:57:29.727037+0000","last_clean":"2024-09-13T17:57:29.727037+0000","last_became_active":"2024-09-13T17:57:25.378681+0000","last_became_peered":"2024-09-13T17:57:25.378681+0000","last_unstale":"2024-09-13T17:57:29.727037+0000","last_undegraded":"2024-09-13T17:57:29.727037+0000","last_fullsized":"2024-09-13T17:57:29.727037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721168+0000","last_change":"2024-09-13T17:57:25.734706+0000","last_active":"2024-09-13T17:57:29.721168+0000","last_peered":"2024-09-13T17:57:29.721168+0000","last_clean":"2024-09-13T17:57:29.721168+0000","last_became_active":"2024-09-13T17:57:25.734432+0000","last_became_peered":"2024-09-13T17:57:25.734432+0000","last_unstale":"2024-09-13T17:57:29.721168+0000","last_undegraded":"2024-09-13T17:57:29.721168+0000","last_fullsized":"2024-09-13T17:57:29.721168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:12.960061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.730686+0000","last_change":"2024-09-13T17:57:29.729380+0000","last_active":"2024-09-13T17:57:29.730686+0000","last_peered":"2024-09-13T17:57:29.730686+0000","last_clean":"2024-09-13T17:57:29.730686+0000","last_became_active":"2024-09-13T17:57:29.729000+0000","last_became_peered":"2024-09-13T17:57:29.729000+0000","last_unstale":"2024-09-13T17:57:29.730686+0000","last_undegraded":"2024-09-13T17:57:29.730686+0000","last_fullsized":"2024-09-13T17:57:29.730686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:31.285328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721336+0000","last_change":"2024-09-13T17:57:25.382416+0000","last_active":"2024-09-13T17:57:29.721336+0000","last_peered":"2024-09-13T17:57:29.721336+0000","last_clean":"2024-09-13T17:57:29.721336+0000","last_became_active":"2024-09-13T17:57:25.382154+0000","last_became_peered":"2024-09-13T17:57:25.382154+0000","last_unstale":"2024-09-13T17:57:29.721336+0000","last_undegraded":"2024-09-13T17:57:29.721336+0000","last_fullsized":"2024-09-13T17:57:29.721336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725491+0000","last_change":"2024-09-13T17:57:25.731759+0000","last_active":"2024-09-13T17:57:29.725491+0000","last_peered":"2024-09-13T17:57:29.725491+0000","last_clean":"2024-09-13T17:57:29.725491+0000","last_became_active":"2024-09-13T17:57:25.731494+0000","last_became_peered":"2024-09-13T17:57:25.731494+0000","last_unstale":"2024-09-13T17:57:29.725491+0000","last_undegraded":"2024-09-13T17:57:29.725491+0000","last_fullsized":"2024-09-13T17:57:29.725491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.751270+0000","last_change":"2024-09-13T17:57:29.750278+0000","last_active":"2024-09-13T17:57:29.751270+0000","last_peered":"2024-09-13T17:57:29.751270+0000","last_clean":"2024-09-13T17:57:29.751270+0000","last_became_active":"2024-09-13T17:57:29.749874+0000","last_became_peered":"2024-09-13T17:57:29.749874+0000","last_unstale":"2024-09-13T17:57:29.751270+0000","last_undegraded":"2024-09-13T17:57:29.751270+0000","last_fullsized":"2024-09-13T17:57:29.751270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:03.749305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725223+0000","last_change":"2024-09-13T17:57:25.377660+0000","last_active":"2024-09-13T17:57:29.725223+0000","last_peered":"2024-09-13T17:57:29.725223+0000","last_clean":"2024-09-13T17:57:29.725223+0000","last_became_active":"2024-09-13T17:57:25.377386+0000","last_became_peered":"2024-09-13T17:57:25.377386+0000","last_unstale":"2024-09-13T17:57:29.725223+0000","last_undegraded":"2024-09-13T17:57:29.725223+0000","last_fullsized":"2024-09-13T17:57:29.725223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:38.038738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725142+0000","last_change":"2024-09-13T17:57:25.728949+0000","last_active":"2024-09-13T17:57:29.725142+0000","last_peered":"2024-09-13T17:57:29.725142+0000","last_clean":"2024-09-13T17:57:29.725142+0000","last_became_active":"2024-09-13T17:57:25.728675+0000","last_became_peered":"2024-09-13T17:57:25.728675+0000","last_unstale":"2024-09-13T17:57:29.725142+0000","last_undegraded":"2024-09-13T17:57:29.725142+0000","last_fullsized":"2024-09-13T17:57:29.725142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749006+0000","last_change":"2024-09-13T17:57:29.745555+0000","last_active":"2024-09-13T17:57:29.749006+0000","last_peered":"2024-09-13T17:57:29.749006+0000","last_clean":"2024-09-13T17:57:29.749006+0000","last_became_active":"2024-09-13T17:57:29.745172+0000","last_became_peered":"2024-09-13T17:57:29.745172+0000","last_unstale":"2024-09-13T17:57:29.749006+0000","last_undegraded":"2024-09-13T17:57:29.749006+0000","last_fullsized":"2024-09-13T17:57:29.749006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:37.101711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725546+0000","last_change":"2024-09-13T17:57:25.375351+0000","last_active":"2024-09-13T17:57:29.725546+0000","last_peered":"2024-09-13T17:57:29.725546+0000","last_clean":"2024-09-13T17:57:29.725546+0000","last_became_active":"2024-09-13T17:57:25.374550+0000","last_became_peered":"2024-09-13T17:57:25.374550+0000","last_unstale":"2024-09-13T17:57:29.725546+0000","last_undegraded":"2024-09-13T17:57:29.725546+0000","last_fullsized":"2024-09-13T17:57:29.725546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:11:57.892225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725579+0000","last_change":"2024-09-13T17:57:25.729527+0000","last_active":"2024-09-13T17:57:29.725579+0000","last_peered":"2024-09-13T17:57:29.725579+0000","last_clean":"2024-09-13T17:57:29.725579+0000","last_became_active":"2024-09-13T17:57:25.729225+0000","last_became_peered":"2024-09-13T17:57:25.729225+0000","last_unstale":"2024-09-13T17:57:29.725579+0000","last_undegraded":"2024-09-13T17:57:29.725579+0000","last_fullsized":"2024-09-13T17:57:29.725579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:27.969805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.748980+0000","last_change":"2024-09-13T17:57:29.745532+0000","last_active":"2024-09-13T17:57:29.748980+0000","last_peered":"2024-09-13T17:57:29.748980+0000","last_clean":"2024-09-13T17:57:29.748980+0000","last_became_active":"2024-09-13T17:57:29.745130+0000","last_became_peered":"2024-09-13T17:57:29.745130+0000","last_unstale":"2024-09-13T17:57:29.748980+0000","last_undegraded":"2024-09-13T17:57:29.748980+0000","last_fullsized":"2024-09-13T17:57:29.748980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:54.172556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728175+0000","last_change":"2024-09-13T17:57:25.735067+0000","last_active":"2024-09-13T17:57:29.728175+0000","last_peered":"2024-09-13T17:57:29.728175+0000","last_clean":"2024-09-13T17:57:29.728175+0000","last_became_active":"2024-09-13T17:57:25.734807+0000","last_became_peered":"2024-09-13T17:57:25.734807+0000","last_unstale":"2024-09-13T17:57:29.728175+0000","last_undegraded":"2024-09-13T17:57:29.728175+0000","last_fullsized":"2024-09-13T17:57:29.728175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:19.536367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725323+0000","last_change":"2024-09-13T17:57:25.374934+0000","last_active":"2024-09-13T17:57:29.725323+0000","last_peered":"2024-09-13T17:57:29.725323+0000","last_clean":"2024-09-13T17:57:29.725323+0000","last_became_active":"2024-09-13T17:57:25.374307+0000","last_became_peered":"2024-09-13T17:57:25.374307+0000","last_unstale":"2024-09-13T17:57:29.725323+0000","last_undegraded":"2024-09-13T17:57:29.725323+0000","last_fullsized":"2024-09-13T17:57:29.725323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:15.663292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.748917+0000","last_change":"2024-09-13T17:57:29.743757+0000","last_active":"2024-09-13T17:57:29.748917+0000","last_peered":"2024-09-13T17:57:29.748917+0000","last_clean":"2024-09-13T17:57:29.748917+0000","last_became_active":"2024-09-13T17:57:29.743362+0000","last_became_peered":"2024-09-13T17:57:29.743362+0000","last_unstale":"2024-09-13T17:57:29.748917+0000","last_undegraded":"2024-09-13T17:57:29.748917+0000","last_fullsized":"2024-09-13T17:57:29.748917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:31.285328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720769+0000","last_change":"2024-09-13T17:57:25.720127+0000","last_active":"2024-09-13T17:57:29.720769+0000","last_peered":"2024-09-13T17:57:29.720769+0000","last_clean":"2024-09-13T17:57:29.720769+0000","last_became_active":"2024-09-13T17:57:25.719766+0000","last_became_peered":"2024-09-13T17:57:25.719766+0000","last_unstale":"2024-09-13T17:57:29.720769+0000","last_undegraded":"2024-09-13T17:57:29.720769+0000","last_fullsized":"2024-09-13T17:57:29.720769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.730752+0000","last_change":"2024-09-13T17:57:25.381085+0000","last_active":"2024-09-13T17:57:29.730752+0000","last_peered":"2024-09-13T17:57:29.730752+0000","last_clean":"2024-09-13T17:57:29.730752+0000","last_became_active":"2024-09-13T17:57:25.380825+0000","last_became_peered":"2024-09-13T17:57:25.380825+0000","last_unstale":"2024-09-13T17:57:29.730752+0000","last_undegraded":"2024-09-13T17:57:29.730752+0000","last_fullsized":"2024-09-13T17:57:29.730752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:15.663292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749031+0000","last_change":"2024-09-13T17:57:29.746426+0000","last_active":"2024-09-13T17:57:29.749031+0000","last_peered":"2024-09-13T17:57:29.749031+0000","last_clean":"2024-09-13T17:57:29.749031+0000","last_became_active":"2024-09-13T17:57:29.745582+0000","last_became_peered":"2024-09-13T17:57:29.745582+0000","last_unstale":"2024-09-13T17:57:29.749031+0000","last_undegraded":"2024-09-13T17:57:29.749031+0000","last_fullsized":"2024-09-13T17:57:29.749031+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:53.991862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721690+0000","last_change":"2024-09-13T17:57:25.724108+0000","last_active":"2024-09-13T17:57:29.721690+0000","last_peered":"2024-09-13T17:57:29.721690+0000","last_clean":"2024-09-13T17:57:29.721690+0000","last_became_active":"2024-09-13T17:57:25.723857+0000","last_became_peered":"2024-09-13T17:57:25.723857+0000","last_unstale":"2024-09-13T17:57:29.721690+0000","last_undegraded":"2024-09-13T17:57:29.721690+0000","last_fullsized":"2024-09-13T17:57:29.721690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:19.536367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729185+0000","last_change":"2024-09-13T17:57:25.374954+0000","last_active":"2024-09-13T17:57:29.729185+0000","last_peered":"2024-09-13T17:57:29.729185+0000","last_clean":"2024-09-13T17:57:29.729185+0000","last_became_active":"2024-09-13T17:57:25.374721+0000","last_became_peered":"2024-09-13T17:57:25.374721+0000","last_unstale":"2024-09-13T17:57:29.729185+0000","last_undegraded":"2024-09-13T17:57:29.729185+0000","last_fullsized":"2024-09-13T17:57:29.729185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:34.109316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749541+0000","last_change":"2024-09-13T17:57:29.744520+0000","last_active":"2024-09-13T17:57:29.749541+0000","last_peered":"2024-09-13T17:57:29.749541+0000","last_clean":"2024-09-13T17:57:29.749541+0000","last_became_active":"2024-09-13T17:57:29.744143+0000","last_became_peered":"2024-09-13T17:57:29.744143+0000","last_unstale":"2024-09-13T17:57:29.749541+0000","last_undegraded":"2024-09-13T17:57:29.749541+0000","last_fullsized":"2024-09-13T17:57:29.749541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:18.526568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.733595+0000","last_change":"2024-09-13T17:57:25.718903+0000","last_active":"2024-09-13T17:57:29.733595+0000","last_peered":"2024-09-13T17:57:29.733595+0000","last_clean":"2024-09-13T17:57:29.733595+0000","last_became_active":"2024-09-13T17:57:25.718483+0000","last_became_peered":"2024-09-13T17:57:25.718483+0000","last_unstale":"2024-09-13T17:57:29.733595+0000","last_undegraded":"2024-09-13T17:57:29.733595+0000","last_fullsized":"2024-09-13T17:57:29.733595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:59.252453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725956+0000","last_change":"2024-09-13T17:57:25.378628+0000","last_active":"2024-09-13T17:57:29.725956+0000","last_peered":"2024-09-13T17:57:29.725956+0000","last_clean":"2024-09-13T17:57:29.725956+0000","last_became_active":"2024-09-13T17:57:25.378359+0000","last_became_peered":"2024-09-13T17:57:25.378359+0000","last_unstale":"2024-09-13T17:57:29.725956+0000","last_undegraded":"2024-09-13T17:57:29.725956+0000","last_fullsized":"2024-09-13T17:57:29.725956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:00.725292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749048+0000","last_change":"2024-09-13T17:57:29.746604+0000","last_active":"2024-09-13T17:57:29.749048+0000","last_peered":"2024-09-13T17:57:29.749048+0000","last_clean":"2024-09-13T17:57:29.749048+0000","last_became_active":"2024-09-13T17:57:29.746216+0000","last_became_peered":"2024-09-13T17:57:29.746216+0000","last_unstale":"2024-09-13T17:57:29.749048+0000","last_undegraded":"2024-09-13T17:57:29.749048+0000","last_fullsized":"2024-09-13T17:57:29.749048+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:18.526568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725404+0000","last_change":"2024-09-13T17:57:25.722540+0000","last_active":"2024-09-13T17:57:29.725404+0000","last_peered":"2024-09-13T17:57:29.725404+0000","last_clean":"2024-09-13T17:57:29.725404+0000","last_became_active":"2024-09-13T17:57:25.722301+0000","last_became_peered":"2024-09-13T17:57:25.722301+0000","last_unstale":"2024-09-13T17:57:29.725404+0000","last_undegraded":"2024-09-13T17:57:29.725404+0000","last_fullsized":"2024-09-13T17:57:29.725404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:34.117674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728850+0000","last_change":"2024-09-13T17:57:25.380087+0000","last_active":"2024-09-13T17:57:29.728850+0000","last_peered":"2024-09-13T17:57:29.728850+0000","last_clean":"2024-09-13T17:57:29.728850+0000","last_became_active":"2024-09-13T17:57:25.379808+0000","last_became_peered":"2024-09-13T17:57:25.379808+0000","last_unstale":"2024-09-13T17:57:29.728850+0000","last_undegraded":"2024-09-13T17:57:29.728850+0000","last_fullsized":"2024-09-13T17:57:29.728850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:38.038738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.743674+0000","last_change":"2024-09-13T17:57:29.740242+0000","last_active":"2024-09-13T17:57:29.743674+0000","last_peered":"2024-09-13T17:57:29.743674+0000","last_clean":"2024-09-13T17:57:29.743674+0000","last_became_active":"2024-09-13T17:57:29.739815+0000","last_became_peered":"2024-09-13T17:57:29.739815+0000","last_unstale":"2024-09-13T17:57:29.743674+0000","last_undegraded":"2024-09-13T17:57:29.743674+0000","last_fullsized":"2024-09-13T17:57:29.743674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:45.306949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720934+0000","last_change":"2024-09-13T17:57:25.718065+0000","last_active":"2024-09-13T17:57:29.720934+0000","last_peered":"2024-09-13T17:57:29.720934+0000","last_clean":"2024-09-13T17:57:29.720934+0000","last_became_active":"2024-09-13T17:57:25.717759+0000","last_became_peered":"2024-09-13T17:57:25.717759+0000","last_unstale":"2024-09-13T17:57:29.720934+0000","last_undegraded":"2024-09-13T17:57:29.720934+0000","last_fullsized":"2024-09-13T17:57:29.720934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.814140+0000","last_change":"2024-09-13T17:57:25.377272+0000","last_active":"2024-09-13T17:57:29.814140+0000","last_peered":"2024-09-13T17:57:29.814140+0000","last_clean":"2024-09-13T17:57:29.814140+0000","last_became_active":"2024-09-13T17:57:25.376938+0000","last_became_peered":"2024-09-13T17:57:25.376938+0000","last_unstale":"2024-09-13T17:57:29.814140+0000","last_undegraded":"2024-09-13T17:57:29.814140+0000","last_fullsized":"2024-09-13T17:57:29.814140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:00.725292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.746741+0000","last_change":"2024-09-13T17:57:29.743363+0000","last_active":"2024-09-13T17:57:29.746741+0000","last_peered":"2024-09-13T17:57:29.746741+0000","last_clean":"2024-09-13T17:57:29.746741+0000","last_became_active":"2024-09-13T17:57:29.738263+0000","last_became_peered":"2024-09-13T17:57:29.738263+0000","last_unstale":"2024-09-13T17:57:29.746741+0000","last_undegraded":"2024-09-13T17:57:29.746741+0000","last_fullsized":"2024-09-13T17:57:29.746741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:42:39.016490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729043+0000","last_change":"2024-09-13T17:57:25.717635+0000","last_active":"2024-09-13T17:57:29.729043+0000","last_peered":"2024-09-13T17:57:29.729043+0000","last_clean":"2024-09-13T17:57:29.729043+0000","last_became_active":"2024-09-13T17:57:25.717360+0000","last_became_peered":"2024-09-13T17:57:25.717360+0000","last_unstale":"2024-09-13T17:57:29.729043+0000","last_undegraded":"2024-09-13T17:57:29.729043+0000","last_fullsized":"2024-09-13T17:57:29.729043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725395+0000","last_change":"2024-09-13T17:57:25.378289+0000","last_active":"2024-09-13T17:57:29.725395+0000","last_peered":"2024-09-13T17:57:29.725395+0000","last_clean":"2024-09-13T17:57:29.725395+0000","last_became_active":"2024-09-13T17:57:25.378038+0000","last_became_peered":"2024-09-13T17:57:25.378038+0000","last_unstale":"2024-09-13T17:57:29.725395+0000","last_undegraded":"2024-09-13T17:57:29.725395+0000","last_fullsized":"2024-09-13T17:57:29.725395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:58.248434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.746870+0000","last_change":"2024-09-13T17:57:29.741347+0000","last_active":"2024-09-13T17:57:29.746870+0000","last_peered":"2024-09-13T17:57:29.746870+0000","last_clean":"2024-09-13T17:57:29.746870+0000","last_became_active":"2024-09-13T17:57:29.740980+0000","last_became_peered":"2024-09-13T17:57:29.740980+0000","last_unstale":"2024-09-13T17:57:29.746870+0000","last_undegraded":"2024-09-13T17:57:29.746870+0000","last_fullsized":"2024-09-13T17:57:29.746870+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:53.991862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725663+0000","last_change":"2024-09-13T17:57:25.728135+0000","last_active":"2024-09-13T17:57:29.725663+0000","last_peered":"2024-09-13T17:57:29.725663+0000","last_clean":"2024-09-13T17:57:29.725663+0000","last_became_active":"2024-09-13T17:57:25.727903+0000","last_became_peered":"2024-09-13T17:57:25.727903+0000","last_unstale":"2024-09-13T17:57:29.725663+0000","last_undegraded":"2024-09-13T17:57:29.725663+0000","last_fullsized":"2024-09-13T17:57:29.725663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:48.163930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729130+0000","last_change":"2024-09-13T17:57:25.375187+0000","last_active":"2024-09-13T17:57:29.729130+0000","last_peered":"2024-09-13T17:57:29.729130+0000","last_clean":"2024-09-13T17:57:29.729130+0000","last_became_active":"2024-09-13T17:57:25.374295+0000","last_became_peered":"2024-09-13T17:57:25.374295+0000","last_unstale":"2024-09-13T17:57:29.729130+0000","last_undegraded":"2024-09-13T17:57:29.729130+0000","last_fullsized":"2024-09-13T17:57:29.729130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:58.248434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742560+0000","last_change":"2024-09-13T17:57:29.740383+0000","last_active":"2024-09-13T17:57:29.742560+0000","last_peered":"2024-09-13T17:57:29.742560+0000","last_clean":"2024-09-13T17:57:29.742560+0000","last_became_active":"2024-09-13T17:57:29.739542+0000","last_became_peered":"2024-09-13T17:57:29.739542+0000","last_unstale":"2024-09-13T17:57:29.742560+0000","last_undegraded":"2024-09-13T17:57:29.742560+0000","last_fullsized":"2024-09-13T17:57:29.742560+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:13.633882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.734117+0000","last_change":"2024-09-13T17:57:25.717973+0000","last_active":"2024-09-13T17:57:29.734117+0000","last_peered":"2024-09-13T17:57:29.734117+0000","last_clean":"2024-09-13T17:57:29.734117+0000","last_became_active":"2024-09-13T17:57:25.717702+0000","last_became_peered":"2024-09-13T17:57:25.717702+0000","last_unstale":"2024-09-13T17:57:29.734117+0000","last_undegraded":"2024-09-13T17:57:29.734117+0000","last_fullsized":"2024-09-13T17:57:29.734117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:35.113334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725035+0000","last_change":"2024-09-13T17:57:25.373814+0000","last_active":"2024-09-13T17:57:29.725035+0000","last_peered":"2024-09-13T17:57:29.725035+0000","last_clean":"2024-09-13T17:57:29.725035+0000","last_became_active":"2024-09-13T17:57:25.373556+0000","last_became_peered":"2024-09-13T17:57:25.373556+0000","last_unstale":"2024-09-13T17:57:29.725035+0000","last_undegraded":"2024-09-13T17:57:29.725035+0000","last_fullsized":"2024-09-13T17:57:29.725035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:04.441662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742509+0000","last_change":"2024-09-13T17:57:29.739908+0000","last_active":"2024-09-13T17:57:29.742509+0000","last_peered":"2024-09-13T17:57:29.742509+0000","last_clean":"2024-09-13T17:57:29.742509+0000","last_became_active":"2024-09-13T17:57:29.739070+0000","last_became_peered":"2024-09-13T17:57:29.739070+0000","last_unstale":"2024-09-13T17:57:29.742509+0000","last_undegraded":"2024-09-13T17:57:29.742509+0000","last_fullsized":"2024-09-13T17:57:29.742509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:37.101711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729849+0000","last_change":"2024-09-13T17:57:25.718810+0000","last_active":"2024-09-13T17:57:29.729849+0000","last_peered":"2024-09-13T17:57:29.729849+0000","last_clean":"2024-09-13T17:57:29.729849+0000","last_became_active":"2024-09-13T17:57:25.718576+0000","last_became_peered":"2024-09-13T17:57:25.718576+0000","last_unstale":"2024-09-13T17:57:29.729849+0000","last_undegraded":"2024-09-13T17:57:29.729849+0000","last_fullsized":"2024-09-13T17:57:29.729849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:13.197645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732373+0000","last_change":"2024-09-13T17:57:25.380336+0000","last_active":"2024-09-13T17:57:29.732373+0000","last_peered":"2024-09-13T17:57:29.732373+0000","last_clean":"2024-09-13T17:57:29.732373+0000","last_became_active":"2024-09-13T17:57:25.380044+0000","last_became_peered":"2024-09-13T17:57:25.380044+0000","last_unstale":"2024-09-13T17:57:29.732373+0000","last_undegraded":"2024-09-13T17:57:29.732373+0000","last_fullsized":"2024-09-13T17:57:29.732373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:41.189680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.746550+0000","last_change":"2024-09-13T17:57:29.741637+0000","last_active":"2024-09-13T17:57:29.746550+0000","last_peered":"2024-09-13T17:57:29.746550+0000","last_clean":"2024-09-13T17:57:29.746550+0000","last_became_active":"2024-09-13T17:57:29.741260+0000","last_became_peered":"2024-09-13T17:57:29.741260+0000","last_unstale":"2024-09-13T17:57:29.746550+0000","last_undegraded":"2024-09-13T17:57:29.746550+0000","last_fullsized":"2024-09-13T17:57:29.746550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:53.991862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724599+0000","last_change":"2024-09-13T17:57:25.718276+0000","last_active":"2024-09-13T17:57:29.724599+0000","last_peered":"2024-09-13T17:57:29.724599+0000","last_clean":"2024-09-13T17:57:29.724599+0000","last_became_active":"2024-09-13T17:57:25.718000+0000","last_became_peered":"2024-09-13T17:57:25.718000+0000","last_unstale":"2024-09-13T17:57:29.724599+0000","last_undegraded":"2024-09-13T17:57:29.724599+0000","last_fullsized":"2024-09-13T17:57:29.724599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:05.710729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724849+0000","last_change":"2024-09-13T17:57:25.375143+0000","last_active":"2024-09-13T17:57:29.724849+0000","last_peered":"2024-09-13T17:57:29.724849+0000","last_clean":"2024-09-13T17:57:29.724849+0000","last_became_active":"2024-09-13T17:57:25.374871+0000","last_became_peered":"2024-09-13T17:57:25.374871+0000","last_unstale":"2024-09-13T17:57:29.724849+0000","last_undegraded":"2024-09-13T17:57:29.724849+0000","last_fullsized":"2024-09-13T17:57:29.724849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:34.109316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.738027+0000","last_change":"2024-09-13T17:57:29.733428+0000","last_active":"2024-09-13T17:57:29.738027+0000","last_peered":"2024-09-13T17:57:29.738027+0000","last_clean":"2024-09-13T17:57:29.738027+0000","last_became_active":"2024-09-13T17:57:29.732952+0000","last_became_peered":"2024-09-13T17:57:29.732952+0000","last_unstale":"2024-09-13T17:57:29.738027+0000","last_undegraded":"2024-09-13T17:57:29.738027+0000","last_fullsized":"2024-09-13T17:57:29.738027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:56:53.973670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720518+0000","last_change":"2024-09-13T17:57:25.732656+0000","last_active":"2024-09-13T17:57:29.720518+0000","last_peered":"2024-09-13T17:57:29.720518+0000","last_clean":"2024-09-13T17:57:29.720518+0000","last_became_active":"2024-09-13T17:57:25.732308+0000","last_became_peered":"2024-09-13T17:57:25.732308+0000","last_unstale":"2024-09-13T17:57:29.720518+0000","last_undegraded":"2024-09-13T17:57:29.720518+0000","last_fullsized":"2024-09-13T17:57:29.720518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:34.797142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724840+0000","last_change":"2024-09-13T17:57:25.374372+0000","last_active":"2024-09-13T17:57:29.724840+0000","last_peered":"2024-09-13T17:57:29.724840+0000","last_clean":"2024-09-13T17:57:29.724840+0000","last_became_active":"2024-09-13T17:57:25.374110+0000","last_became_peered":"2024-09-13T17:57:25.374110+0000","last_unstale":"2024-09-13T17:57:29.724840+0000","last_undegraded":"2024-09-13T17:57:29.724840+0000","last_fullsized":"2024-09-13T17:57:29.724840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:38.038738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724941+0000","last_change":"2024-09-13T17:57:15.560506+0000","last_active":"2024-09-13T17:57:29.724941+0000","last_peered":"2024-09-13T17:57:29.724941+0000","last_clean":"2024-09-13T17:57:29.724941+0000","last_became_active":"2024-09-13T17:57:15.559612+0000","last_became_peered":"2024-09-13T17:57:15.559612+0000","last_unstale":"2024-09-13T17:57:29.724941+0000","last_undegraded":"2024-09-13T17:57:29.724941+0000","last_fullsized":"2024-09-13T17:57:29.724941+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_clean_scrub_stamp":"2024-09-13T17:57:14.317378+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:16:27.663312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.743992+0000","last_change":"2024-09-13T17:57:29.739891+0000","last_active":"2024-09-13T17:57:29.743992+0000","last_peered":"2024-09-13T17:57:29.743992+0000","last_clean":"2024-09-13T17:57:29.743992+0000","last_became_active":"2024-09-13T17:57:29.739331+0000","last_became_peered":"2024-09-13T17:57:29.739331+0000","last_unstale":"2024-09-13T17:57:29.743992+0000","last_undegraded":"2024-09-13T17:57:29.743992+0000","last_fullsized":"2024-09-13T17:57:29.743992+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:45.306949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725127+0000","last_change":"2024-09-13T17:57:25.725987+0000","last_active":"2024-09-13T17:57:29.725127+0000","last_peered":"2024-09-13T17:57:29.725127+0000","last_clean":"2024-09-13T17:57:29.725127+0000","last_became_active":"2024-09-13T17:57:25.725695+0000","last_became_peered":"2024-09-13T17:57:25.725695+0000","last_unstale":"2024-09-13T17:57:29.725127+0000","last_undegraded":"2024-09-13T17:57:29.725127+0000","last_fullsized":"2024-09-13T17:57:29.725127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:48.163930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732467+0000","last_change":"2024-09-13T17:57:25.376170+0000","last_active":"2024-09-13T17:57:29.732467+0000","last_peered":"2024-09-13T17:57:29.732467+0000","last_clean":"2024-09-13T17:57:29.732467+0000","last_became_active":"2024-09-13T17:57:25.375798+0000","last_became_peered":"2024-09-13T17:57:25.375798+0000","last_unstale":"2024-09-13T17:57:29.732467+0000","last_undegraded":"2024-09-13T17:57:29.732467+0000","last_fullsized":"2024-09-13T17:57:29.732467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:08.374958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744268+0000","last_change":"2024-09-13T17:57:29.741819+0000","last_active":"2024-09-13T17:57:29.744268+0000","last_peered":"2024-09-13T17:57:29.744268+0000","last_clean":"2024-09-13T17:57:29.744268+0000","last_became_active":"2024-09-13T17:57:29.741410+0000","last_became_peered":"2024-09-13T17:57:29.741410+0000","last_unstale":"2024-09-13T17:57:29.744268+0000","last_undegraded":"2024-09-13T17:57:29.744268+0000","last_fullsized":"2024-09-13T17:57:29.744268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:15.299874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725178+0000","last_change":"2024-09-13T17:57:25.729870+0000","last_active":"2024-09-13T17:57:29.725178+0000","last_peered":"2024-09-13T17:57:29.725178+0000","last_clean":"2024-09-13T17:57:29.725178+0000","last_became_active":"2024-09-13T17:57:25.728383+0000","last_became_peered":"2024-09-13T17:57:25.728383+0000","last_unstale":"2024-09-13T17:57:29.725178+0000","last_undegraded":"2024-09-13T17:57:29.725178+0000","last_fullsized":"2024-09-13T17:57:29.725178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:51.804478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.730112+0000","last_change":"2024-09-13T17:57:25.373843+0000","last_active":"2024-09-13T17:57:29.730112+0000","last_peered":"2024-09-13T17:57:29.730112+0000","last_clean":"2024-09-13T17:57:29.730112+0000","last_became_active":"2024-09-13T17:57:25.373498+0000","last_became_peered":"2024-09-13T17:57:25.373498+0000","last_unstale":"2024-09-13T17:57:29.730112+0000","last_undegraded":"2024-09-13T17:57:29.730112+0000","last_fullsized":"2024-09-13T17:57:29.730112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:11:57.892225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742650+0000","last_change":"2024-09-13T17:57:29.735791+0000","last_active":"2024-09-13T17:57:29.742650+0000","last_peered":"2024-09-13T17:57:29.742650+0000","last_clean":"2024-09-13T17:57:29.742650+0000","last_became_active":"2024-09-13T17:57:29.735444+0000","last_became_peered":"2024-09-13T17:57:29.735444+0000","last_unstale":"2024-09-13T17:57:29.742650+0000","last_undegraded":"2024-09-13T17:57:29.742650+0000","last_fullsized":"2024-09-13T17:57:29.742650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:13.633882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725552+0000","last_change":"2024-09-13T17:57:25.724498+0000","last_active":"2024-09-13T17:57:29.725552+0000","last_peered":"2024-09-13T17:57:29.725552+0000","last_clean":"2024-09-13T17:57:29.725552+0000","last_became_active":"2024-09-13T17:57:25.724159+0000","last_became_peered":"2024-09-13T17:57:25.724159+0000","last_unstale":"2024-09-13T17:57:29.725552+0000","last_undegraded":"2024-09-13T17:57:29.725552+0000","last_fullsized":"2024-09-13T17:57:29.725552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:34.797142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732295+0000","last_change":"2024-09-13T17:57:25.380132+0000","last_active":"2024-09-13T17:57:29.732295+0000","last_peered":"2024-09-13T17:57:29.732295+0000","last_clean":"2024-09-13T17:57:29.732295+0000","last_became_active":"2024-09-13T17:57:25.379862+0000","last_became_peered":"2024-09-13T17:57:25.379862+0000","last_unstale":"2024-09-13T17:57:29.732295+0000","last_undegraded":"2024-09-13T17:57:29.732295+0000","last_fullsized":"2024-09-13T17:57:29.732295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:43:20.708183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.740512+0000","last_change":"2024-09-13T17:57:29.738502+0000","last_active":"2024-09-13T17:57:29.740512+0000","last_peered":"2024-09-13T17:57:29.740512+0000","last_clean":"2024-09-13T17:57:29.740512+0000","last_became_active":"2024-09-13T17:57:29.737664+0000","last_became_peered":"2024-09-13T17:57:29.737664+0000","last_unstale":"2024-09-13T17:57:29.740512+0000","last_undegraded":"2024-09-13T17:57:29.740512+0000","last_fullsized":"2024-09-13T17:57:29.740512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:13.633882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725043+0000","last_change":"2024-09-13T17:57:25.716646+0000","last_active":"2024-09-13T17:57:29.725043+0000","last_peered":"2024-09-13T17:57:29.725043+0000","last_clean":"2024-09-13T17:57:29.725043+0000","last_became_active":"2024-09-13T17:57:25.716005+0000","last_became_peered":"2024-09-13T17:57:25.716005+0000","last_unstale":"2024-09-13T17:57:29.725043+0000","last_undegraded":"2024-09-13T17:57:29.725043+0000","last_fullsized":"2024-09-13T17:57:29.725043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:43.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.730970+0000","last_change":"2024-09-13T17:57:25.374986+0000","last_active":"2024-09-13T17:57:29.730970+0000","last_peered":"2024-09-13T17:57:29.730970+0000","last_clean":"2024-09-13T17:57:29.730970+0000","last_became_active":"2024-09-13T17:57:25.374609+0000","last_became_peered":"2024-09-13T17:57:25.374609+0000","last_unstale":"2024-09-13T17:57:29.730970+0000","last_undegraded":"2024-09-13T17:57:29.730970+0000","last_fullsized":"2024-09-13T17:57:29.730970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:43:20.708183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749009+0000","last_change":"2024-09-13T17:57:29.745707+0000","last_active":"2024-09-13T17:57:29.749009+0000","last_peered":"2024-09-13T17:57:29.749009+0000","last_clean":"2024-09-13T17:57:29.749009+0000","last_became_active":"2024-09-13T17:57:29.745303+0000","last_became_peered":"2024-09-13T17:57:29.745303+0000","last_unstale":"2024-09-13T17:57:29.749009+0000","last_undegraded":"2024-09-13T17:57:29.749009+0000","last_fullsized":"2024-09-13T17:57:29.749009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:06.371464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725823+0000","last_change":"2024-09-13T17:57:25.717385+0000","last_active":"2024-09-13T17:57:29.725823+0000","last_peered":"2024-09-13T17:57:29.725823+0000","last_clean":"2024-09-13T17:57:29.725823+0000","last_became_active":"2024-09-13T17:57:25.717042+0000","last_became_peered":"2024-09-13T17:57:25.717042+0000","last_unstale":"2024-09-13T17:57:29.725823+0000","last_undegraded":"2024-09-13T17:57:29.725823+0000","last_fullsized":"2024-09-13T17:57:29.725823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:49.905668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732190+0000","last_change":"2024-09-13T17:57:25.369802+0000","last_active":"2024-09-13T17:57:29.732190+0000","last_peered":"2024-09-13T17:57:29.732190+0000","last_clean":"2024-09-13T17:57:29.732190+0000","last_became_active":"2024-09-13T17:57:25.369534+0000","last_became_peered":"2024-09-13T17:57:25.369534+0000","last_unstale":"2024-09-13T17:57:29.732190+0000","last_undegraded":"2024-09-13T17:57:29.732190+0000","last_fullsized":"2024-09-13T17:57:29.732190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742623+0000","last_change":"2024-09-13T17:57:29.737826+0000","last_active":"2024-09-13T17:57:29.742623+0000","last_peered":"2024-09-13T17:57:29.742623+0000","last_clean":"2024-09-13T17:57:29.742623+0000","last_became_active":"2024-09-13T17:57:29.737181+0000","last_became_peered":"2024-09-13T17:57:29.737181+0000","last_unstale":"2024-09-13T17:57:29.742623+0000","last_undegraded":"2024-09-13T17:57:29.742623+0000","last_fullsized":"2024-09-13T17:57:29.742623+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:06.225969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728642+0000","last_change":"2024-09-13T17:57:25.734250+0000","last_active":"2024-09-13T17:57:29.728642+0000","last_peered":"2024-09-13T17:57:29.728642+0000","last_clean":"2024-09-13T17:57:29.728642+0000","last_became_active":"2024-09-13T17:57:25.733836+0000","last_became_peered":"2024-09-13T17:57:25.733836+0000","last_unstale":"2024-09-13T17:57:29.728642+0000","last_undegraded":"2024-09-13T17:57:29.728642+0000","last_fullsized":"2024-09-13T17:57:29.728642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:05.710729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728072+0000","last_change":"2024-09-13T17:57:25.374153+0000","last_active":"2024-09-13T17:57:29.728072+0000","last_peered":"2024-09-13T17:57:29.728072+0000","last_clean":"2024-09-13T17:57:29.728072+0000","last_became_active":"2024-09-13T17:57:25.373891+0000","last_became_peered":"2024-09-13T17:57:25.373891+0000","last_unstale":"2024-09-13T17:57:29.728072+0000","last_undegraded":"2024-09-13T17:57:29.728072+0000","last_fullsized":"2024-09-13T17:57:29.728072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744947+0000","last_change":"2024-09-13T17:57:29.741648+0000","last_active":"2024-09-13T17:57:29.744947+0000","last_peered":"2024-09-13T17:57:29.744947+0000","last_clean":"2024-09-13T17:57:29.744947+0000","last_became_active":"2024-09-13T17:57:29.741315+0000","last_became_peered":"2024-09-13T17:57:29.741315+0000","last_unstale":"2024-09-13T17:57:29.744947+0000","last_undegraded":"2024-09-13T17:57:29.744947+0000","last_fullsized":"2024-09-13T17:57:29.744947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:18.432911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725228+0000","last_change":"2024-09-13T17:57:25.716312+0000","last_active":"2024-09-13T17:57:29.725228+0000","last_peered":"2024-09-13T17:57:29.725228+0000","last_clean":"2024-09-13T17:57:29.725228+0000","last_became_active":"2024-09-13T17:57:25.716067+0000","last_became_peered":"2024-09-13T17:57:25.716067+0000","last_unstale":"2024-09-13T17:57:29.725228+0000","last_undegraded":"2024-09-13T17:57:29.725228+0000","last_fullsized":"2024-09-13T17:57:29.725228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:33.971487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724948+0000","last_change":"2024-09-13T17:57:25.380277+0000","last_active":"2024-09-13T17:57:29.724948+0000","last_peered":"2024-09-13T17:57:29.724948+0000","last_clean":"2024-09-13T17:57:29.724948+0000","last_became_active":"2024-09-13T17:57:25.379988+0000","last_became_peered":"2024-09-13T17:57:25.379988+0000","last_unstale":"2024-09-13T17:57:29.724948+0000","last_undegraded":"2024-09-13T17:57:29.724948+0000","last_fullsized":"2024-09-13T17:57:29.724948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744773+0000","last_change":"2024-09-13T17:57:29.733138+0000","last_active":"2024-09-13T17:57:29.744773+0000","last_peered":"2024-09-13T17:57:29.744773+0000","last_clean":"2024-09-13T17:57:29.744773+0000","last_became_active":"2024-09-13T17:57:29.732804+0000","last_became_peered":"2024-09-13T17:57:29.732804+0000","last_unstale":"2024-09-13T17:57:29.744773+0000","last_undegraded":"2024-09-13T17:57:29.744773+0000","last_fullsized":"2024-09-13T17:57:29.744773+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:08.695591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724921+0000","last_change":"2024-09-13T17:57:25.714463+0000","last_active":"2024-09-13T17:57:29.724921+0000","last_peered":"2024-09-13T17:57:29.724921+0000","last_clean":"2024-09-13T17:57:29.724921+0000","last_became_active":"2024-09-13T17:57:25.714209+0000","last_became_peered":"2024-09-13T17:57:25.714209+0000","last_unstale":"2024-09-13T17:57:29.724921+0000","last_undegraded":"2024-09-13T17:57:29.724921+0000","last_fullsized":"2024-09-13T17:57:29.724921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727847+0000","last_change":"2024-09-13T17:57:25.374886+0000","last_active":"2024-09-13T17:57:29.727847+0000","last_peered":"2024-09-13T17:57:29.727847+0000","last_clean":"2024-09-13T17:57:29.727847+0000","last_became_active":"2024-09-13T17:57:25.374025+0000","last_became_peered":"2024-09-13T17:57:25.374025+0000","last_unstale":"2024-09-13T17:57:29.727847+0000","last_undegraded":"2024-09-13T17:57:29.727847+0000","last_fullsized":"2024-09-13T17:57:29.727847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:41.189680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.739379+0000","last_change":"2024-09-13T17:57:29.735654+0000","last_active":"2024-09-13T17:57:29.739379+0000","last_peered":"2024-09-13T17:57:29.739379+0000","last_clean":"2024-09-13T17:57:29.739379+0000","last_became_active":"2024-09-13T17:57:29.735267+0000","last_became_peered":"2024-09-13T17:57:29.735267+0000","last_unstale":"2024-09-13T17:57:29.739379+0000","last_undegraded":"2024-09-13T17:57:29.739379+0000","last_fullsized":"2024-09-13T17:57:29.739379+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:59:14.990893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725041+0000","last_change":"2024-09-13T17:57:25.722101+0000","last_active":"2024-09-13T17:57:29.725041+0000","last_peered":"2024-09-13T17:57:29.725041+0000","last_clean":"2024-09-13T17:57:29.725041+0000","last_became_active":"2024-09-13T17:57:25.721845+0000","last_became_peered":"2024-09-13T17:57:25.721845+0000","last_unstale":"2024-09-13T17:57:29.725041+0000","last_undegraded":"2024-09-13T17:57:29.725041+0000","last_fullsized":"2024-09-13T17:57:29.725041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:11.996031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:57:34.418487+0000","last_change":"2024-09-13T17:57:25.378020+0000","last_active":"2024-09-13T17:57:34.418487+0000","last_peered":"2024-09-13T17:57:34.418487+0000","last_clean":"2024-09-13T17:57:34.418487+0000","last_became_active":"2024-09-13T17:57:25.377746+0000","last_became_peered":"2024-09-13T17:57:25.377746+0000","last_unstale":"2024-09-13T17:57:34.418487+0000","last_undegraded":"2024-09-13T17:57:34.418487+0000","last_fullsized":"2024-09-13T17:57:34.418487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:58.248434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742293+0000","last_change":"2024-09-13T17:57:29.738264+0000","last_active":"2024-09-13T17:57:29.742293+0000","last_peered":"2024-09-13T17:57:29.742293+0000","last_clean":"2024-09-13T17:57:29.742293+0000","last_became_active":"2024-09-13T17:57:29.737897+0000","last_became_peered":"2024-09-13T17:57:29.737897+0000","last_unstale":"2024-09-13T17:57:29.742293+0000","last_undegraded":"2024-09-13T17:57:29.742293+0000","last_fullsized":"2024-09-13T17:57:29.742293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:59:14.990893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.735362+0000","last_change":"2024-09-13T17:57:25.720986+0000","last_active":"2024-09-13T17:57:29.735362+0000","last_peered":"2024-09-13T17:57:29.735362+0000","last_clean":"2024-09-13T17:57:29.735362+0000","last_became_active":"2024-09-13T17:57:25.720696+0000","last_became_peered":"2024-09-13T17:57:25.720696+0000","last_unstale":"2024-09-13T17:57:29.735362+0000","last_undegraded":"2024-09-13T17:57:29.735362+0000","last_fullsized":"2024-09-13T17:57:29.735362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:01.729310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729997+0000","last_change":"2024-09-13T17:57:25.376707+0000","last_active":"2024-09-13T17:57:29.729997+0000","last_peered":"2024-09-13T17:57:29.729997+0000","last_clean":"2024-09-13T17:57:29.729997+0000","last_became_active":"2024-09-13T17:57:25.376080+0000","last_became_peered":"2024-09-13T17:57:25.376080+0000","last_unstale":"2024-09-13T17:57:29.729997+0000","last_undegraded":"2024-09-13T17:57:29.729997+0000","last_fullsized":"2024-09-13T17:57:29.729997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:28.985664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744127+0000","last_change":"2024-09-13T17:57:29.741053+0000","last_active":"2024-09-13T17:57:29.744127+0000","last_peered":"2024-09-13T17:57:29.744127+0000","last_clean":"2024-09-13T17:57:29.744127+0000","last_became_active":"2024-09-13T17:57:29.740751+0000","last_became_peered":"2024-09-13T17:57:29.740751+0000","last_unstale":"2024-09-13T17:57:29.744127+0000","last_undegraded":"2024-09-13T17:57:29.744127+0000","last_fullsized":"2024-09-13T17:57:29.744127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:56:53.973670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728126+0000","last_change":"2024-09-13T17:57:25.725767+0000","last_active":"2024-09-13T17:57:29.728126+0000","last_peered":"2024-09-13T17:57:29.728126+0000","last_clean":"2024-09-13T17:57:29.728126+0000","last_became_active":"2024-09-13T17:57:25.725565+0000","last_became_peered":"2024-09-13T17:57:25.725565+0000","last_unstale":"2024-09-13T17:57:29.728126+0000","last_undegraded":"2024-09-13T17:57:29.728126+0000","last_fullsized":"2024-09-13T17:57:29.728126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:19.536367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725379+0000","last_change":"2024-09-13T17:57:25.380232+0000","last_active":"2024-09-13T17:57:29.725379+0000","last_peered":"2024-09-13T17:57:29.725379+0000","last_clean":"2024-09-13T17:57:29.725379+0000","last_became_active":"2024-09-13T17:57:25.379910+0000","last_became_peered":"2024-09-13T17:57:25.379910+0000","last_unstale":"2024-09-13T17:57:29.725379+0000","last_undegraded":"2024-09-13T17:57:29.725379+0000","last_fullsized":"2024-09-13T17:57:29.725379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:28.958967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749501+0000","last_change":"2024-09-13T17:57:29.745584+0000","last_active":"2024-09-13T17:57:29.749501+0000","last_peered":"2024-09-13T17:57:29.749501+0000","last_clean":"2024-09-13T17:57:29.749501+0000","last_became_active":"2024-09-13T17:57:29.745250+0000","last_became_peered":"2024-09-13T17:57:29.745250+0000","last_unstale":"2024-09-13T17:57:29.749501+0000","last_undegraded":"2024-09-13T17:57:29.749501+0000","last_fullsized":"2024-09-13T17:57:29.749501+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:12.564762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725293+0000","last_change":"2024-09-13T17:57:25.724239+0000","last_active":"2024-09-13T17:57:29.725293+0000","last_peered":"2024-09-13T17:57:29.725293+0000","last_clean":"2024-09-13T17:57:29.725293+0000","last_became_active":"2024-09-13T17:57:25.723999+0000","last_became_peered":"2024-09-13T17:57:25.723999+0000","last_unstale":"2024-09-13T17:57:29.725293+0000","last_undegraded":"2024-09-13T17:57:29.725293+0000","last_fullsized":"2024-09-13T17:57:29.725293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728742+0000","last_change":"2024-09-13T17:57:25.376498+0000","last_active":"2024-09-13T17:57:29.728742+0000","last_peered":"2024-09-13T17:57:29.728742+0000","last_clean":"2024-09-13T17:57:29.728742+0000","last_became_active":"2024-09-13T17:57:25.376171+0000","last_became_peered":"2024-09-13T17:57:25.376171+0000","last_unstale":"2024-09-13T17:57:29.728742+0000","last_undegraded":"2024-09-13T17:57:29.728742+0000","last_fullsized":"2024-09-13T17:57:29.728742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:00.725292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744624+0000","last_change":"2024-09-13T17:57:29.737219+0000","last_active":"2024-09-13T17:57:29.744624+0000","last_peered":"2024-09-13T17:57:29.744624+0000","last_clean":"2024-09-13T17:57:29.744624+0000","last_became_active":"2024-09-13T17:57:29.736853+0000","last_became_peered":"2024-09-13T17:57:29.736853+0000","last_unstale":"2024-09-13T17:57:29.744624+0000","last_undegraded":"2024-09-13T17:57:29.744624+0000","last_fullsized":"2024-09-13T17:57:29.744624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:18.526568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.738047+0000","last_change":"2024-09-13T17:57:29.733906+0000","last_active":"2024-09-13T17:57:29.738047+0000","last_peered":"2024-09-13T17:57:29.738047+0000","last_clean":"2024-09-13T17:57:29.738047+0000","last_became_active":"2024-09-13T17:57:29.733543+0000","last_became_peered":"2024-09-13T17:57:29.733543+0000","last_unstale":"2024-09-13T17:57:29.738047+0000","last_undegraded":"2024-09-13T17:57:29.738047+0000","last_fullsized":"2024-09-13T17:57:29.738047+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:37.101711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729014+0000","last_change":"2024-09-13T17:57:25.721000+0000","last_active":"2024-09-13T17:57:29.729014+0000","last_peered":"2024-09-13T17:57:29.729014+0000","last_clean":"2024-09-13T17:57:29.729014+0000","last_became_active":"2024-09-13T17:57:25.720696+0000","last_became_peered":"2024-09-13T17:57:25.720696+0000","last_unstale":"2024-09-13T17:57:29.729014+0000","last_undegraded":"2024-09-13T17:57:29.729014+0000","last_fullsized":"2024-09-13T17:57:29.729014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:34.797142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724465+0000","last_change":"2024-09-13T17:57:25.374530+0000","last_active":"2024-09-13T17:57:29.724465+0000","last_peered":"2024-09-13T17:57:29.724465+0000","last_clean":"2024-09-13T17:57:29.724465+0000","last_became_active":"2024-09-13T17:57:25.374296+0000","last_became_peered":"2024-09-13T17:57:25.374296+0000","last_unstale":"2024-09-13T17:57:29.724465+0000","last_undegraded":"2024-09-13T17:57:29.724465+0000","last_fullsized":"2024-09-13T17:57:29.724465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:15.663292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744054+0000","last_change":"2024-09-13T17:57:29.740208+0000","last_active":"2024-09-13T17:57:29.744054+0000","last_peered":"2024-09-13T17:57:29.744054+0000","last_clean":"2024-09-13T17:57:29.744054+0000","last_became_active":"2024-09-13T17:57:29.739839+0000","last_became_peered":"2024-09-13T17:57:29.739839+0000","last_unstale":"2024-09-13T17:57:29.744054+0000","last_undegraded":"2024-09-13T17:57:29.744054+0000","last_fullsized":"2024-09-13T17:57:29.744054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:32.682495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721617+0000","last_change":"2024-09-13T17:57:25.720286+0000","last_active":"2024-09-13T17:57:29.721617+0000","last_peered":"2024-09-13T17:57:29.721617+0000","last_clean":"2024-09-13T17:57:29.721617+0000","last_became_active":"2024-09-13T17:57:25.719918+0000","last_became_peered":"2024-09-13T17:57:25.719918+0000","last_unstale":"2024-09-13T17:57:29.721617+0000","last_undegraded":"2024-09-13T17:57:29.721617+0000","last_fullsized":"2024-09-13T17:57:29.721617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:13.197645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721496+0000","last_change":"2024-09-13T17:57:25.380282+0000","last_active":"2024-09-13T17:57:29.721496+0000","last_peered":"2024-09-13T17:57:29.721496+0000","last_clean":"2024-09-13T17:57:29.721496+0000","last_became_active":"2024-09-13T17:57:25.379992+0000","last_became_peered":"2024-09-13T17:57:25.379992+0000","last_unstale":"2024-09-13T17:57:29.721496+0000","last_undegraded":"2024-09-13T17:57:29.721496+0000","last_fullsized":"2024-09-13T17:57:29.721496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:04.441662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728265+0000","last_change":"2024-09-13T17:57:25.373463+0000","last_active":"2024-09-13T17:57:29.728265+0000","last_peered":"2024-09-13T17:57:29.728265+0000","last_clean":"2024-09-13T17:57:29.728265+0000","last_became_active":"2024-09-13T17:57:25.373127+0000","last_became_peered":"2024-09-13T17:57:25.373127+0000","last_unstale":"2024-09-13T17:57:29.728265+0000","last_undegraded":"2024-09-13T17:57:29.728265+0000","last_fullsized":"2024-09-13T17:57:29.728265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:15.663292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727753+0000","last_change":"2024-09-13T17:57:25.719108+0000","last_active":"2024-09-13T17:57:29.727753+0000","last_peered":"2024-09-13T17:57:29.727753+0000","last_clean":"2024-09-13T17:57:29.727753+0000","last_became_active":"2024-09-13T17:57:25.718674+0000","last_became_peered":"2024-09-13T17:57:25.718674+0000","last_unstale":"2024-09-13T17:57:29.727753+0000","last_undegraded":"2024-09-13T17:57:29.727753+0000","last_fullsized":"2024-09-13T17:57:29.727753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:49.905668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.740987+0000","last_change":"2024-09-13T17:57:29.736695+0000","last_active":"2024-09-13T17:57:29.740987+0000","last_peered":"2024-09-13T17:57:29.740987+0000","last_clean":"2024-09-13T17:57:29.740987+0000","last_became_active":"2024-09-13T17:57:29.735361+0000","last_became_peered":"2024-09-13T17:57:29.735361+0000","last_unstale":"2024-09-13T17:57:29.740987+0000","last_undegraded":"2024-09-13T17:57:29.740987+0000","last_fullsized":"2024-09-13T17:57:29.740987+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:50.823708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720788+0000","last_change":"2024-09-13T17:57:25.378442+0000","last_active":"2024-09-13T17:57:29.720788+0000","last_peered":"2024-09-13T17:57:29.720788+0000","last_clean":"2024-09-13T17:57:29.720788+0000","last_became_active":"2024-09-13T17:57:25.378092+0000","last_became_peered":"2024-09-13T17:57:25.378092+0000","last_unstale":"2024-09-13T17:57:29.720788+0000","last_undegraded":"2024-09-13T17:57:29.720788+0000","last_fullsized":"2024-09-13T17:57:29.720788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:41.189680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.733727+0000","last_change":"2024-09-13T17:57:25.714970+0000","last_active":"2024-09-13T17:57:29.733727+0000","last_peered":"2024-09-13T17:57:29.733727+0000","last_clean":"2024-09-13T17:57:29.733727+0000","last_became_active":"2024-09-13T17:57:25.714338+0000","last_became_peered":"2024-09-13T17:57:25.714338+0000","last_unstale":"2024-09-13T17:57:29.733727+0000","last_undegraded":"2024-09-13T17:57:29.733727+0000","last_fullsized":"2024-09-13T17:57:29.733727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:11:58.896243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.745004+0000","last_change":"2024-09-13T17:57:29.741838+0000","last_active":"2024-09-13T17:57:29.745004+0000","last_peered":"2024-09-13T17:57:29.745004+0000","last_clean":"2024-09-13T17:57:29.745004+0000","last_became_active":"2024-09-13T17:57:29.741478+0000","last_became_peered":"2024-09-13T17:57:29.741478+0000","last_unstale":"2024-09-13T17:57:29.745004+0000","last_undegraded":"2024-09-13T17:57:29.745004+0000","last_fullsized":"2024-09-13T17:57:29.745004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:15.815856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728583+0000","last_change":"2024-09-13T17:57:25.734081+0000","last_active":"2024-09-13T17:57:29.728583+0000","last_peered":"2024-09-13T17:57:29.728583+0000","last_clean":"2024-09-13T17:57:29.728583+0000","last_became_active":"2024-09-13T17:57:25.733489+0000","last_became_peered":"2024-09-13T17:57:25.733489+0000","last_unstale":"2024-09-13T17:57:29.728583+0000","last_undegraded":"2024-09-13T17:57:29.728583+0000","last_fullsized":"2024-09-13T17:57:29.728583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:49.905668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.731942+0000","last_change":"2024-09-13T17:57:25.721413+0000","last_active":"2024-09-13T17:57:29.731942+0000","last_peered":"2024-09-13T17:57:29.731942+0000","last_clean":"2024-09-13T17:57:29.731942+0000","last_became_active":"2024-09-13T17:57:25.721167+0000","last_became_peered":"2024-09-13T17:57:25.721167+0000","last_unstale":"2024-09-13T17:57:29.731942+0000","last_undegraded":"2024-09-13T17:57:29.731942+0000","last_fullsized":"2024-09-13T17:57:29.731942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:39.042757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724511+0000","last_change":"2024-09-13T17:57:25.728836+0000","last_active":"2024-09-13T17:57:29.724511+0000","last_peered":"2024-09-13T17:57:29.724511+0000","last_clean":"2024-09-13T17:57:29.724511+0000","last_became_active":"2024-09-13T17:57:25.728566+0000","last_became_peered":"2024-09-13T17:57:25.728566+0000","last_unstale":"2024-09-13T17:57:29.724511+0000","last_undegraded":"2024-09-13T17:57:29.724511+0000","last_fullsized":"2024-09-13T17:57:29.724511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:50.812435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727862+0000","last_change":"2024-09-13T17:57:25.719773+0000","last_active":"2024-09-13T17:57:29.727862+0000","last_peered":"2024-09-13T17:57:29.727862+0000","last_clean":"2024-09-13T17:57:29.727862+0000","last_became_active":"2024-09-13T17:57:25.719057+0000","last_became_peered":"2024-09-13T17:57:25.719057+0000","last_unstale":"2024-09-13T17:57:29.727862+0000","last_undegraded":"2024-09-13T17:57:29.727862+0000","last_fullsized":"2024-09-13T17:57:29.727862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:13.197645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721032+0000","last_change":"2024-09-13T17:57:25.719998+0000","last_active":"2024-09-13T17:57:29.721032+0000","last_peered":"2024-09-13T17:57:29.721032+0000","last_clean":"2024-09-13T17:57:29.721032+0000","last_became_active":"2024-09-13T17:57:25.719739+0000","last_became_peered":"2024-09-13T17:57:25.719739+0000","last_unstale":"2024-09-13T17:57:29.721032+0000","last_undegraded":"2024-09-13T17:57:29.721032+0000","last_fullsized":"2024-09-13T17:57:29.721032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:47.463588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729592+0000","last_change":"2024-09-13T17:57:25.720277+0000","last_active":"2024-09-13T17:57:29.729592+0000","last_peered":"2024-09-13T17:57:29.729592+0000","last_clean":"2024-09-13T17:57:29.729592+0000","last_became_active":"2024-09-13T17:57:25.720034+0000","last_became_peered":"2024-09-13T17:57:25.720034+0000","last_unstale":"2024-09-13T17:57:29.729592+0000","last_undegraded":"2024-09-13T17:57:29.729592+0000","last_fullsized":"2024-09-13T17:57:29.729592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728536+0000","last_change":"2024-09-13T17:57:25.733794+0000","last_active":"2024-09-13T17:57:29.728536+0000","last_peered":"2024-09-13T17:57:29.728536+0000","last_clean":"2024-09-13T17:57:29.728536+0000","last_became_active":"2024-09-13T17:57:25.733512+0000","last_became_peered":"2024-09-13T17:57:25.733512+0000","last_unstale":"2024-09-13T17:57:29.728536+0000","last_undegraded":"2024-09-13T17:57:29.728536+0000","last_fullsized":"2024-09-13T17:57:29.728536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720839+0000","last_change":"2024-09-13T17:57:25.723300+0000","last_active":"2024-09-13T17:57:29.720839+0000","last_peered":"2024-09-13T17:57:29.720839+0000","last_clean":"2024-09-13T17:57:29.720839+0000","last_became_active":"2024-09-13T17:57:25.720475+0000","last_became_peered":"2024-09-13T17:57:25.720475+0000","last_unstale":"2024-09-13T17:57:29.720839+0000","last_undegraded":"2024-09-13T17:57:29.720839+0000","last_fullsized":"2024-09-13T17:57:29.720839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:37:23.859769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724381+0000","last_change":"2024-09-13T17:57:25.721201+0000","last_active":"2024-09-13T17:57:29.724381+0000","last_peered":"2024-09-13T17:57:29.724381+0000","last_clean":"2024-09-13T17:57:29.724381+0000","last_became_active":"2024-09-13T17:57:25.719998+0000","last_became_peered":"2024-09-13T17:57:25.719998+0000","last_unstale":"2024-09-13T17:57:29.724381+0000","last_undegraded":"2024-09-13T17:57:29.724381+0000","last_fullsized":"2024-09-13T17:57:29.724381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:33.971487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721180+0000","last_change":"2024-09-13T17:57:25.715726+0000","last_active":"2024-09-13T17:57:29.721180+0000","last_peered":"2024-09-13T17:57:29.721180+0000","last_clean":"2024-09-13T17:57:29.721180+0000","last_became_active":"2024-09-13T17:57:25.715459+0000","last_became_peered":"2024-09-13T17:57:25.715459+0000","last_unstale":"2024-09-13T17:57:29.721180+0000","last_undegraded":"2024-09-13T17:57:29.721180+0000","last_fullsized":"2024-09-13T17:57:29.721180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:49.905668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724413+0000","last_change":"2024-09-13T17:57:25.730059+0000","last_active":"2024-09-13T17:57:29.724413+0000","last_peered":"2024-09-13T17:57:29.724413+0000","last_clean":"2024-09-13T17:57:29.724413+0000","last_became_active":"2024-09-13T17:57:25.729083+0000","last_became_peered":"2024-09-13T17:57:25.729083+0000","last_unstale":"2024-09-13T17:57:29.724413+0000","last_undegraded":"2024-09-13T17:57:29.724413+0000","last_fullsized":"2024-09-13T17:57:29.724413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:12.960061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727633+0000","last_change":"2024-09-13T17:57:25.713791+0000","last_active":"2024-09-13T17:57:29.727633+0000","last_peered":"2024-09-13T17:57:29.727633+0000","last_clean":"2024-09-13T17:57:29.727633+0000","last_became_active":"2024-09-13T17:57:25.713540+0000","last_became_peered":"2024-09-13T17:57:25.713540+0000","last_unstale":"2024-09-13T17:57:29.727633+0000","last_undegraded":"2024-09-13T17:57:29.727633+0000","last_fullsized":"2024-09-13T17:57:29.727633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728046+0000","last_change":"2024-09-13T17:57:25.734509+0000","last_active":"2024-09-13T17:57:29.728046+0000","last_peered":"2024-09-13T17:57:29.728046+0000","last_clean":"2024-09-13T17:57:29.728046+0000","last_became_active":"2024-09-13T17:57:25.734233+0000","last_became_peered":"2024-09-13T17:57:25.734233+0000","last_unstale":"2024-09-13T17:57:29.728046+0000","last_undegraded":"2024-09-13T17:57:29.728046+0000","last_fullsized":"2024-09-13T17:57:29.728046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725310+0000","last_change":"2024-09-13T17:57:25.718661+0000","last_active":"2024-09-13T17:57:29.725310+0000","last_peered":"2024-09-13T17:57:29.725310+0000","last_clean":"2024-09-13T17:57:29.725310+0000","last_became_active":"2024-09-13T17:57:25.718366+0000","last_became_peered":"2024-09-13T17:57:25.718366+0000","last_unstale":"2024-09-13T17:57:29.725310+0000","last_undegraded":"2024-09-13T17:57:29.725310+0000","last_fullsized":"2024-09-13T17:57:29.725310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:12.960061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732206+0000","last_change":"2024-09-13T17:57:25.734683+0000","last_active":"2024-09-13T17:57:29.732206+0000","last_peered":"2024-09-13T17:57:29.732206+0000","last_clean":"2024-09-13T17:57:29.732206+0000","last_became_active":"2024-09-13T17:57:25.734409+0000","last_became_peered":"2024-09-13T17:57:25.734409+0000","last_unstale":"2024-09-13T17:57:29.732206+0000","last_undegraded":"2024-09-13T17:57:29.732206+0000","last_fullsized":"2024-09-13T17:57:29.732206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:59.252453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725864+0000","last_change":"2024-09-13T17:57:25.729680+0000","last_active":"2024-09-13T17:57:29.725864+0000","last_peered":"2024-09-13T17:57:29.725864+0000","last_clean":"2024-09-13T17:57:29.725864+0000","last_became_active":"2024-09-13T17:57:25.729402+0000","last_became_peered":"2024-09-13T17:57:25.729402+0000","last_unstale":"2024-09-13T17:57:29.725864+0000","last_undegraded":"2024-09-13T17:57:29.725864+0000","last_fullsized":"2024-09-13T17:57:29.725864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:43.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724296+0000","last_change":"2024-09-13T17:57:25.728126+0000","last_active":"2024-09-13T17:57:29.724296+0000","last_peered":"2024-09-13T17:57:29.724296+0000","last_clean":"2024-09-13T17:57:29.724296+0000","last_became_active":"2024-09-13T17:57:25.727793+0000","last_became_peered":"2024-09-13T17:57:25.727793+0000","last_unstale":"2024-09-13T17:57:29.724296+0000","last_undegraded":"2024-09-13T17:57:29.724296+0000","last_fullsized":"2024-09-13T17:57:29.724296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:34.797142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729099+0000","last_change":"2024-09-13T17:57:25.716918+0000","last_active":"2024-09-13T17:57:29.729099+0000","last_peered":"2024-09-13T17:57:29.729099+0000","last_clean":"2024-09-13T17:57:29.729099+0000","last_became_active":"2024-09-13T17:57:25.716550+0000","last_became_peered":"2024-09-13T17:57:25.716550+0000","last_unstale":"2024-09-13T17:57:29.729099+0000","last_undegraded":"2024-09-13T17:57:29.729099+0000","last_fullsized":"2024-09-13T17:57:29.729099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:19.536367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724836+0000","last_change":"2024-09-13T17:57:25.717079+0000","last_active":"2024-09-13T17:57:29.724836+0000","last_peered":"2024-09-13T17:57:29.724836+0000","last_clean":"2024-09-13T17:57:29.724836+0000","last_became_active":"2024-09-13T17:57:25.716782+0000","last_became_peered":"2024-09-13T17:57:25.716782+0000","last_unstale":"2024-09-13T17:57:29.724836+0000","last_undegraded":"2024-09-13T17:57:29.724836+0000","last_fullsized":"2024-09-13T17:57:29.724836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:26:19.049804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720610+0000","last_change":"2024-09-13T17:57:25.731733+0000","last_active":"2024-09-13T17:57:29.720610+0000","last_peered":"2024-09-13T17:57:29.720610+0000","last_clean":"2024-09-13T17:57:29.720610+0000","last_became_active":"2024-09-13T17:57:25.731460+0000","last_became_peered":"2024-09-13T17:57:25.731460+0000","last_unstale":"2024-09-13T17:57:29.720610+0000","last_undegraded":"2024-09-13T17:57:29.720610+0000","last_fullsized":"2024-09-13T17:57:29.720610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:43.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.735275+0000","last_change":"2024-09-13T17:57:25.734897+0000","last_active":"2024-09-13T17:57:29.735275+0000","last_peered":"2024-09-13T17:57:29.735275+0000","last_clean":"2024-09-13T17:57:29.735275+0000","last_became_active":"2024-09-13T17:57:25.734639+0000","last_became_peered":"2024-09-13T17:57:25.734639+0000","last_unstale":"2024-09-13T17:57:29.735275+0000","last_undegraded":"2024-09-13T17:57:29.735275+0000","last_fullsized":"2024-09-13T17:57:29.735275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:29.989683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728450+0000","last_change":"2024-09-13T17:57:25.736997+0000","last_active":"2024-09-13T17:57:29.728450+0000","last_peered":"2024-09-13T17:57:29.728450+0000","last_clean":"2024-09-13T17:57:29.728450+0000","last_became_active":"2024-09-13T17:57:25.736735+0000","last_became_peered":"2024-09-13T17:57:25.736735+0000","last_unstale":"2024-09-13T17:57:29.728450+0000","last_undegraded":"2024-09-13T17:57:29.728450+0000","last_fullsized":"2024-09-13T17:57:29.728450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728451+0000","last_change":"2024-09-13T17:57:25.736821+0000","last_active":"2024-09-13T17:57:29.728451+0000","last_peered":"2024-09-13T17:57:29.728451+0000","last_clean":"2024-09-13T17:57:29.728451+0000","last_became_active":"2024-09-13T17:57:25.736572+0000","last_became_peered":"2024-09-13T17:57:25.736572+0000","last_unstale":"2024-09-13T17:57:29.728451+0000","last_undegraded":"2024-09-13T17:57:29.728451+0000","last_fullsized":"2024-09-13T17:57:29.728451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:13.197645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725470+0000","last_change":"2024-09-13T17:57:25.725989+0000","last_active":"2024-09-13T17:57:29.725470+0000","last_peered":"2024-09-13T17:57:29.725470+0000","last_clean":"2024-09-13T17:57:29.725470+0000","last_became_active":"2024-09-13T17:57:25.725725+0000","last_became_peered":"2024-09-13T17:57:25.725725+0000","last_unstale":"2024-09-13T17:57:29.725470+0000","last_undegraded":"2024-09-13T17:57:29.725470+0000","last_fullsized":"2024-09-13T17:57:29.725470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:50.812435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720423+0000","last_change":"2024-09-13T17:57:25.723786+0000","last_active":"2024-09-13T17:57:29.720423+0000","last_peered":"2024-09-13T17:57:29.720423+0000","last_clean":"2024-09-13T17:57:29.720423+0000","last_became_active":"2024-09-13T17:57:25.723510+0000","last_became_peered":"2024-09-13T17:57:25.723510+0000","last_unstale":"2024-09-13T17:57:29.720423+0000","last_undegraded":"2024-09-13T17:57:29.720423+0000","last_fullsized":"2024-09-13T17:57:29.720423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725741+0000","last_change":"2024-09-13T17:57:25.721799+0000","last_active":"2024-09-13T17:57:29.725741+0000","last_peered":"2024-09-13T17:57:29.725741+0000","last_clean":"2024-09-13T17:57:29.725741+0000","last_became_active":"2024-09-13T17:57:25.721309+0000","last_became_peered":"2024-09-13T17:57:25.721309+0000","last_unstale":"2024-09-13T17:57:29.725741+0000","last_undegraded":"2024-09-13T17:57:29.725741+0000","last_fullsized":"2024-09-13T17:57:29.725741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:27.969805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724305+0000","last_change":"2024-09-13T17:57:25.727329+0000","last_active":"2024-09-13T17:57:29.724305+0000","last_peered":"2024-09-13T17:57:29.724305+0000","last_clean":"2024-09-13T17:57:29.724305+0000","last_became_active":"2024-09-13T17:57:25.727064+0000","last_became_peered":"2024-09-13T17:57:25.727064+0000","last_unstale":"2024-09-13T17:57:29.724305+0000","last_undegraded":"2024-09-13T17:57:29.724305+0000","last_fullsized":"2024-09-13T17:57:29.724305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725134+0000","last_change":"2024-09-13T17:57:25.714493+0000","last_active":"2024-09-13T17:57:29.725134+0000","last_peered":"2024-09-13T17:57:29.725134+0000","last_clean":"2024-09-13T17:57:29.725134+0000","last_became_active":"2024-09-13T17:57:25.714091+0000","last_became_peered":"2024-09-13T17:57:25.714091+0000","last_unstale":"2024-09-13T17:57:29.725134+0000","last_undegraded":"2024-09-13T17:57:29.725134+0000","last_fullsized":"2024-09-13T17:57:29.725134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:51.804478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725105+0000","last_change":"2024-09-13T17:57:25.730275+0000","last_active":"2024-09-13T17:57:29.725105+0000","last_peered":"2024-09-13T17:57:29.725105+0000","last_clean":"2024-09-13T17:57:29.725105+0000","last_became_active":"2024-09-13T17:57:25.729845+0000","last_became_peered":"2024-09-13T17:57:25.729845+0000","last_unstale":"2024-09-13T17:57:29.725105+0000","last_undegraded":"2024-09-13T17:57:29.725105+0000","last_fullsized":"2024-09-13T17:57:29.725105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:11.996031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725148+0000","last_change":"2024-09-13T17:57:25.714547+0000","last_active":"2024-09-13T17:57:29.725148+0000","last_peered":"2024-09-13T17:57:29.725148+0000","last_clean":"2024-09-13T17:57:29.725148+0000","last_became_active":"2024-09-13T17:57:25.714288+0000","last_became_peered":"2024-09-13T17:57:25.714288+0000","last_unstale":"2024-09-13T17:57:29.725148+0000","last_undegraded":"2024-09-13T17:57:29.725148+0000","last_fullsized":"2024-09-13T17:57:29.725148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:37:23.859769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729416+0000","last_change":"2024-09-13T17:57:25.727117+0000","last_active":"2024-09-13T17:57:29.729416+0000","last_peered":"2024-09-13T17:57:29.729416+0000","last_clean":"2024-09-13T17:57:29.729416+0000","last_became_active":"2024-09-13T17:57:25.726864+0000","last_became_peered":"2024-09-13T17:57:25.726864+0000","last_unstale":"2024-09-13T17:57:29.729416+0000","last_undegraded":"2024-09-13T17:57:29.729416+0000","last_fullsized":"2024-09-13T17:57:29.729416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725395+0000","last_change":"2024-09-13T17:57:25.734566+0000","last_active":"2024-09-13T17:57:29.725395+0000","last_peered":"2024-09-13T17:57:29.725395+0000","last_clean":"2024-09-13T17:57:29.725395+0000","last_became_active":"2024-09-13T17:57:25.733963+0000","last_became_peered":"2024-09-13T17:57:25.733963+0000","last_unstale":"2024-09-13T17:57:29.725395+0000","last_undegraded":"2024-09-13T17:57:29.725395+0000","last_fullsized":"2024-09-13T17:57:29.725395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:44.248672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":690730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":99870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":99906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":99780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":99940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":689767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:57:47.611 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T17:57:47.611 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T17:57:47.611 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T17:57:47.611 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T17:57:47.612 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T17:57:47.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.740+0000 7f90e7301640 1 Processor -- start 2024-09-13T17:57:47.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.741+0000 7f90e7301640 1 -- start start 2024-09-13T17:57:47.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.741+0000 7f90e7301640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e00ceef0 0x7f90e00af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:47.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.741+0000 7f90e7301640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f90e015c860 0x7f90e00cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:47.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.741+0000 7f90e7301640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e006bb10 con 0x7f90e00d0690 2024-09-13T17:57:47.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.741+0000 7f90e7301640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e00cf810 con 0x7f90e015c860 2024-09-13T17:57:47.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.741+0000 7f90e7301640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00ccb30 con 0x7f90e00ceef0 2024-09-13T17:57:47.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.741+0000 7f90e589e640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f90e00d0690 0x7f90e00d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49662/0 (socket says 172.21.15.111:49662) 2024-09-13T17:57:47.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.741+0000 7f90e489c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f90e015c860 0x7f90e00cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:47.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.741+0000 7f90e589e640 1 -- 172.21.15.111:0/2147127275 learned_addr learned my addr 172.21.15.111:0/2147127275 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:47.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.742+0000 7f90e509d640 1 --2- 172.21.15.111:0/2147127275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e00ceef0 0x7f90e00af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:47.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.742+0000 7f90dffff640 1 -- 172.21.15.111:0/2147127275 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210902635 0 0) 0x7f90e006bb10 con 0x7f90e00d0690 2024-09-13T17:57:47.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.742+0000 7f90dffff640 1 -- 172.21.15.111:0/2147127275 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90c8003520 con 0x7f90e00d0690 2024-09-13T17:57:47.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.742+0000 7f90e489c640 1 -- 172.21.15.111:0/2147127275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e00ceef0 msgr2=0x7f90e00af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:47.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.742+0000 7f90e489c640 1 --2- 172.21.15.111:0/2147127275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e00ceef0 0x7f90e00af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.742+0000 7f90e489c640 1 -- 172.21.15.111:0/2147127275 >> v1:172.21.15.111:6789/0 conn(0x7f90e00d0690 legacy=0x7f90e00d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.742+0000 7f90e489c640 1 -- 172.21.15.111:0/2147127275 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e00d13a0 con 0x7f90e015c860 2024-09-13T17:57:47.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.743+0000 7f90e489c640 1 --2- 172.21.15.111:0/2147127275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90e015c860 0x7f90e00cf2d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009790 tx=0x7f90d402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818e45f0c0ec9974 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:47.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.743+0000 7f90dffff640 1 -- 172.21.15.111:0/2147127275 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4032440 con 0x7f90e015c860 2024-09-13T17:57:47.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.743+0000 7f90dffff640 1 -- 172.21.15.111:0/2147127275 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d40325a0 con 0x7f90e015c860 2024-09-13T17:57:47.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.743+0000 7f90dffff640 1 -- 172.21.15.111:0/2147127275 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4032930 con 0x7f90e015c860 2024-09-13T17:57:47.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.743+0000 7f90e7301640 1 -- 172.21.15.111:0/2147127275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90e015c860 msgr2=0x7f90e00cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.743+0000 7f90e7301640 1 --2- 172.21.15.111:0/2147127275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90e015c860 0x7f90e00cf2d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009790 tx=0x7f90d402a210 comp rx=0 tx=0).stop 2024-09-13T17:57:47.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.743+0000 7f90e7301640 1 -- 172.21.15.111:0/2147127275 shutdown_connections 2024-09-13T17:57:47.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.743+0000 7f90e7301640 1 --2- 172.21.15.111:0/2147127275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90e015c860 0x7f90e00cf2d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.743+0000 7f90e7301640 1 --2- 172.21.15.111:0/2147127275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e00ceef0 0x7f90e00af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.743+0000 7f90e7301640 1 -- 172.21.15.111:0/2147127275 >> 172.21.15.111:0/2147127275 conn(0x7f90e009b870 msgr2=0x7f90e0089f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:47.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.743+0000 7f90e7301640 1 -- 172.21.15.111:0/2147127275 shutdown_connections 2024-09-13T17:57:47.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.743+0000 7f90e7301640 1 -- 172.21.15.111:0/2147127275 wait complete. 2024-09-13T17:57:47.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.744+0000 7f90e7301640 1 Processor -- start 2024-09-13T17:57:47.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.745+0000 7f90e7301640 1 -- start start 2024-09-13T17:57:47.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.745+0000 7f90e7301640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e00ceef0 0x7f90e015b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:47.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.745+0000 7f90e7301640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90e00d0690 0x7f90e00b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:47.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.745+0000 7f90e7301640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90e015c860 0x7f90e014a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:47.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.745+0000 7f90e7301640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00b4150 con 0x7f90e00d0690 2024-09-13T17:57:47.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.745+0000 7f90e7301640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00b3540 con 0x7f90e015c860 2024-09-13T17:57:47.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.745+0000 7f90e7301640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00b3820 con 0x7f90e00ceef0 2024-09-13T17:57:47.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.745+0000 7f90e489c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90e00d0690 0x7f90e00b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:47.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.745+0000 7f90e489c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90e00d0690 0x7f90e00b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53892/0 (socket says 172.21.15.111:53892) 2024-09-13T17:57:47.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.745+0000 7f90e489c640 1 -- 172.21.15.111:0/1807417164 learned_addr learned my addr 172.21.15.111:0/1807417164 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:47.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.746+0000 7f90e509d640 1 --2- 172.21.15.111:0/1807417164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e00ceef0 0x7f90e015b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:47.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.746+0000 7f90e589e640 1 --2- 172.21.15.111:0/1807417164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90e015c860 0x7f90e014a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:47.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.746+0000 7f90e489c640 1 -- 172.21.15.111:0/1807417164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e00ceef0 msgr2=0x7f90e015b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.746+0000 7f90e489c640 1 --2- 172.21.15.111:0/1807417164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e00ceef0 0x7f90e015b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.746+0000 7f90e489c640 1 -- 172.21.15.111:0/1807417164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90e015c860 msgr2=0x7f90e014a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.746+0000 7f90e489c640 1 --2- 172.21.15.111:0/1807417164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90e015c860 0x7f90e014a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.746+0000 7f90e489c640 1 -- 172.21.15.111:0/1807417164 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d40093a0 con 0x7f90e00d0690 2024-09-13T17:57:47.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.746+0000 7f90e509d640 1 --2- 172.21.15.111:0/1807417164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e00ceef0 0x7f90e015b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:47.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.746+0000 7f90e489c640 1 --2- 172.21.15.111:0/1807417164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90e00d0690 0x7f90e00b2af0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f90d4035040 tx=0x7f90d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:47.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.746+0000 7f90ddffb640 1 -- 172.21.15.111:0/1807417164 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4005da0 con 0x7f90e00d0690 2024-09-13T17:57:47.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.746+0000 7f90ddffb640 1 -- 172.21.15.111:0/1807417164 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d4032e90 con 0x7f90e00d0690 2024-09-13T17:57:47.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.746+0000 7f90e7301640 1 -- 172.21.15.111:0/1807417164 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e00b3b00 con 0x7f90e00d0690 2024-09-13T17:57:47.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.746+0000 7f90ddffb640 1 -- 172.21.15.111:0/1807417164 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d40309b0 con 0x7f90e00d0690 2024-09-13T17:57:47.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.746+0000 7f90e7301640 1 -- 172.21.15.111:0/1807417164 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e015b9c0 con 0x7f90e00d0690 2024-09-13T17:57:47.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.747+0000 7f90e7301640 1 -- 172.21.15.111:0/1807417164 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e00b4300 con 0x7f90e00d0690 2024-09-13T17:57:47.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.748+0000 7f90ddffb640 1 -- 172.21.15.111:0/1807417164 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f90d4039040 con 0x7f90e00d0690 2024-09-13T17:57:47.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.748+0000 7f90ddffb640 1 --2- 172.21.15.111:0/1807417164 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f90cc06ac30 0x7f90cc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:47.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.748+0000 7f90ddffb640 1 -- 172.21.15.111:0/1807417164 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f90d40a9f70 con 0x7f90e00d0690 2024-09-13T17:57:47.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.748+0000 7f90e509d640 1 --2- 172.21.15.111:0/1807417164 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f90cc06ac30 0x7f90cc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:47.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.748+0000 7f90e509d640 1 --2- 172.21.15.111:0/1807417164 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f90cc06ac30 0x7f90cc06d0f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f90d0006fd0 tx=0x7f90d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:47.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.752+0000 7f90ddffb640 1 -- 172.21.15.111:0/1807417164 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f90d40dd9c0 con 0x7f90e00d0690 2024-09-13T17:57:47.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.957+0000 7f90e7301640 1 -- 172.21.15.111:0/1807417164 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f90e00b0250 con 0x7f90e00d0690 2024-09-13T17:57:47.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.960+0000 7f90ddffb640 1 -- 172.21.15.111:0/1807417164 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170957 (secure 0 0 0) 0x7f90d40dabe0 con 0x7f90e00d0690 2024-09-13T17:57:47.965 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:47.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.963+0000 7f90e7301640 1 -- 172.21.15.111:0/1807417164 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f90cc06ac30 msgr2=0x7f90cc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.963+0000 7f90e7301640 1 --2- 172.21.15.111:0/1807417164 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f90cc06ac30 0x7f90cc06d0f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f90d0006fd0 tx=0x7f90d0008040 comp rx=0 tx=0).stop 2024-09-13T17:57:47.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.963+0000 7f90e7301640 1 -- 172.21.15.111:0/1807417164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90e00d0690 msgr2=0x7f90e00b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:47.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.963+0000 7f90e7301640 1 --2- 172.21.15.111:0/1807417164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90e00d0690 0x7f90e00b2af0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f90d4035040 tx=0x7f90d40046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.963+0000 7f90e7301640 1 -- 172.21.15.111:0/1807417164 shutdown_connections 2024-09-13T17:57:47.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.963+0000 7f90e7301640 1 --2- 172.21.15.111:0/1807417164 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f90cc06ac30 0x7f90cc06d0f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.963+0000 7f90e7301640 1 --2- 172.21.15.111:0/1807417164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90e015c860 0x7f90e014a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.963+0000 7f90e7301640 1 --2- 172.21.15.111:0/1807417164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90e00d0690 0x7f90e00b2af0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.963+0000 7f90e7301640 1 --2- 172.21.15.111:0/1807417164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e00ceef0 0x7f90e015b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:47.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.963+0000 7f90e7301640 1 -- 172.21.15.111:0/1807417164 >> 172.21.15.111:0/1807417164 conn(0x7f90e009b870 msgr2=0x7f90e00d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:47.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.963+0000 7f90e7301640 1 -- 172.21.15.111:0/1807417164 shutdown_connections 2024-09-13T17:57:47.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:47.963+0000 7f90e7301640 1 -- 172.21.15.111:0/1807417164 wait complete. 2024-09-13T17:57:47.976 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2183469065},{"type":"v1","addr":"172.21.15.111:6833","nonce":2183469065}]},"active_addr":"172.21.15.111:6833/2183469065","active_change":"2024-09-13T17:57:06.258609+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":2470174188}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":461992946}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":3381953266}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":553103951}]}]} 2024-09-13T17:57:47.979 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T17:57:47.979 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T17:57:47.979 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:48.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.074+0000 7f01e6253640 1 Processor -- start 2024-09-13T17:57:48.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.075+0000 7f01e6253640 1 -- start start 2024-09-13T17:57:48.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.075+0000 7f01e6253640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f01e0071d30 0x7f01e0070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.075+0000 7f01e6253640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f01e00c8920 0x7f01e00c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.075+0000 7f01e6253640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e00abdf0 con 0x7f01e0072110 2024-09-13T17:57:48.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.075+0000 7f01e6253640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e00c92f0 con 0x7f01e00c8920 2024-09-13T17:57:48.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.075+0000 7f01e6253640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e00b2280 con 0x7f01e0071d30 2024-09-13T17:57:48.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.075+0000 7f01dffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f01e00c8920 0x7f01e00c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.075+0000 7f01deffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f01e0072110 0x7f01e00b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49674/0 (socket says 172.21.15.111:49674) 2024-09-13T17:57:48.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.075+0000 7f01deffd640 1 -- 172.21.15.111:0/143994129 learned_addr learned my addr 172.21.15.111:0/143994129 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:48.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.076+0000 7f01df7fe640 1 --2- 172.21.15.111:0/143994129 >> v2:172.21.15.188:3300/0 conn(0x7f01e0071d30 0x7f01e0070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.076+0000 7f01dffff640 1 -- 172.21.15.111:0/143994129 >> v2:172.21.15.188:3300/0 conn(0x7f01e0071d30 msgr2=0x7f01e0070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:48.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.076+0000 7f01dffff640 1 --2- 172.21.15.111:0/143994129 >> v2:172.21.15.188:3300/0 conn(0x7f01e0071d30 0x7f01e0070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.076+0000 7f01dffff640 1 -- 172.21.15.111:0/143994129 >> v1:172.21.15.111:6789/0 conn(0x7f01e0072110 legacy=0x7f01e00b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.076+0000 7f01de7fc640 1 -- 172.21.15.111:0/143994129 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191803460 0 0) 0x7f01e00abdf0 con 0x7f01e0072110 2024-09-13T17:57:48.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.076+0000 7f01dffff640 1 -- 172.21.15.111:0/143994129 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e00ab1e0 con 0x7f01e00c8920 2024-09-13T17:57:48.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.076+0000 7f01dffff640 1 --2- 172.21.15.111:0/143994129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01e00c8920 0x7f01e00c8db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f01e00cee60 tx=0x7f01d802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba86f6a67b3d4c26 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.077+0000 7f01de7fc640 1 -- 172.21.15.111:0/143994129 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8032440 con 0x7f01e00c8920 2024-09-13T17:57:48.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.077+0000 7f01de7fc640 1 -- 172.21.15.111:0/143994129 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d80325a0 con 0x7f01e00c8920 2024-09-13T17:57:48.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.077+0000 7f01de7fc640 1 -- 172.21.15.111:0/143994129 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8032930 con 0x7f01e00c8920 2024-09-13T17:57:48.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.077+0000 7f01e6253640 1 -- 172.21.15.111:0/143994129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01e00c8920 msgr2=0x7f01e00c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.077+0000 7f01e6253640 1 --2- 172.21.15.111:0/143994129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01e00c8920 0x7f01e00c8db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f01e00cee60 tx=0x7f01d802a1a0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.077+0000 7f01e6253640 1 -- 172.21.15.111:0/143994129 shutdown_connections 2024-09-13T17:57:48.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.077+0000 7f01e6253640 1 --2- 172.21.15.111:0/143994129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01e00c8920 0x7f01e00c8db0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.077+0000 7f01e6253640 1 --2- 172.21.15.111:0/143994129 >> v2:172.21.15.188:3300/0 conn(0x7f01e0071d30 0x7f01e0070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.077+0000 7f01e6253640 1 -- 172.21.15.111:0/143994129 >> 172.21.15.111:0/143994129 conn(0x7f01e0069ed0 msgr2=0x7f01e006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.077+0000 7f01e6253640 1 -- 172.21.15.111:0/143994129 shutdown_connections 2024-09-13T17:57:48.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.077+0000 7f01e6253640 1 -- 172.21.15.111:0/143994129 wait complete. 2024-09-13T17:57:48.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.078+0000 7f01e6253640 1 Processor -- start 2024-09-13T17:57:48.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.078+0000 7f01e6253640 1 -- start start 2024-09-13T17:57:48.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.079+0000 7f01e6253640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01e0071d30 0x7f01e015c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.079+0000 7f01e6253640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01e0072110 0x7f01e01f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.079+0000 7f01e6253640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01e00c8920 0x7f01e006c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.079+0000 7f01e6253640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e01f3040 con 0x7f01e0071d30 2024-09-13T17:57:48.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.079+0000 7f01e6253640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e01f5d60 con 0x7f01e00c8920 2024-09-13T17:57:48.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.079+0000 7f01e6253640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e01f6040 con 0x7f01e0072110 2024-09-13T17:57:48.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.079+0000 7f01df7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01e0071d30 0x7f01e015c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.079+0000 7f01df7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01e0071d30 0x7f01e015c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53924/0 (socket says 172.21.15.111:53924) 2024-09-13T17:57:48.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.079+0000 7f01df7fe640 1 -- 172.21.15.111:0/889850638 learned_addr learned my addr 172.21.15.111:0/889850638 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:48.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.079+0000 7f01dffff640 1 --2- 172.21.15.111:0/889850638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01e00c8920 0x7f01e006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.079+0000 7f01deffd640 1 --2- 172.21.15.111:0/889850638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01e0072110 0x7f01e01f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.080+0000 7f01df7fe640 1 -- 172.21.15.111:0/889850638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01e0072110 msgr2=0x7f01e01f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.080+0000 7f01df7fe640 1 --2- 172.21.15.111:0/889850638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01e0072110 0x7f01e01f5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.080+0000 7f01df7fe640 1 -- 172.21.15.111:0/889850638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01e00c8920 msgr2=0x7f01e006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.080+0000 7f01df7fe640 1 --2- 172.21.15.111:0/889850638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01e00c8920 0x7f01e006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.080+0000 7f01df7fe640 1 -- 172.21.15.111:0/889850638 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d8009400 con 0x7f01e0071d30 2024-09-13T17:57:48.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.080+0000 7f01deffd640 1 --2- 172.21.15.111:0/889850638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01e0072110 0x7f01e01f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:48.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.080+0000 7f01df7fe640 1 --2- 172.21.15.111:0/889850638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01e0071d30 0x7f01e015c1d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f01d0007ef0 tx=0x7f01d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.080+0000 7f01c7fff640 1 -- 172.21.15.111:0/889850638 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0014620 con 0x7f01e0071d30 2024-09-13T17:57:48.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.080+0000 7f01c7fff640 1 -- 172.21.15.111:0/889850638 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d0014780 con 0x7f01e0071d30 2024-09-13T17:57:48.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.080+0000 7f01e6253640 1 -- 172.21.15.111:0/889850638 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e01f3300 con 0x7f01e0071d30 2024-09-13T17:57:48.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.080+0000 7f01c7fff640 1 -- 172.21.15.111:0/889850638 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0014b30 con 0x7f01e0071d30 2024-09-13T17:57:48.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.080+0000 7f01e6253640 1 -- 172.21.15.111:0/889850638 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e01f1f50 con 0x7f01e0071d30 2024-09-13T17:57:48.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.081+0000 7f01e6253640 1 -- 172.21.15.111:0/889850638 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e01f2ea0 con 0x7f01e0071d30 2024-09-13T17:57:48.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.082+0000 7f01c7fff640 1 -- 172.21.15.111:0/889850638 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f01d0014c90 con 0x7f01e0071d30 2024-09-13T17:57:48.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.082+0000 7f01c7fff640 1 --2- 172.21.15.111:0/889850638 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f01cc06ac30 0x7f01cc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.082+0000 7f01c7fff640 1 -- 172.21.15.111:0/889850638 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f01d0089f10 con 0x7f01e0071d30 2024-09-13T17:57:48.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.082+0000 7f01deffd640 1 --2- 172.21.15.111:0/889850638 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f01cc06ac30 0x7f01cc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.082+0000 7f01deffd640 1 --2- 172.21.15.111:0/889850638 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f01cc06ac30 0x7f01cc06d0f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f01e00acc90 tx=0x7f01d4007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.086+0000 7f01c7fff640 1 -- 172.21.15.111:0/889850638 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f01e01f2ea0 con 0x7f01e0071d30 2024-09-13T17:57:48.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.249+0000 7f01e6253640 1 -- 172.21.15.111:0/889850638 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01e01f2ea0 con 0x7f01e0071d30 2024-09-13T17:57:48.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.250+0000 7f01c7fff640 1 -- 172.21.15.111:0/889850638 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f01e01f2ea0 con 0x7f01e0071d30 2024-09-13T17:57:48.253 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:48.253 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:30.065235+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:48.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.252+0000 7f01e6253640 1 -- 172.21.15.111:0/889850638 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f01cc06ac30 msgr2=0x7f01cc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.252+0000 7f01e6253640 1 --2- 172.21.15.111:0/889850638 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f01cc06ac30 0x7f01cc06d0f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f01e00acc90 tx=0x7f01d4007c10 comp rx=0 tx=0).stop 2024-09-13T17:57:48.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.252+0000 7f01e6253640 1 -- 172.21.15.111:0/889850638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01e0071d30 msgr2=0x7f01e015c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.252+0000 7f01e6253640 1 --2- 172.21.15.111:0/889850638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01e0071d30 0x7f01e015c1d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f01d0007ef0 tx=0x7f01d000e4b0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.252+0000 7f01e6253640 1 -- 172.21.15.111:0/889850638 shutdown_connections 2024-09-13T17:57:48.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.252+0000 7f01e6253640 1 --2- 172.21.15.111:0/889850638 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f01cc06ac30 0x7f01cc06d0f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.252+0000 7f01e6253640 1 --2- 172.21.15.111:0/889850638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01e00c8920 0x7f01e006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.252+0000 7f01e6253640 1 --2- 172.21.15.111:0/889850638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01e0072110 0x7f01e01f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.252+0000 7f01e6253640 1 --2- 172.21.15.111:0/889850638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01e0071d30 0x7f01e015c1d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.252+0000 7f01e6253640 1 -- 172.21.15.111:0/889850638 >> 172.21.15.111:0/889850638 conn(0x7f01e0069ed0 msgr2=0x7f01e006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.252+0000 7f01e6253640 1 -- 172.21.15.111:0/889850638 shutdown_connections 2024-09-13T17:57:48.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.252+0000 7f01e6253640 1 -- 172.21.15.111:0/889850638 wait complete. 2024-09-13T17:57:48.266 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T17:57:48.266 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:48.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.361+0000 7fc6f51ec640 1 Processor -- start 2024-09-13T17:57:48.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.361+0000 7fc6f51ec640 1 -- start start 2024-09-13T17:57:48.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.362+0000 7fc6f51ec640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc6f015caf0 0x7fc6f015cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.362+0000 7fc6f51ec640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6f01640c0 0x7fc6f01664b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.362+0000 7fc6f51ec640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f015d470 con 0x7fc6f015c700 2024-09-13T17:57:48.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.362+0000 7fc6f51ec640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f00cf570 con 0x7fc6f015caf0 2024-09-13T17:57:48.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.362+0000 7fc6f51ec640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f00cf990 con 0x7fc6f01640c0 2024-09-13T17:57:48.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.362+0000 7fc6e7fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc6f015caf0 0x7fc6f015cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.362+0000 7fc6e7fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc6f015caf0 0x7fc6f015cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53934/0 (socket says 172.21.15.111:53934) 2024-09-13T17:57:48.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.362+0000 7fc6e7fff640 1 -- 172.21.15.111:0/3072745915 learned_addr learned my addr 172.21.15.111:0/3072745915 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:48.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.362+0000 7fc6ef59e640 1 --2- 172.21.15.111:0/3072745915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6f01640c0 0x7fc6f01664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.362+0000 7fc6e7fff640 1 -- 172.21.15.111:0/3072745915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6f01640c0 msgr2=0x7fc6f01664b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:48.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.362+0000 7fc6e7fff640 1 --2- 172.21.15.111:0/3072745915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6f01640c0 0x7fc6f01664b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.362+0000 7fc6e7fff640 1 -- 172.21.15.111:0/3072745915 >> v1:172.21.15.188:6789/0 conn(0x7fc6f015c700 legacy=0x7fc6f00993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.362+0000 7fc6e7fff640 1 -- 172.21.15.111:0/3072745915 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f00ce8f0 con 0x7fc6f015caf0 2024-09-13T17:57:48.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.363+0000 7fc6e7fff640 1 --2- 172.21.15.111:0/3072745915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f015caf0 0x7fc6f015cf30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009870 tx=0x7fc6d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192d3fcb6eb9eecd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.363+0000 7fc6ee59c640 1 -- 172.21.15.111:0/3072745915 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8032440 con 0x7fc6f015caf0 2024-09-13T17:57:48.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.363+0000 7fc6ee59c640 1 -- 172.21.15.111:0/3072745915 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d80325a0 con 0x7fc6f015caf0 2024-09-13T17:57:48.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.363+0000 7fc6ee59c640 1 -- 172.21.15.111:0/3072745915 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8032930 con 0x7fc6f015caf0 2024-09-13T17:57:48.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.364+0000 7fc6f51ec640 1 -- 172.21.15.111:0/3072745915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f015caf0 msgr2=0x7fc6f015cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.364+0000 7fc6f51ec640 1 --2- 172.21.15.111:0/3072745915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f015caf0 0x7fc6f015cf30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009870 tx=0x7fc6d802a270 comp rx=0 tx=0).stop 2024-09-13T17:57:48.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.364+0000 7fc6f51ec640 1 -- 172.21.15.111:0/3072745915 shutdown_connections 2024-09-13T17:57:48.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.364+0000 7fc6f51ec640 1 --2- 172.21.15.111:0/3072745915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f015caf0 0x7fc6f015cf30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.364+0000 7fc6f51ec640 1 --2- 172.21.15.111:0/3072745915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6f01640c0 0x7fc6f01664b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.364+0000 7fc6f51ec640 1 -- 172.21.15.111:0/3072745915 >> 172.21.15.111:0/3072745915 conn(0x7fc6f00b1dc0 msgr2=0x7fc6f00b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.364+0000 7fc6f51ec640 1 -- 172.21.15.111:0/3072745915 shutdown_connections 2024-09-13T17:57:48.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.364+0000 7fc6f51ec640 1 -- 172.21.15.111:0/3072745915 wait complete. 2024-09-13T17:57:48.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.365+0000 7fc6f51ec640 1 Processor -- start 2024-09-13T17:57:48.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.365+0000 7fc6f51ec640 1 -- start start 2024-09-13T17:57:48.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.366+0000 7fc6f51ec640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6f015caf0 0x7fc6f0150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.366+0000 7fc6f51ec640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f01640c0 0x7fc6f0140120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.366+0000 7fc6f51ec640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6f0140660 0x7fc6f00b3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.366+0000 7fc6f51ec640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f0140b10 con 0x7fc6f01640c0 2024-09-13T17:57:48.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.366+0000 7fc6f51ec640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f006ec90 con 0x7fc6f0140660 2024-09-13T17:57:48.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.366+0000 7fc6f51ec640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f006ef10 con 0x7fc6f015caf0 2024-09-13T17:57:48.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.366+0000 7fc6e7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f01640c0 0x7fc6f0140120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.366+0000 7fc6e7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f01640c0 0x7fc6f0140120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53942/0 (socket says 172.21.15.111:53942) 2024-09-13T17:57:48.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.366+0000 7fc6e7fff640 1 -- 172.21.15.111:0/3033470094 learned_addr learned my addr 172.21.15.111:0/3033470094 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:48.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.366+0000 7fc6eed9d640 1 --2- 172.21.15.111:0/3033470094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6f015caf0 0x7fc6f0150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.366+0000 7fc6ef59e640 1 --2- 172.21.15.111:0/3033470094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6f0140660 0x7fc6f00b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.366+0000 7fc6e7fff640 1 -- 172.21.15.111:0/3033470094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6f015caf0 msgr2=0x7fc6f0150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.366+0000 7fc6e7fff640 1 --2- 172.21.15.111:0/3033470094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6f015caf0 0x7fc6f0150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.366+0000 7fc6e7fff640 1 -- 172.21.15.111:0/3033470094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6f0140660 msgr2=0x7fc6f00b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.366+0000 7fc6e7fff640 1 --2- 172.21.15.111:0/3033470094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6f0140660 0x7fc6f00b3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.366+0000 7fc6e7fff640 1 -- 172.21.15.111:0/3033470094 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d80094d0 con 0x7fc6f01640c0 2024-09-13T17:57:48.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.366+0000 7fc6e7fff640 1 --2- 172.21.15.111:0/3033470094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f01640c0 0x7fc6f0140120 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc6d802a2b0 tx=0x7fc6d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.367+0000 7fc6e77fe640 1 -- 172.21.15.111:0/3033470094 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8032440 con 0x7fc6f01640c0 2024-09-13T17:57:48.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.367+0000 7fc6e77fe640 1 -- 172.21.15.111:0/3033470094 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d8030980 con 0x7fc6f01640c0 2024-09-13T17:57:48.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.367+0000 7fc6f51ec640 1 -- 172.21.15.111:0/3033470094 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f006f1f0 con 0x7fc6f01640c0 2024-09-13T17:57:48.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.367+0000 7fc6e77fe640 1 -- 172.21.15.111:0/3033470094 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8030d30 con 0x7fc6f01640c0 2024-09-13T17:57:48.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.367+0000 7fc6f51ec640 1 -- 172.21.15.111:0/3033470094 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f006f7b0 con 0x7fc6f01640c0 2024-09-13T17:57:48.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.368+0000 7fc6f51ec640 1 -- 172.21.15.111:0/3033470094 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f00ce510 con 0x7fc6f01640c0 2024-09-13T17:57:48.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.368+0000 7fc6e77fe640 1 -- 172.21.15.111:0/3033470094 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc6d8039040 con 0x7fc6f01640c0 2024-09-13T17:57:48.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.368+0000 7fc6e77fe640 1 --2- 172.21.15.111:0/3033470094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc6d006ac00 0x7fc6d006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.368+0000 7fc6e77fe640 1 -- 172.21.15.111:0/3033470094 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc6d80a9e30 con 0x7fc6f01640c0 2024-09-13T17:57:48.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.368+0000 7fc6eed9d640 1 --2- 172.21.15.111:0/3033470094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc6d006ac00 0x7fc6d006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.369+0000 7fc6eed9d640 1 --2- 172.21.15.111:0/3033470094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc6d006ac00 0x7fc6d006d0c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0000f60 tx=0x7fc6e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.373+0000 7fc6e77fe640 1 -- 172.21.15.111:0/3033470094 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc6d8074dd0 con 0x7fc6f01640c0 2024-09-13T17:57:48.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.535+0000 7fc6f51ec640 1 -- 172.21.15.111:0/3033470094 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6f00d00b0 con 0x7fc6f01640c0 2024-09-13T17:57:48.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.536+0000 7fc6e77fe640 1 -- 172.21.15.111:0/3033470094 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7fc6d80783f0 con 0x7fc6f01640c0 2024-09-13T17:57:48.539 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:48.539 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:30.065235+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:48.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.538+0000 7fc6f51ec640 1 -- 172.21.15.111:0/3033470094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc6d006ac00 msgr2=0x7fc6d006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.538+0000 7fc6f51ec640 1 --2- 172.21.15.111:0/3033470094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc6d006ac00 0x7fc6d006d0c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0000f60 tx=0x7fc6e0009040 comp rx=0 tx=0).stop 2024-09-13T17:57:48.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.538+0000 7fc6f51ec640 1 -- 172.21.15.111:0/3033470094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f01640c0 msgr2=0x7fc6f0140120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.538+0000 7fc6f51ec640 1 --2- 172.21.15.111:0/3033470094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f01640c0 0x7fc6f0140120 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc6d802a2b0 tx=0x7fc6d80046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.538+0000 7fc6f51ec640 1 -- 172.21.15.111:0/3033470094 shutdown_connections 2024-09-13T17:57:48.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.538+0000 7fc6f51ec640 1 --2- 172.21.15.111:0/3033470094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc6d006ac00 0x7fc6d006d0c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.538+0000 7fc6f51ec640 1 --2- 172.21.15.111:0/3033470094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6f0140660 0x7fc6f00b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.538+0000 7fc6f51ec640 1 --2- 172.21.15.111:0/3033470094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f01640c0 0x7fc6f0140120 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.538+0000 7fc6f51ec640 1 --2- 172.21.15.111:0/3033470094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6f015caf0 0x7fc6f0150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.538+0000 7fc6f51ec640 1 -- 172.21.15.111:0/3033470094 >> 172.21.15.111:0/3033470094 conn(0x7fc6f00b1dc0 msgr2=0x7fc6f00afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.538+0000 7fc6f51ec640 1 -- 172.21.15.111:0/3033470094 shutdown_connections 2024-09-13T17:57:48.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.538+0000 7fc6f51ec640 1 -- 172.21.15.111:0/3033470094 wait complete. 2024-09-13T17:57:48.552 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T17:57:48.552 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T17:57:48.553 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T17:57:48.553 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T17:57:48.553 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T17:57:48.554 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T17:57:48.554 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T17:57:48.555 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T17:57:48.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.718+0000 7ff94b5f4640 1 Processor -- start 2024-09-13T17:57:48.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.718+0000 7ff94b5f4640 1 -- start start 2024-09-13T17:57:48.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.718+0000 7fec12273640 1 Processor -- start 2024-09-13T17:57:48.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.719+0000 7f4b13253640 1 Processor -- start 2024-09-13T17:57:48.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.719+0000 7fec12273640 1 -- start start 2024-09-13T17:57:48.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.719+0000 7f606fd5e640 1 Processor -- start 2024-09-13T17:57:48.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.719+0000 7f5f109a0640 1 Processor -- start 2024-09-13T17:57:48.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.719+0000 7f5f109a0640 1 -- start start 2024-09-13T17:57:48.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.719+0000 7ff94b5f4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff9440b4cb0 0x7ff94415a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.719+0000 7f6a8e8d0640 1 Processor -- start 2024-09-13T17:57:48.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.719+0000 7ff94b5f4640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9440b2740 con 0x7ff944153660 2024-09-13T17:57:48.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.719+0000 7ff94b5f4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff944153a40 con 0x7ff9440b2a20 2024-09-13T17:57:48.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.719+0000 7ff94b5f4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff94406d1e0 con 0x7ff9440b4cb0 2024-09-13T17:57:48.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.720+0000 7ff949b91640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ff9440b2a20 0x7ff9440b2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:45644/0 (socket says 172.21.15.111:45644) 2024-09-13T17:57:48.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.720+0000 7ff949b91640 1 -- 172.21.15.111:0/13454204 learned_addr learned my addr 172.21.15.111:0/13454204 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:48.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.719+0000 7f6a8e8d0640 1 -- start start 2024-09-13T17:57:48.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.719+0000 7f4b13253640 1 -- start start 2024-09-13T17:57:48.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.720+0000 7ff948b8f640 1 --2- 172.21.15.111:0/13454204 >> v2:172.21.15.188:3300/0 conn(0x7ff9440b4cb0 0x7ff94415a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.720+0000 7f4b13253640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b0c160140 0x7f4b0c0d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.720+0000 7f606fd5e640 1 -- start start 2024-09-13T17:57:48.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.720+0000 7ff948b8f640 1 -- 172.21.15.111:0/13454204 >> v1:172.21.15.188:6789/0 conn(0x7ff9440b2a20 legacy=0x7ff9440b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.720+0000 7ff948b8f640 1 -- 172.21.15.111:0/13454204 >> v1:172.21.15.111:6789/0 conn(0x7ff944153660 legacy=0x7ff9440b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.720+0000 7ff948b8f640 1 -- 172.21.15.111:0/13454204 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94406d4c0 con 0x7ff9440b4cb0 2024-09-13T17:57:48.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.720+0000 7f5f109a0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f04005a20 0x7f5f04053f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.720+0000 7f4b10fef640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b0c160140 0x7f4b0c0d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.720+0000 7f4b10fef640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b0c160140 0x7f4b0c0d11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:34386/0 (socket says 172.21.15.111:34386) 2024-09-13T17:57:48.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f4b10fef640 1 -- 172.21.15.111:0/2297403161 learned_addr learned my addr 172.21.15.111:0/2297403161 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:48.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f5f109a0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5f040e70e0 0x7f5f04005e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f5f0b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f04005a20 0x7f5f04053f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f5f0b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f04005a20 0x7f5f04053f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:34396/0 (socket says 172.21.15.111:34396) 2024-09-13T17:57:48.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7fb596b53640 1 Processor -- start 2024-09-13T17:57:48.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f6a8e8d0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a880b0980 0x7f6a8806dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f6a8e8d0640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a880ada80 con 0x7f6a88158ae0 2024-09-13T17:57:48.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f6a8e8d0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8806ce40 con 0x7f6a880cbea0 2024-09-13T17:57:48.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f4b13253640 1 -- 172.21.15.111:0/2297403161 --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b0c0d2e90 con 0x7f4b0c0cd860 2024-09-13T17:57:48.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f4b13253640 1 -- 172.21.15.111:0/2297403161 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b0c0ce150 con 0x7f4b0c0cdca0 2024-09-13T17:57:48.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f4b13253640 1 -- 172.21.15.111:0/2297403161 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c0ccb30 con 0x7f4b0c160140 2024-09-13T17:57:48.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f6a8e8d0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a8806d120 con 0x7f6a880b0980 2024-09-13T17:57:48.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f5f0affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5f040e70e0 0x7f5f04005e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f5f0affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5f040e70e0 0x7f5f04005e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53954/0 (socket says 172.21.15.111:53954) 2024-09-13T17:57:48.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f5f0b7fe640 1 -- 172.21.15.111:0/2790084556 learned_addr learned my addr 172.21.15.111:0/2790084556 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:48.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f6a877fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6a88158ae0 0x7f6a880ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49704/0 (socket says 172.21.15.111:49704) 2024-09-13T17:57:48.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f6a877fe640 1 -- 172.21.15.111:0/3511215785 learned_addr learned my addr 172.21.15.111:0/3511215785 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:48.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f5f109a0640 1 -- 172.21.15.111:0/2790084556 --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f040520a0 con 0x7f5f0404eb20 2024-09-13T17:57:48.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f6a8ce6d640 1 --2- 172.21.15.111:0/3511215785 >> v2:172.21.15.188:3300/0 conn(0x7f6a880b0980 0x7f6a8806dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f5f109a0640 1 -- 172.21.15.111:0/2790084556 --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f04006340 con 0x7f5f040e70e0 2024-09-13T17:57:48.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f5f109a0640 1 -- 172.21.15.111:0/2790084556 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f040522f0 con 0x7f5f04005a20 2024-09-13T17:57:48.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7fb596b53640 1 -- start start 2024-09-13T17:57:48.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.720+0000 7fec12273640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fec04008c50 0x7fec0400c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.720+0000 7fec12273640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec04056570 con 0x7fec04037e20 2024-09-13T17:57:48.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.720+0000 7fec12273640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec040370f0 con 0x7fec04051cc0 2024-09-13T17:57:48.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.720+0000 7fec12273640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec040e6550 con 0x7fec04008c50 2024-09-13T17:57:48.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7fec0bfff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fec04008c50 0x7fec0400c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7fec0b7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fec04037e20 0x7fec040515b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49686/0 (socket says 172.21.15.111:49686) 2024-09-13T17:57:48.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7fec0b7fe640 1 -- 172.21.15.111:0/3838228075 learned_addr learned my addr 172.21.15.111:0/3838228075 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:48.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7ff948b8f640 1 --2- 172.21.15.111:0/13454204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9440b4cb0 0x7ff94415a740 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff9340097a0 tx=0x7ff93402c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84e9eee220df0735 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f606fd5e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f606815c760 0x7f606815d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f606fd5e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f60680af5c0 0x7f60680afa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f606fd5e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60680b5020 con 0x7f6068069ed0 2024-09-13T17:57:48.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f606fd5e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60680affb0 con 0x7f60680af5c0 2024-09-13T17:57:48.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f606fd5e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60680cc310 con 0x7f606815c760 2024-09-13T17:57:48.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7ff93bfff640 1 -- 172.21.15.111:0/13454204 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934034640 con 0x7ff9440b4cb0 2024-09-13T17:57:48.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f606dafa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f606815c760 0x7f606815d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f606dafa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f606815c760 0x7f606815d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:34402/0 (socket says 172.21.15.111:34402) 2024-09-13T17:57:48.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f606dafa640 1 -- 172.21.15.111:0/3661879560 learned_addr learned my addr 172.21.15.111:0/3661879560 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:48.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7ff93bfff640 1 -- 172.21.15.111:0/13454204 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9340347a0 con 0x7ff9440b4cb0 2024-09-13T17:57:48.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f4b10fef640 1 -- 172.21.15.111:0/2297403161 >> v1:172.21.15.188:6789/0 conn(0x7f4b0c0cdca0 legacy=0x7f4b0c160520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:48.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f6a8ce6d640 1 -- 172.21.15.111:0/3511215785 >> v1:172.21.15.188:6789/0 conn(0x7f6a880cbea0 legacy=0x7f6a880cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f6a8ce6d640 1 -- 172.21.15.111:0/3511215785 >> v1:172.21.15.111:6789/0 conn(0x7f6a88158ae0 legacy=0x7f6a880ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f4b10fef640 1 -- 172.21.15.111:0/2297403161 >> v1:172.21.15.111:6789/0 conn(0x7f4b0c0cd860 legacy=0x7f4b0c15df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:48.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.721+0000 7f4b10fef640 1 -- 172.21.15.111:0/2297403161 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c0d18c0 con 0x7f4b0c160140 2024-09-13T17:57:48.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f6a86ffd640 1 -- 172.21.15.111:0/3511215785 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384490175 0 0) 0x7f6a880ada80 con 0x7f6a88158ae0 2024-09-13T17:57:48.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f59ec145640 1 Processor -- start 2024-09-13T17:57:48.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7ff93bfff640 1 -- 172.21.15.111:0/13454204 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934034640 con 0x7ff9440b4cb0 2024-09-13T17:57:48.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f4b10fef640 1 --2- 172.21.15.111:0/2297403161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b0c160140 0x7f4b0c0d11c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4b000098a0 tx=0x7f4b0002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fec6e6736d84969 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f4b0b7fe640 1 -- 172.21.15.111:0/2297403161 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00032440 con 0x7f4b0c160140 2024-09-13T17:57:48.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7f4b0b7fe640 1 -- 172.21.15.111:0/2297403161 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b000325a0 con 0x7f4b0c160140 2024-09-13T17:57:48.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7f4b0b7fe640 1 -- 172.21.15.111:0/2297403161 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00032930 con 0x7f4b0c160140 2024-09-13T17:57:48.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f5f0b7fe640 1 -- 172.21.15.111:0/2790084556 >> v2:172.21.15.111:3300/0 conn(0x7f5f040e70e0 msgr2=0x7f5f04005e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:48.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0affd640 1 -- 172.21.15.111:0/3838228075 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373548089 0 0) 0x7fec040370f0 con 0x7fec04051cc0 2024-09-13T17:57:48.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0affd640 1 -- 172.21.15.111:0/3838228075 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febfc003520 con 0x7fec04051cc0 2024-09-13T17:57:48.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0affd640 1 -- 172.21.15.111:0/3838228075 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854642257 0 0) 0x7febf8001110 con 0x7fec04037e20 2024-09-13T17:57:48.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0affd640 1 -- 172.21.15.111:0/3838228075 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec040370f0 con 0x7fec04037e20 2024-09-13T17:57:48.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f5f0b7fe640 1 --2- 172.21.15.111:0/2790084556 >> v2:172.21.15.111:3300/0 conn(0x7f5f040e70e0 0x7f5f04005e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f5f0b7fe640 1 -- 172.21.15.111:0/2790084556 >> v1:172.21.15.111:6789/0 conn(0x7f5f0404eb20 legacy=0x7f5f0401c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:48.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f5f0b7fe640 1 -- 172.21.15.111:0/2790084556 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f04006c40 con 0x7f5f04005a20 2024-09-13T17:57:48.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fb5948ef640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fb590159850 0x7fb5900b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:45668/0 (socket says 172.21.15.111:45668) 2024-09-13T17:57:48.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fb596b53640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb5900b3490 0x7fb590069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7f5f0b7fe640 1 --2- 172.21.15.111:0/2790084556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f04005a20 0x7f5f04053f10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c04efc0 tx=0x7f5f0c0a4140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2e1cab1e000c432 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fb596b53640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5900b2980 con 0x7fb590142dd0 2024-09-13T17:57:48.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fb596b53640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5900acc00 con 0x7fb590159850 2024-09-13T17:57:48.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fb596b53640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5900ad880 con 0x7fb5900b3490 2024-09-13T17:57:48.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f6a8ce6d640 1 -- 172.21.15.111:0/3511215785 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8806d400 con 0x7f6a880b0980 2024-09-13T17:57:48.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fb5948ef640 1 -- 172.21.15.111:0/68264163 learned_addr learned my addr 172.21.15.111:0/68264163 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:48.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7ff94b5f4640 1 -- 172.21.15.111:0/13454204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9440b4cb0 msgr2=0x7ff94415a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0affd640 1 -- 172.21.15.111:0/3838228075 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3104245302 0 0) 0x7fec040370f0 con 0x7fec04037e20 2024-09-13T17:57:48.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0affd640 1 -- 172.21.15.111:0/3838228075 >> v2:172.21.15.111:3300/0 conn(0x7fec04008c50 msgr2=0x7fec0400c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:48.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7f59ec145640 1 -- start start 2024-09-13T17:57:48.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0affd640 1 --2- 172.21.15.111:0/3838228075 >> v2:172.21.15.111:3300/0 conn(0x7fec04008c50 0x7fec0400c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0affd640 1 -- 172.21.15.111:0/3838228075 >> v1:172.21.15.188:6789/0 conn(0x7fec04051cc0 legacy=0x7fec0400ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0affd640 1 -- 172.21.15.111:0/3838228075 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec04009030 con 0x7fec04037e20 2024-09-13T17:57:48.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0affd640 1 -- 172.21.15.111:0/3838228075 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7febf8001780 con 0x7fec04037e20 2024-09-13T17:57:48.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0affd640 1 -- 172.21.15.111:0/3838228075 >> v1:172.21.15.111:6789/0 conn(0x7fec04037e20 legacy=0x7fec040515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0affd640 1 --2- 172.21.15.111:0/3838228075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febfc004b90 0x7febfc004f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0affd640 1 --2- 172.21.15.111:0/3838228075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7febfc003760 0x7febfc003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0affd640 1 --2- 172.21.15.111:0/3838228075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febfc004120 0x7febfc014190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0affd640 1 -- 172.21.15.111:0/3838228075 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c04e730 con 0x7febfc003760 2024-09-13T17:57:48.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0affd640 1 -- 172.21.15.111:0/3838228075 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec040e6550 con 0x7febfc004120 2024-09-13T17:57:48.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0affd640 1 -- 172.21.15.111:0/3838228075 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febfc014800 con 0x7febfc004b90 2024-09-13T17:57:48.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7f5f0a7fc640 1 -- 172.21.15.111:0/2790084556 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c0ac440 con 0x7f5f04005a20 2024-09-13T17:57:48.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7ff94b5f4640 1 --2- 172.21.15.111:0/13454204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9440b4cb0 0x7ff94415a740 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff9340097a0 tx=0x7ff93402c310 comp rx=0 tx=0).stop 2024-09-13T17:57:48.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec0bfff640 1 --2- 172.21.15.111:0/3838228075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febfc004b90 0x7febfc004f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7f6a8ce6d640 1 --2- 172.21.15.111:0/3511215785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a880b0980 0x7f6a8806dee0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009d90 tx=0x7f6a8002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f14d72e66232f3c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fec10810640 1 --2- 172.21.15.111:0/3838228075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febfc004120 0x7febfc014190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.722+0000 7f606e2fb640 1 --2- 172.21.15.111:0/3661879560 >> v2:172.21.15.111:3300/0 conn(0x7f60680af5c0 0x7f60680afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7f606dafa640 1 -- 172.21.15.111:0/3661879560 >> v2:172.21.15.111:3300/0 conn(0x7f60680af5c0 msgr2=0x7f60680afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:48.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7f606dafa640 1 --2- 172.21.15.111:0/3661879560 >> v2:172.21.15.111:3300/0 conn(0x7f60680af5c0 0x7f60680afa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7f606dafa640 1 -- 172.21.15.111:0/3661879560 >> v1:172.21.15.111:6789/0 conn(0x7f6068069ed0 legacy=0x7f60680cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:48.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7f606dafa640 1 -- 172.21.15.111:0/3661879560 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606815db60 con 0x7f606815c760 2024-09-13T17:57:48.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7f4b13253640 1 -- 172.21.15.111:0/2297403161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b0c160140 msgr2=0x7f4b0c0d11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7f4b13253640 1 --2- 172.21.15.111:0/2297403161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b0c160140 0x7f4b0c0d11c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4b000098a0 tx=0x7f4b0002a250 comp rx=0 tx=0).stop 2024-09-13T17:57:48.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7f4b13253640 1 -- 172.21.15.111:0/2297403161 shutdown_connections 2024-09-13T17:57:48.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7f4b13253640 1 --2- 172.21.15.111:0/2297403161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b0c160140 0x7f4b0c0d11c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7f4b13253640 1 -- 172.21.15.111:0/2297403161 >> 172.21.15.111:0/2297403161 conn(0x7f4b0c089f60 msgr2=0x7f4b0c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7f4b13253640 1 -- 172.21.15.111:0/2297403161 shutdown_connections 2024-09-13T17:57:48.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fb58f7fe640 1 -- 172.21.15.111:0/68264163 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571102700 0 0) 0x7fb5900acc00 con 0x7fb590159850 2024-09-13T17:57:48.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7fb58f7fe640 1 -- 172.21.15.111:0/68264163 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb580003520 con 0x7fb590159850 2024-09-13T17:57:48.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fb58f7fe640 1 -- 172.21.15.111:0/68264163 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1176663566 0 0) 0x7fb580003520 con 0x7fb590159850 2024-09-13T17:57:48.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fb58f7fe640 1 -- 172.21.15.111:0/68264163 >> v2:172.21.15.111:3300/0 conn(0x7fb5900b3490 msgr2=0x7fb590069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:48.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fb58f7fe640 1 --2- 172.21.15.111:0/68264163 >> v2:172.21.15.111:3300/0 conn(0x7fb5900b3490 0x7fb590069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fb58f7fe640 1 -- 172.21.15.111:0/68264163 >> v1:172.21.15.111:6789/0 conn(0x7fb590142dd0 legacy=0x7fb5900b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:48.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fb58f7fe640 1 -- 172.21.15.111:0/68264163 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5900acee0 con 0x7fb590159850 2024-09-13T17:57:48.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fb58f7fe640 1 -- 172.21.15.111:0/68264163 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7fb57c001670 con 0x7fb590159850 2024-09-13T17:57:48.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fb58f7fe640 1 -- 172.21.15.111:0/68264163 >> v1:172.21.15.188:6789/0 conn(0x7fb590159850 legacy=0x7fb5900b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7f4b13253640 1 -- 172.21.15.111:0/2297403161 wait complete. 2024-09-13T17:57:48.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fb58f7fe640 1 --2- 172.21.15.111:0/68264163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb580003970 0x7fb580003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fb58f7fe640 1 --2- 172.21.15.111:0/68264163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5800047c0 0x7fb580004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fec0bfff640 1 -- 172.21.15.111:0/3838228075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febfc004120 msgr2=0x7febfc014190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:48.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fec0bfff640 1 --2- 172.21.15.111:0/3838228075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febfc004120 0x7febfc014190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fec0bfff640 1 -- 172.21.15.111:0/3838228075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7febfc003760 msgr2=0x7febfc003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:48.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fec0bfff640 1 --2- 172.21.15.111:0/3838228075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7febfc003760 0x7febfc003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fec0bfff640 1 -- 172.21.15.111:0/3838228075 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec04009030 con 0x7febfc004b90 2024-09-13T17:57:48.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fb58f7fe640 1 --2- 172.21.15.111:0/68264163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5800052a0 0x7fb58000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fb58f7fe640 1 -- 172.21.15.111:0/68264163 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5900ad880 con 0x7fb5800047c0 2024-09-13T17:57:48.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fb58f7fe640 1 -- 172.21.15.111:0/68264163 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58000c1a0 con 0x7fb580003970 2024-09-13T17:57:48.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fb58f7fe640 1 -- 172.21.15.111:0/68264163 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58000c360 con 0x7fb5800052a0 2024-09-13T17:57:48.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7f5f0a7fc640 1 -- 172.21.15.111:0/2790084556 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c0ac5a0 con 0x7f5f04005a20 2024-09-13T17:57:48.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.723+0000 7f5f0a7fc640 1 -- 172.21.15.111:0/2790084556 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c0ac9b0 con 0x7f5f04005a20 2024-09-13T17:57:48.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fb58ffff640 1 --2- 172.21.15.111:0/68264163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5800047c0 0x7fb580004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fb5950f0640 1 --2- 172.21.15.111:0/68264163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5800052a0 0x7fb58000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7fb5948ef640 1 --2- 172.21.15.111:0/68264163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb580003970 0x7fb580003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7f6a86ffd640 1 -- 172.21.15.111:0/3511215785 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8002aea0 con 0x7f6a880b0980 2024-09-13T17:57:48.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7f6a86ffd640 1 -- 172.21.15.111:0/3511215785 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8002d050 con 0x7f6a880b0980 2024-09-13T17:57:48.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7f6a86ffd640 1 -- 172.21.15.111:0/3511215785 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800079a0 con 0x7f6a880b0980 2024-09-13T17:57:48.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7ff94b5f4640 1 -- 172.21.15.111:0/13454204 shutdown_connections 2024-09-13T17:57:48.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7ff94b5f4640 1 --2- 172.21.15.111:0/13454204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9440b4cb0 0x7ff94415a740 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.724+0000 7ff94b5f4640 1 -- 172.21.15.111:0/13454204 >> 172.21.15.111:0/13454204 conn(0x7ff9440989d0 msgr2=0x7ff9440b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7ff94b5f4640 1 -- 172.21.15.111:0/13454204 shutdown_connections 2024-09-13T17:57:48.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7ff94b5f4640 1 -- 172.21.15.111:0/13454204 wait complete. 2024-09-13T17:57:48.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f59ec145640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f59e41641b0 0x7f59e41645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f59ec145640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e4164c00 con 0x7f59e41604c0 2024-09-13T17:57:48.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f59ec145640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e4164ee0 con 0x7f59e40ce320 2024-09-13T17:57:48.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f59ec145640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e40b0f70 con 0x7f59e41641b0 2024-09-13T17:57:48.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f5f109a0640 1 -- 172.21.15.111:0/2790084556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f04005a20 msgr2=0x7f5f04053f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f5f109a0640 1 --2- 172.21.15.111:0/2790084556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f04005a20 0x7f5f04053f10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c04efc0 tx=0x7f5f0c0a4140 comp rx=0 tx=0).stop 2024-09-13T17:57:48.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f606dafa640 1 --2- 172.21.15.111:0/3661879560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f606815c760 0x7f606815d620 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f605c009870 tx=0x7f605c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91ebb789cd110d1d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7fec10810640 1 -- 172.21.15.111:0/3838228075 reap_dead start 2024-09-13T17:57:48.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f5f109a0640 1 -- 172.21.15.111:0/2790084556 shutdown_connections 2024-09-13T17:57:48.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f5f109a0640 1 --2- 172.21.15.111:0/2790084556 >> v2:172.21.15.111:3300/0 conn(0x7f5f040e70e0 0x7f5f04005e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f5f109a0640 1 --2- 172.21.15.111:0/2790084556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f04005a20 0x7f5f04053f10 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c04efc0 tx=0x7f5f0c0a4140 comp rx=0 tx=0).stop 2024-09-13T17:57:48.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f5f109a0640 1 -- 172.21.15.111:0/2790084556 >> 172.21.15.111:0/2790084556 conn(0x7f5f04036050 msgr2=0x7f5f04036460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7fb58ffff640 1 -- 172.21.15.111:0/68264163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5800052a0 msgr2=0x7fb58000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:48.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7fb58ffff640 1 --2- 172.21.15.111:0/68264163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5800052a0 0x7fb58000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7fb58ffff640 1 -- 172.21.15.111:0/68264163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb580003970 msgr2=0x7fb580003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:48.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7fb58ffff640 1 --2- 172.21.15.111:0/68264163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb580003970 0x7fb580003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7fb58ffff640 1 -- 172.21.15.111:0/68264163 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5900acee0 con 0x7fb5800047c0 2024-09-13T17:57:48.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7fb5950f0640 1 -- 172.21.15.111:0/68264163 reap_dead start 2024-09-13T17:57:48.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7fec0bfff640 1 --2- 172.21.15.111:0/3838228075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febfc004b90 0x7febfc004f70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fec00009b40 tx=0x7febfc0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad53f0960e630c57 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f606caf8640 1 -- 172.21.15.111:0/3661879560 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c02ae10 con 0x7f606815c760 2024-09-13T17:57:48.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f6a8e8d0640 1 -- 172.21.15.111:0/3511215785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a880b0980 msgr2=0x7f6a8806dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f6a8e8d0640 1 --2- 172.21.15.111:0/3511215785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a880b0980 0x7f6a8806dee0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009d90 tx=0x7f6a8002a680 comp rx=0 tx=0).stop 2024-09-13T17:57:48.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f59e9ee1640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f59e41604c0 0x7f59e40b1980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49738/0 (socket says 172.21.15.111:49738) 2024-09-13T17:57:48.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f59e9ee1640 1 -- 172.21.15.111:0/2856323434 learned_addr learned my addr 172.21.15.111:0/2856323434 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:48.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7fec0affd640 1 -- 172.21.15.111:0/3838228075 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec00031400 con 0x7febfc004b90 2024-09-13T17:57:48.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7fec0affd640 1 -- 172.21.15.111:0/3838228075 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0002d050 con 0x7febfc004b90 2024-09-13T17:57:48.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7fec0affd640 1 -- 172.21.15.111:0/3838228075 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec00031780 con 0x7febfc004b90 2024-09-13T17:57:48.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f6a8e8d0640 1 -- 172.21.15.111:0/3511215785 shutdown_connections 2024-09-13T17:57:48.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f6a8e8d0640 1 --2- 172.21.15.111:0/3511215785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a880b0980 0x7f6a8806dee0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f6a8e8d0640 1 -- 172.21.15.111:0/3511215785 >> 172.21.15.111:0/3511215785 conn(0x7f6a880afe60 msgr2=0x7f6a880b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7f4b13253640 1 Processor -- start 2024-09-13T17:57:48.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7f4b13253640 1 -- start start 2024-09-13T17:57:48.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f59e96e0640 1 --2- 172.21.15.111:0/2856323434 >> v2:172.21.15.188:3300/0 conn(0x7f59e41641b0 0x7f59e41645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.725+0000 7f606caf8640 1 -- 172.21.15.111:0/3661879560 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605c032440 con 0x7f606815c760 2024-09-13T17:57:48.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7f5f109a0640 1 -- 172.21.15.111:0/2790084556 shutdown_connections 2024-09-13T17:57:48.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7fb58ffff640 1 --2- 172.21.15.111:0/68264163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5800047c0 0x7fb580004c40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb584001620 tx=0x7fb5800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de60d18fd33cb838 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7f5f109a0640 1 -- 172.21.15.111:0/2790084556 wait complete. 2024-09-13T17:57:48.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7f606fd5e640 1 -- 172.21.15.111:0/3661879560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f606815c760 msgr2=0x7f606815d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7f606caf8640 1 -- 172.21.15.111:0/3661879560 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c02ae10 con 0x7f606815c760 2024-09-13T17:57:48.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7f606fd5e640 1 --2- 172.21.15.111:0/3661879560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f606815c760 0x7f606815d620 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f605c009870 tx=0x7f605c02a100 comp rx=0 tx=0).stop 2024-09-13T17:57:48.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7fb58f7fe640 1 -- 172.21.15.111:0/68264163 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb584005270 con 0x7fb5800047c0 2024-09-13T17:57:48.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7fb58f7fe640 1 -- 172.21.15.111:0/68264163 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5840053f0 con 0x7fb5800047c0 2024-09-13T17:57:48.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7fb58f7fe640 1 -- 172.21.15.111:0/68264163 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58402a9b0 con 0x7fb5800047c0 2024-09-13T17:57:48.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7f59e96e0640 1 -- 172.21.15.111:0/2856323434 >> v1:172.21.15.188:6789/0 conn(0x7f59e40ce320 legacy=0x7f59e40aaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:48.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7f59e96e0640 1 -- 172.21.15.111:0/2856323434 >> v1:172.21.15.111:6789/0 conn(0x7f59e41604c0 legacy=0x7f59e40b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7f59e96e0640 1 -- 172.21.15.111:0/2856323434 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e40b1180 con 0x7f59e41641b0 2024-09-13T17:57:48.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7fec12273640 1 -- 172.21.15.111:0/3838228075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febfc004b90 msgr2=0x7febfc004f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7fec12273640 1 --2- 172.21.15.111:0/3838228075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febfc004b90 0x7febfc004f70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fec00009b40 tx=0x7febfc0057f0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7fec12273640 1 -- 172.21.15.111:0/3838228075 shutdown_connections 2024-09-13T17:57:48.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7fec12273640 1 --2- 172.21.15.111:0/3838228075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febfc004b90 0x7febfc004f70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7fec12273640 1 -- 172.21.15.111:0/3838228075 >> 172.21.15.111:0/3838228075 conn(0x7fec0401ea00 msgr2=0x7fec04055670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7ff94b5f4640 1 Processor -- start 2024-09-13T17:57:48.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.726+0000 7ff94b5f4640 1 -- start start 2024-09-13T17:57:48.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.727+0000 7f6a8e8d0640 1 -- 172.21.15.111:0/3511215785 shutdown_connections 2024-09-13T17:57:48.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.727+0000 7f606fd5e640 1 -- 172.21.15.111:0/3661879560 shutdown_connections 2024-09-13T17:57:48.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.727+0000 7f606fd5e640 1 --2- 172.21.15.111:0/3661879560 >> v2:172.21.15.111:3300/0 conn(0x7f60680af5c0 0x7f60680afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.727+0000 7f606fd5e640 1 --2- 172.21.15.111:0/3661879560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f606815c760 0x7f606815d620 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.727+0000 7f606fd5e640 1 -- 172.21.15.111:0/3661879560 >> 172.21.15.111:0/3661879560 conn(0x7f60680989d0 msgr2=0x7f60680b45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.727+0000 7f6a8e8d0640 1 -- 172.21.15.111:0/3511215785 wait complete. 2024-09-13T17:57:48.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.727+0000 7fec12273640 1 -- 172.21.15.111:0/3838228075 shutdown_connections 2024-09-13T17:57:48.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.727+0000 7fec12273640 1 -- 172.21.15.111:0/3838228075 wait complete. 2024-09-13T17:57:48.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.727+0000 7f4b13253640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b0c0cd860 0x7f4b0c158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.727+0000 7f606fd5e640 1 -- 172.21.15.111:0/3661879560 shutdown_connections 2024-09-13T17:57:48.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.727+0000 7fb596b53640 1 -- 172.21.15.111:0/68264163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5800047c0 msgr2=0x7fb580004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.727+0000 7fb596b53640 1 --2- 172.21.15.111:0/68264163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5800047c0 0x7fb580004c40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb584001620 tx=0x7fb5800057f0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.727+0000 7f4b10fef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b0c0cd860 0x7f4b0c158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.727+0000 7f4b10fef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b0c0cd860 0x7f4b0c158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53986/0 (socket says 172.21.15.111:53986) 2024-09-13T17:57:48.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.727+0000 7f4b10fef640 1 -- 172.21.15.111:0/2936972192 learned_addr learned my addr 172.21.15.111:0/2936972192 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:48.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.727+0000 7f59e96e0640 1 --2- 172.21.15.111:0/2856323434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59e41641b0 0x7f59e41645d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59e00097a0 tx=0x7f59e002a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee641cd223b3d31 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7f4b13253640 1 --2- 172.21.15.111:0/2936972192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b0c0cdca0 0x7f4b0c0b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7f5f109a0640 1 Processor -- start 2024-09-13T17:57:48.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7fb596b53640 1 -- 172.21.15.111:0/68264163 shutdown_connections 2024-09-13T17:57:48.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7fb596b53640 1 --2- 172.21.15.111:0/68264163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5800047c0 0x7fb580004c40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7fb596b53640 1 -- 172.21.15.111:0/68264163 >> 172.21.15.111:0/68264163 conn(0x7fb5900989d0 msgr2=0x7fb5900b4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7ff94b5f4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9440b2a20 0x7ff9440ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7f4b13253640 1 --2- 172.21.15.111:0/2936972192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b0c160140 0x7f4b0c0b7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7fb596b53640 1 -- 172.21.15.111:0/68264163 shutdown_connections 2024-09-13T17:57:48.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7f4b13253640 1 -- 172.21.15.111:0/2936972192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c0cb1e0 con 0x7f4b0c0cd860 2024-09-13T17:57:48.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7f4b13253640 1 -- 172.21.15.111:0/2936972192 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c0ca660 con 0x7f4b0c0cdca0 2024-09-13T17:57:48.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7f4b0bfff640 1 --2- 172.21.15.111:0/2936972192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b0c0cdca0 0x7f4b0c0b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7ff949390640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9440b2a20 0x7ff9440ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7ff949390640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9440b2a20 0x7ff9440ae4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53998/0 (socket says 172.21.15.111:53998) 2024-09-13T17:57:48.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7f4b117f0640 1 --2- 172.21.15.111:0/2936972192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b0c160140 0x7f4b0c0b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7f4b13253640 1 -- 172.21.15.111:0/2936972192 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c0ca940 con 0x7f4b0c160140 2024-09-13T17:57:48.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7ff949390640 1 -- 172.21.15.111:0/1319984696 learned_addr learned my addr 172.21.15.111:0/1319984696 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:48.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7ff94b5f4640 1 --2- 172.21.15.111:0/1319984696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9440b4cb0 0x7ff944143e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7ff94b5f4640 1 --2- 172.21.15.111:0/1319984696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff944153660 0x7ff9441571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7fb596b53640 1 -- 172.21.15.111:0/68264163 wait complete. 2024-09-13T17:57:48.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7ff948b8f640 1 --2- 172.21.15.111:0/1319984696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9440b4cb0 0x7ff944143e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.728+0000 7f5f109a0640 1 -- start start 2024-09-13T17:57:48.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7fec12273640 1 Processor -- start 2024-09-13T17:57:48.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7ff94b5f4640 1 -- 172.21.15.111:0/1319984696 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94415a010 con 0x7ff9440b2a20 2024-09-13T17:57:48.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7ff94b5f4640 1 -- 172.21.15.111:0/1319984696 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9441599c0 con 0x7ff944153660 2024-09-13T17:57:48.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7ff94b5f4640 1 -- 172.21.15.111:0/1319984696 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff944157f70 con 0x7ff9440b4cb0 2024-09-13T17:57:48.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7ff949b91640 1 --2- 172.21.15.111:0/1319984696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff944153660 0x7ff9441571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.727+0000 7f606fd5e640 1 -- 172.21.15.111:0/3661879560 wait complete. 2024-09-13T17:57:48.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7f606fd5e640 1 Processor -- start 2024-09-13T17:57:48.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7f6a8e8d0640 1 Processor -- start 2024-09-13T17:57:48.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7fec12273640 1 -- start start 2024-09-13T17:57:48.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7f6a8e8d0640 1 -- start start 2024-09-13T17:57:48.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7f4b10fef640 1 -- 172.21.15.111:0/2936972192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b0c160140 msgr2=0x7f4b0c0b7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7f4b10fef640 1 --2- 172.21.15.111:0/2936972192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b0c160140 0x7f4b0c0b7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7f5f109a0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f0404eb20 0x7f5f040d9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7f5f109a0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f040e70e0 0x7f5f0404d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7ff949390640 1 -- 172.21.15.111:0/1319984696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9440b4cb0 msgr2=0x7ff944143e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7ff949390640 1 --2- 172.21.15.111:0/1319984696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9440b4cb0 0x7ff944143e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7ff949390640 1 -- 172.21.15.111:0/1319984696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff944153660 msgr2=0x7ff9441571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7ff949390640 1 --2- 172.21.15.111:0/1319984696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff944153660 0x7ff9441571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7f4b10fef640 1 -- 172.21.15.111:0/2936972192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b0c0cdca0 msgr2=0x7f4b0c0b6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7fb596b53640 1 Processor -- start 2024-09-13T17:57:48.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f5f0b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f0404eb20 0x7f5f040d9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7f4b10fef640 1 --2- 172.21.15.111:0/2936972192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b0c0cdca0 0x7f4b0c0b6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7f4b10fef640 1 -- 172.21.15.111:0/2936972192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b000094d0 con 0x7f4b0c0cd860 2024-09-13T17:57:48.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f5f0b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f0404eb20 0x7f5f040d9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:49038/0 (socket says 172.21.15.111:49038) 2024-09-13T17:57:48.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f5f109a0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f0404d870 0x7f5f040e46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7fb596b53640 1 -- start start 2024-09-13T17:57:48.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f5f0affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f040e70e0 0x7f5f0404d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f5f0affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f040e70e0 0x7f5f0404d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54010/0 (socket says 172.21.15.111:54010) 2024-09-13T17:57:48.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f4b117f0640 1 --2- 172.21.15.111:0/2936972192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b0c160140 0x7f4b0c0b7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:48.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f5f0b7fe640 1 -- 172.21.15.111:0/1285331186 learned_addr learned my addr 172.21.15.111:0/1285331186 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:48.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7ff949390640 1 -- 172.21.15.111:0/1319984696 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff934009400 con 0x7ff9440b2a20 2024-09-13T17:57:48.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f5f109a0640 1 -- 172.21.15.111:0/1285331186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f040e8610 con 0x7f5f040e70e0 2024-09-13T17:57:48.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f5f109a0640 1 -- 172.21.15.111:0/1285331186 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f040e79b0 con 0x7f5f0404eb20 2024-09-13T17:57:48.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f5f109a0640 1 -- 172.21.15.111:0/1285331186 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f040e7c70 con 0x7f5f0404d870 2024-09-13T17:57:48.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7ff949b91640 1 --2- 172.21.15.111:0/1319984696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff944153660 0x7ff9441571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:48.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f5f0bfff640 1 --2- 172.21.15.111:0/1285331186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f0404d870 0x7f5f040e46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f59e8edf640 1 -- 172.21.15.111:0/2856323434 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e0032440 con 0x7f59e41641b0 2024-09-13T17:57:48.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.729+0000 7f606fd5e640 1 -- start start 2024-09-13T17:57:48.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f606fd5e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6068069ed0 0x7f606815a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f606fd5e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60680af5c0 0x7f60681560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f606fd5e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f606815c760 0x7f6068156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f606fd5e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606815a980 con 0x7f606815c760 2024-09-13T17:57:48.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f606fd5e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60680acbf0 con 0x7f6068069ed0 2024-09-13T17:57:48.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f606fd5e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60680aced0 con 0x7f60680af5c0 2024-09-13T17:57:48.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7fec12273640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febfc004b90 0x7fec040d7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7ff949390640 1 --2- 172.21.15.111:0/1319984696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9440b2a20 0x7ff9440ae4c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00caf0 tx=0x7ff93c002100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f4b10fef640 1 --2- 172.21.15.111:0/2936972192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b0c0cd860 0x7f4b0c158170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4b00000c00 tx=0x7f4b000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7fec12273640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec040df8d0 0x7fec040dfd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f6a8e8d0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a880b0980 0x7f6a880c10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7fec0bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febfc004b90 0x7fec040d7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f4b097fa640 1 -- 172.21.15.111:0/2936972192 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00004230 con 0x7f4b0c0cd860 2024-09-13T17:57:48.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7fec0bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febfc004b90 0x7fec040d7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:34458/0 (socket says 172.21.15.111:34458) 2024-09-13T17:57:48.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7fec0bfff640 1 -- 172.21.15.111:0/4250213537 learned_addr learned my addr 172.21.15.111:0/4250213537 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:48.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f6a8e8d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a880cbea0 0x7f6a88157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f6a8e8d0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a88158ae0 0x7f6a880cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f6a8e8d0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a88097e80 con 0x7f6a88158ae0 2024-09-13T17:57:48.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f6a8e8d0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a880c97b0 con 0x7f6a880b0980 2024-09-13T17:57:48.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f6a8e8d0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a881586b0 con 0x7f6a880cbea0 2024-09-13T17:57:48.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7fec12273640 1 --2- 172.21.15.111:0/4250213537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec040e0290 0x7fec04006730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f4b097fa640 1 -- 172.21.15.111:0/2936972192 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b00004390 con 0x7f4b0c0cd860 2024-09-13T17:57:48.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f4b13253640 1 -- 172.21.15.111:0/2936972192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0c0cac20 con 0x7f4b0c0cd860 2024-09-13T17:57:48.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f5f0b7fe640 1 -- 172.21.15.111:0/1285331186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f0404d870 msgr2=0x7f5f040e46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f5f0b7fe640 1 --2- 172.21.15.111:0/1285331186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f0404d870 0x7f5f040e46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f5f0b7fe640 1 -- 172.21.15.111:0/1285331186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f040e70e0 msgr2=0x7f5f0404d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7fec0b7fe640 1 --2- 172.21.15.111:0/4250213537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec040df8d0 0x7fec040dfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7ff939ffb640 1 -- 172.21.15.111:0/1319984696 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c003b70 con 0x7ff9440b2a20 2024-09-13T17:57:48.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f4b13253640 1 -- 172.21.15.111:0/2936972192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0c1586b0 con 0x7f4b0c0cd860 2024-09-13T17:57:48.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f5f0b7fe640 1 --2- 172.21.15.111:0/1285331186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f040e70e0 0x7f5f0404d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f5f0b7fe640 1 -- 172.21.15.111:0/1285331186 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c072900 con 0x7f5f0404eb20 2024-09-13T17:57:48.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7ff939ffb640 1 -- 172.21.15.111:0/1319984696 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93c003cd0 con 0x7ff9440b2a20 2024-09-13T17:57:48.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f6a8ce6d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a88158ae0 0x7f6a880cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7ff939ffb640 1 -- 172.21.15.111:0/1319984696 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c003b70 con 0x7ff9440b2a20 2024-09-13T17:57:48.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f4b097fa640 1 -- 172.21.15.111:0/2936972192 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00030a50 con 0x7f4b0c0cd860 2024-09-13T17:57:48.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f5f0affd640 1 --2- 172.21.15.111:0/1285331186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f040e70e0 0x7f5f0404d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:48.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f606dafa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6068069ed0 0x7f606815a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.730+0000 7f59e8edf640 1 -- 172.21.15.111:0/2856323434 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e00325a0 con 0x7f59e41641b0 2024-09-13T17:57:48.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f59ec145640 1 -- 172.21.15.111:0/2856323434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59e41641b0 msgr2=0x7f59e41645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f6a87fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a880b0980 0x7f6a880c10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f59e8edf640 1 -- 172.21.15.111:0/2856323434 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e0032440 con 0x7f59e41641b0 2024-09-13T17:57:48.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f5f0b7fe640 1 --2- 172.21.15.111:0/1285331186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f0404eb20 0x7f5f040d9b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c02c970 tx=0x7f5f0c069bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7fb596b53640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5800047c0 0x7fb59009af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f59ec145640 1 --2- 172.21.15.111:0/2856323434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59e41641b0 0x7f59e41645d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59e00097a0 tx=0x7f59e002a1e0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f6a877fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a880cbea0 0x7f6a88157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f606e2fb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f606815c760 0x7f6068156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7fec12273640 1 -- 172.21.15.111:0/4250213537 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec04033c10 con 0x7fec040e0290 2024-09-13T17:57:48.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7fec12273640 1 -- 172.21.15.111:0/4250213537 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec040e0740 con 0x7fec040df8d0 2024-09-13T17:57:48.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7fec12273640 1 -- 172.21.15.111:0/4250213537 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec04033090 con 0x7febfc004b90 2024-09-13T17:57:48.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7fec10810640 1 --2- 172.21.15.111:0/4250213537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec040e0290 0x7fec04006730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f59ec145640 1 -- 172.21.15.111:0/2856323434 shutdown_connections 2024-09-13T17:57:48.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f59ec145640 1 --2- 172.21.15.111:0/2856323434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59e41641b0 0x7f59e41645d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f59ec145640 1 -- 172.21.15.111:0/2856323434 >> 172.21.15.111:0/2856323434 conn(0x7f59e406a2f0 msgr2=0x7f59e406a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7f59ec145640 1 -- 172.21.15.111:0/2856323434 shutdown_connections 2024-09-13T17:57:48.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7f59ec145640 1 -- 172.21.15.111:0/2856323434 wait complete. 2024-09-13T17:57:48.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f6a87fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a880b0980 0x7f6a880c10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:49050/0 (socket says 172.21.15.111:49050) 2024-09-13T17:57:48.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f6a87fff640 1 -- 172.21.15.111:0/2120811019 learned_addr learned my addr 172.21.15.111:0/2120811019 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:48.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f6a87fff640 1 -- 172.21.15.111:0/2120811019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a880cbea0 msgr2=0x7f6a88157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f6a87fff640 1 --2- 172.21.15.111:0/2120811019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a880cbea0 0x7f6a88157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f6a87fff640 1 -- 172.21.15.111:0/2120811019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a88158ae0 msgr2=0x7f6a880cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f606e2fb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f606815c760 0x7f6068156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54016/0 (socket says 172.21.15.111:54016) 2024-09-13T17:57:48.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f606e2fb640 1 -- 172.21.15.111:0/3186799715 learned_addr learned my addr 172.21.15.111:0/3186799715 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:48.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.731+0000 7f606d2f9640 1 --2- 172.21.15.111:0/3186799715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60680af5c0 0x7f60681560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7f606e2fb640 1 -- 172.21.15.111:0/3186799715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60680af5c0 msgr2=0x7f60681560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7f606e2fb640 1 --2- 172.21.15.111:0/3186799715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60680af5c0 0x7f60681560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7f6a87fff640 1 --2- 172.21.15.111:0/2120811019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a88158ae0 0x7f6a880cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7f606e2fb640 1 -- 172.21.15.111:0/3186799715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6068069ed0 msgr2=0x7f606815a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7f6a87fff640 1 -- 172.21.15.111:0/2120811019 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a80009a70 con 0x7f6a880b0980 2024-09-13T17:57:48.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7f606e2fb640 1 --2- 172.21.15.111:0/3186799715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6068069ed0 0x7f606815a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7f606e2fb640 1 -- 172.21.15.111:0/3186799715 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c0094d0 con 0x7f606815c760 2024-09-13T17:57:48.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7f6a8ce6d640 1 --2- 172.21.15.111:0/2120811019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a88158ae0 0x7f6a880cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:48.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7f5ef7fff640 1 -- 172.21.15.111:0/1285331186 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c0b6410 con 0x7f5f0404eb20 2024-09-13T17:57:48.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7f606d2f9640 1 --2- 172.21.15.111:0/3186799715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60680af5c0 0x7f60681560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:48.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7f606dafa640 1 --2- 172.21.15.111:0/3186799715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6068069ed0 0x7f606815a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:48.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7fec0bfff640 1 -- 172.21.15.111:0/4250213537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec040df8d0 msgr2=0x7fec040dfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7fec0bfff640 1 --2- 172.21.15.111:0/4250213537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec040df8d0 0x7fec040dfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7fec0bfff640 1 -- 172.21.15.111:0/4250213537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec040e0290 msgr2=0x7fec04006730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7fec0bfff640 1 --2- 172.21.15.111:0/4250213537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec040e0290 0x7fec04006730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7fec0bfff640 1 -- 172.21.15.111:0/4250213537 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec04033370 con 0x7febfc004b90 2024-09-13T17:57:48.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7ff94b5f4640 1 -- 172.21.15.111:0/1319984696 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff944159c30 con 0x7ff9440b2a20 2024-09-13T17:57:48.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7f6a87fff640 1 --2- 172.21.15.111:0/2120811019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a880b0980 0x7f6a880c10e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00c450 tx=0x7f6a7c00c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7f5ef7fff640 1 -- 172.21.15.111:0/1285331186 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c0b6570 con 0x7f5f0404eb20 2024-09-13T17:57:48.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7f5f109a0640 1 -- 172.21.15.111:0/1285331186 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f040e7f50 con 0x7f5f0404eb20 2024-09-13T17:57:48.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.733+0000 7f5f109a0640 1 -- 172.21.15.111:0/1285331186 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0400bc30 con 0x7f5f0404eb20 2024-09-13T17:57:48.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.733+0000 7f6a84ff9640 1 -- 172.21.15.111:0/2120811019 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c0052d0 con 0x7f6a880b0980 2024-09-13T17:57:48.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7fb596b53640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb59009cd10 0x7fb5901589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.733+0000 7fec0bfff640 1 --2- 172.21.15.111:0/4250213537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febfc004b90 0x7fec040d7b30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fec00000c00 tx=0x7fec00004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.733+0000 7f6a84ff9640 1 -- 172.21.15.111:0/2120811019 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7c005430 con 0x7f6a880b0980 2024-09-13T17:57:48.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7f606e2fb640 1 --2- 172.21.15.111:0/3186799715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f606815c760 0x7f6068156600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f606000d800 tx=0x7f606000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.733+0000 7ff94b5f4640 1 -- 172.21.15.111:0/1319984696 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff944158250 con 0x7ff9440b2a20 2024-09-13T17:57:48.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7fb596b53640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb59006e100 0x7fb59006e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7fb596b53640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5900cfd50 con 0x7fb5800047c0 2024-09-13T17:57:48.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.733+0000 7f4b097fa640 1 -- 172.21.15.111:0/2936972192 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4b00039040 con 0x7f4b0c0cd860 2024-09-13T17:57:48.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7fb596b53640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59009b440 con 0x7fb59009cd10 2024-09-13T17:57:48.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7fb596b53640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59006eac0 con 0x7fb59006e100 2024-09-13T17:57:48.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.733+0000 7f4b13253640 1 -- 172.21.15.111:0/2936972192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4b0c0cd340 con 0x7f4b0c0cd860 2024-09-13T17:57:48.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7fb5948ef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5800047c0 0x7fb59009af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7fb5948ef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5800047c0 0x7fb59009af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54032/0 (socket says 172.21.15.111:54032) 2024-09-13T17:57:48.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.732+0000 7fb5948ef640 1 -- 172.21.15.111:0/2410301176 learned_addr learned my addr 172.21.15.111:0/2410301176 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:48.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.733+0000 7fec08ff9640 1 -- 172.21.15.111:0/4250213537 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec00003e10 con 0x7febfc004b90 2024-09-13T17:57:48.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.733+0000 7fec08ff9640 1 -- 172.21.15.111:0/4250213537 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0003b410 con 0x7febfc004b90 2024-09-13T17:57:48.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.733+0000 7f59ec145640 1 Processor -- start 2024-09-13T17:57:48.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.733+0000 7f4b097fa640 1 --2- 172.21.15.111:0/2936972192 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4af006acd0 0x7f4af006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.733+0000 7fb5948ef640 1 -- 172.21.15.111:0/2410301176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb59006e100 msgr2=0x7fb59006e580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:57:48.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7ff9277fe640 1 -- 172.21.15.111:0/1319984696 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff94406dd60 con 0x7ff9440b2a20 2024-09-13T17:57:48.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7f4b097fa640 1 -- 172.21.15.111:0/2936972192 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4b000a9af0 con 0x7f4b0c0cd860 2024-09-13T17:57:48.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7f4b097fa640 1 --2- 172.21.15.111:0/2936972192 >> [v2:172.21.15.111:6808/2899699337,v1:172.21.15.111:6809/2899699337] conn(0x7f4af0074290 0x7f4af00766d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.735+0000 7f4b0bfff640 1 --2- 172.21.15.111:0/2936972192 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4af006acd0 0x7f4af006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.735+0000 7f5ef7fff640 1 -- 172.21.15.111:0/1285331186 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c0b68e0 con 0x7f5f0404eb20 2024-09-13T17:57:48.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.735+0000 7f5ef7fff640 1 -- 172.21.15.111:0/1285331186 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5f0c0b3050 con 0x7f5f0404eb20 2024-09-13T17:57:48.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7f60667fc640 1 -- 172.21.15.111:0/3186799715 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606000b6c0 con 0x7f606815c760 2024-09-13T17:57:48.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7f606fd5e640 1 -- 172.21.15.111:0/3186799715 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60680ad1b0 con 0x7f606815c760 2024-09-13T17:57:48.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7f6a84ff9640 1 -- 172.21.15.111:0/2120811019 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c012870 con 0x7f6a880b0980 2024-09-13T17:57:48.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7f6a8e8d0640 1 -- 172.21.15.111:0/2120811019 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a88157dd0 con 0x7f6a880b0980 2024-09-13T17:57:48.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7f6a8e8d0640 1 -- 172.21.15.111:0/2120811019 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a880c9d50 con 0x7f6a880b0980 2024-09-13T17:57:48.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7f606fd5e640 1 -- 172.21.15.111:0/3186799715 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60680ad770 con 0x7f606815c760 2024-09-13T17:57:48.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7f60667fc640 1 -- 172.21.15.111:0/3186799715 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606000b840 con 0x7f606815c760 2024-09-13T17:57:48.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.735+0000 7f4b117f0640 1 --2- 172.21.15.111:0/2936972192 >> [v2:172.21.15.111:6808/2899699337,v1:172.21.15.111:6809/2899699337] conn(0x7f4af0074290 0x7f4af00766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.735+0000 7f60667fc640 1 -- 172.21.15.111:0/3186799715 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606000bbf0 con 0x7f606815c760 2024-09-13T17:57:48.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.735+0000 7f606fd5e640 1 -- 172.21.15.111:0/3186799715 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6048000fc0 con 0x7f606815c760 2024-09-13T17:57:48.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.736+0000 7f5ef7fff640 1 --2- 172.21.15.111:0/1285331186 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5ef006ad80 0x7f5ef006d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7fec08ff9640 1 -- 172.21.15.111:0/4250213537 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0003b7c0 con 0x7febfc004b90 2024-09-13T17:57:48.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7fec12273640 1 -- 172.21.15.111:0/4250213537 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec04033650 con 0x7febfc004b90 2024-09-13T17:57:48.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7fec12273640 1 -- 172.21.15.111:0/4250213537 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec040d8070 con 0x7febfc004b90 2024-09-13T17:57:48.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.735+0000 7febf67fc640 1 -- 172.21.15.111:0/4250213537 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec04037620 con 0x7febfc004b90 2024-09-13T17:57:48.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.736+0000 7f6a6e7fc640 1 -- 172.21.15.111:0/2120811019 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6a8806d9c0 con 0x7f6a880b0980 2024-09-13T17:57:48.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7f59ec145640 1 -- start start 2024-09-13T17:57:48.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.735+0000 7f59ec145640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59e40ce320 0x7f59e415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.735+0000 7f59ec145640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e41604c0 0x7f59e4099960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.735+0000 7f59ec145640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59e41641b0 0x7f59e40afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.735+0000 7f59ec145640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e40ae020 con 0x7f59e41604c0 2024-09-13T17:57:48.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.735+0000 7f59ec145640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e415f480 con 0x7f59e41641b0 2024-09-13T17:57:48.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.735+0000 7f59ec145640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e40adc80 con 0x7f59e40ce320 2024-09-13T17:57:48.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.736+0000 7f5ef7fff640 1 -- 172.21.15.111:0/1285331186 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5f0c123990 con 0x7f5f0404eb20 2024-09-13T17:57:48.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.736+0000 7f4b117f0640 1 --2- 172.21.15.111:0/2936972192 >> [v2:172.21.15.111:6808/2899699337,v1:172.21.15.111:6809/2899699337] conn(0x7f4af0074290 0x7f4af00766d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.736+0000 7f4b097fa640 1 -- 172.21.15.111:0/2936972192 --> [v2:172.21.15.111:6808/2899699337,v1:172.21.15.111:6809/2899699337] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4af0076da0 con 0x7f4af0074290 2024-09-13T17:57:48.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.733+0000 7fb5948ef640 1 --2- 172.21.15.111:0/2410301176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb59006e100 0x7fb59006e580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.733+0000 7fb5948ef640 1 -- 172.21.15.111:0/2410301176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb59009cd10 msgr2=0x7fb5901589e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:57:48.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.733+0000 7fb5948ef640 1 --2- 172.21.15.111:0/2410301176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb59009cd10 0x7fb5901589e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.733+0000 7fb5948ef640 1 -- 172.21.15.111:0/2410301176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59009c260 con 0x7fb5800047c0 2024-09-13T17:57:48.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.736+0000 7f59e9ee1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59e40ce320 0x7f59e415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.736+0000 7f59ea6e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59e41641b0 0x7f59e40afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.736+0000 7f59e9ee1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59e40ce320 0x7f59e415ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:34476/0 (socket says 172.21.15.111:34476) 2024-09-13T17:57:48.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.736+0000 7f59e96e0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e41604c0 0x7f59e4099960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.736+0000 7f59e9ee1640 1 -- 172.21.15.111:0/2425761205 learned_addr learned my addr 172.21.15.111:0/2425761205 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:48.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.733+0000 7fb5948ef640 1 --2- 172.21.15.111:0/2410301176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5800047c0 0x7fb59009af00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00a2e0 tx=0x7fb57c00a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7fb58d7fa640 1 -- 172.21.15.111:0/2410301176 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c019070 con 0x7fb5800047c0 2024-09-13T17:57:48.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.736+0000 7f4b0bfff640 1 --2- 172.21.15.111:0/2936972192 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4af006acd0 0x7f4af006d190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0cc260 tx=0x7f4af8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.737+0000 7f59e9ee1640 1 -- 172.21.15.111:0/2425761205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59e41641b0 msgr2=0x7f59e40afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.737+0000 7f59e9ee1640 1 --2- 172.21.15.111:0/2425761205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59e41641b0 0x7f59e40afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.737+0000 7f59e9ee1640 1 -- 172.21.15.111:0/2425761205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e41604c0 msgr2=0x7f59e4099960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.737+0000 7f59e9ee1640 1 --2- 172.21.15.111:0/2425761205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e41604c0 0x7f59e4099960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.737+0000 7f59e9ee1640 1 -- 172.21.15.111:0/2425761205 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e0009400 con 0x7f59e40ce320 2024-09-13T17:57:48.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.737+0000 7f5f0affd640 1 --2- 172.21.15.111:0/1285331186 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5ef006ad80 0x7f5ef006d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.737+0000 7fec08ff9640 1 -- 172.21.15.111:0/4250213537 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fec00038070 con 0x7febfc004b90 2024-09-13T17:57:48.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.737+0000 7f59e9ee1640 1 --2- 172.21.15.111:0/2425761205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59e40ce320 0x7f59e415ef40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f59d400c4a0 tx=0x7f59d400c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.737+0000 7f4b097fa640 1 -- 172.21.15.111:0/2936972192 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b00074a70 con 0x7f4b0c0cd860 2024-09-13T17:57:48.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.737+0000 7f4b097fa640 1 -- 172.21.15.111:0/2936972192 <== osd.1 v2:172.21.15.111:6808/2899699337 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f4af0076da0 con 0x7f4af0074290 2024-09-13T17:57:48.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7fb58d7fa640 1 -- 172.21.15.111:0/2410301176 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57c013af0 con 0x7fb5800047c0 2024-09-13T17:57:48.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7fb58d7fa640 1 -- 172.21.15.111:0/2410301176 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c019070 con 0x7fb5800047c0 2024-09-13T17:57:48.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.734+0000 7fb596b53640 1 -- 172.21.15.111:0/2410301176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59009b650 con 0x7fb5800047c0 2024-09-13T17:57:48.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.736+0000 7fb58d7fa640 1 -- 172.21.15.111:0/2410301176 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb57c013c60 con 0x7fb5800047c0 2024-09-13T17:57:48.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.736+0000 7fb596b53640 1 -- 172.21.15.111:0/2410301176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59009bc10 con 0x7fb5800047c0 2024-09-13T17:57:48.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.737+0000 7f59da7fc640 1 -- 172.21.15.111:0/2425761205 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4005ce0 con 0x7f59e40ce320 2024-09-13T17:57:48.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.737+0000 7f59da7fc640 1 -- 172.21.15.111:0/2425761205 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d4005e40 con 0x7f59e40ce320 2024-09-13T17:57:48.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.737+0000 7fec08ff9640 1 --2- 172.21.15.111:0/4250213537 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7febf006ac50 0x7febf006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.737+0000 7f59da7fc640 1 -- 172.21.15.111:0/2425761205 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4007270 con 0x7f59e40ce320 2024-09-13T17:57:48.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.737+0000 7f5f0affd640 1 --2- 172.21.15.111:0/1285331186 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5ef006ad80 0x7f5ef006d240 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5f0403a5f0 tx=0x7f5f00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.737+0000 7f59ec145640 1 -- 172.21.15.111:0/2425761205 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e409a320 con 0x7f59e40ce320 2024-09-13T17:57:48.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.737+0000 7f59ec145640 1 -- 172.21.15.111:0/2425761205 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e409a8e0 con 0x7f59e40ce320 2024-09-13T17:57:48.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.738+0000 7fec08ff9640 1 -- 172.21.15.111:0/4250213537 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fec000a9d70 con 0x7febfc004b90 2024-09-13T17:57:48.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.738+0000 7fec0b7fe640 1 --2- 172.21.15.111:0/4250213537 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7febf006ac50 0x7febf006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.738+0000 7f6a84ff9640 1 -- 172.21.15.111:0/2120811019 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6a7c0129d0 con 0x7f6a880b0980 2024-09-13T17:57:48.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.738+0000 7f5f109a0640 1 --2- 172.21.15.111:0/1285331186 >> [v2:172.21.15.111:6800/2426719794,v1:172.21.15.111:6801/2426719794] conn(0x7f5f04031860 0x7f5f04009ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.738+0000 7f5f109a0640 1 -- 172.21.15.111:0/1285331186 --> [v2:172.21.15.111:6800/2426719794,v1:172.21.15.111:6801/2426719794] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5f04007200 con 0x7f5f04031860 2024-09-13T17:57:48.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.738+0000 7fec08ff9640 1 --2- 172.21.15.111:0/4250213537 >> [v2:172.21.15.188:6808/3407929299,v1:172.21.15.188:6809/3407929299] conn(0x7febf0074290 0x7febf00766d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.738+0000 7fec08ff9640 1 -- 172.21.15.111:0/4250213537 --> [v2:172.21.15.188:6808/3407929299,v1:172.21.15.188:6809/3407929299] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7febf0076da0 con 0x7febf0074290 2024-09-13T17:57:48.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.738+0000 7fec08ff9640 1 -- 172.21.15.111:0/4250213537 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec00066020 con 0x7febfc004b90 2024-09-13T17:57:48.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.738+0000 7f5f0bfff640 1 --2- 172.21.15.111:0/1285331186 >> [v2:172.21.15.111:6800/2426719794,v1:172.21.15.111:6801/2426719794] conn(0x7f5f04031860 0x7f5f04009ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.738+0000 7fec10810640 1 --2- 172.21.15.111:0/4250213537 >> [v2:172.21.15.188:6808/3407929299,v1:172.21.15.188:6809/3407929299] conn(0x7febf0074290 0x7febf00766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.738+0000 7f6a84ff9640 1 --2- 172.21.15.111:0/2120811019 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6a7406acd0 0x7f6a7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.738+0000 7f59ec145640 1 -- 172.21.15.111:0/2425761205 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f59cc000fc0 con 0x7f59e40ce320 2024-09-13T17:57:48.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.739+0000 7f6a84ff9640 1 -- 172.21.15.111:0/2120811019 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6a7c089be0 con 0x7f6a880b0980 2024-09-13T17:57:48.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.739+0000 7f6a84ff9640 1 --2- 172.21.15.111:0/2120811019 >> [v2:172.21.15.188:6800/1593506388,v1:172.21.15.188:6801/1593506388] conn(0x7f6a740741a0 0x7f6a740765e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.739+0000 7fec0b7fe640 1 --2- 172.21.15.111:0/4250213537 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7febf006ac50 0x7febf006d110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fec040df880 tx=0x7febf8001b70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.739+0000 7fb58d7fa640 1 --2- 172.21.15.111:0/2410301176 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb57406acf0 0x7fb57406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.739+0000 7f6a84ff9640 1 -- 172.21.15.111:0/2120811019 --> [v2:172.21.15.188:6800/1593506388,v1:172.21.15.188:6801/1593506388] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6a74076cb0 con 0x7f6a740741a0 2024-09-13T17:57:48.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.739+0000 7f6a84ff9640 1 -- 172.21.15.111:0/2120811019 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a7c089f90 con 0x7f6a880b0980 2024-09-13T17:57:48.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.739+0000 7f5f0bfff640 1 --2- 172.21.15.111:0/1285331186 >> [v2:172.21.15.111:6800/2426719794,v1:172.21.15.111:6801/2426719794] conn(0x7f5f04031860 0x7f5f04009ab0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.738+0000 7f60667fc640 1 -- 172.21.15.111:0/3186799715 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f606000bd50 con 0x7f606815c760 2024-09-13T17:57:48.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.739+0000 7fec10810640 1 --2- 172.21.15.111:0/4250213537 >> [v2:172.21.15.188:6808/3407929299,v1:172.21.15.188:6809/3407929299] conn(0x7febf0074290 0x7febf00766d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.739+0000 7fb58ffff640 1 --2- 172.21.15.111:0/2410301176 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb57406acf0 0x7fb57406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.739+0000 7f60667fc640 1 --2- 172.21.15.111:0/3186799715 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f605406acd0 0x7f605406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.739+0000 7f60667fc640 1 -- 172.21.15.111:0/3186799715 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f606008a520 con 0x7f606815c760 2024-09-13T17:57:48.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.739+0000 7f60667fc640 1 --2- 172.21.15.111:0/3186799715 >> [v2:172.21.15.111:6824/3696423188,v1:172.21.15.111:6825/3696423188] conn(0x7f6054074120 0x7f6054076560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.739+0000 7f60667fc640 1 -- 172.21.15.111:0/3186799715 --> [v2:172.21.15.111:6824/3696423188,v1:172.21.15.111:6825/3696423188] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6054076c30 con 0x7f6054074120 2024-09-13T17:57:48.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7fb596b53640 1 -- 172.21.15.111:0/2410301176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb5900ad4a0 con 0x7fb5800047c0 2024-09-13T17:57:48.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.739+0000 7f60667fc640 1 -- 172.21.15.111:0/3186799715 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60600553f0 con 0x7f606815c760 2024-09-13T17:57:48.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7f59da7fc640 1 -- 172.21.15.111:0/2425761205 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f59d40073d0 con 0x7f59e40ce320 2024-09-13T17:57:48.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7f6a877fe640 1 --2- 172.21.15.111:0/2120811019 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6a7406acd0 0x7f6a7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.739+0000 7ff939ffb640 1 -- 172.21.15.111:0/1319984696 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff93c00a730 con 0x7ff9440b2a20 2024-09-13T17:57:48.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7ff939ffb640 1 --2- 172.21.15.111:0/1319984696 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff92806acc0 0x7ff92806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7f6a8ce6d640 1 --2- 172.21.15.111:0/2120811019 >> [v2:172.21.15.188:6800/1593506388,v1:172.21.15.188:6801/1593506388] conn(0x7f6a740741a0 0x7f6a740765e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7ff939ffb640 1 -- 172.21.15.111:0/1319984696 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff93c0890e0 con 0x7ff9440b2a20 2024-09-13T17:57:48.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7ff939ffb640 1 --2- 172.21.15.111:0/1319984696 >> [v2:172.21.15.111:6814/4024408960,v1:172.21.15.111:6816/4024408960] conn(0x7ff928074280 0x7ff9280766c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7ff939ffb640 1 -- 172.21.15.111:0/1319984696 --> [v2:172.21.15.111:6814/4024408960,v1:172.21.15.111:6816/4024408960] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff928076d90 con 0x7ff928074280 2024-09-13T17:57:48.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7ff939ffb640 1 -- 172.21.15.111:0/1319984696 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff93c003b70 con 0x7ff9440b2a20 2024-09-13T17:57:48.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7ff948b8f640 1 --2- 172.21.15.111:0/1319984696 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff92806acc0 0x7ff92806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7f5ef7fff640 1 -- 172.21.15.111:0/1285331186 <== osd.2 v2:172.21.15.111:6800/2426719794 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f5ef80050b0 con 0x7f5f04031860 2024-09-13T17:57:48.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7f606dafa640 1 --2- 172.21.15.111:0/3186799715 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f605406acd0 0x7f605406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7f59da7fc640 1 --2- 172.21.15.111:0/2425761205 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f59c806ac80 0x7f59c806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7f606d2f9640 1 --2- 172.21.15.111:0/3186799715 >> [v2:172.21.15.111:6824/3696423188,v1:172.21.15.111:6825/3696423188] conn(0x7f6054074120 0x7f6054076560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7f59da7fc640 1 -- 172.21.15.111:0/2425761205 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f59d4088d10 con 0x7f59e40ce320 2024-09-13T17:57:48.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7f59da7fc640 1 --2- 172.21.15.111:0/2425761205 >> [v2:172.21.15.188:6824/2518120865,v1:172.21.15.188:6825/2518120865] conn(0x7f59c80742c0 0x7f59c8076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7f59da7fc640 1 -- 172.21.15.111:0/2425761205 --> [v2:172.21.15.188:6824/2518120865,v1:172.21.15.188:6825/2518120865] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59c8076dd0 con 0x7f59c80742c0 2024-09-13T17:57:48.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7f59da7fc640 1 -- 172.21.15.111:0/2425761205 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59d4089080 con 0x7f59e40ce320 2024-09-13T17:57:48.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7fb58ffff640 1 --2- 172.21.15.111:0/2410301176 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb57406acf0 0x7fb57406d1b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb584032000 tx=0x7fb584005000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7fb58d7fa640 1 -- 172.21.15.111:0/2410301176 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb57c01b2d0 con 0x7fb5800047c0 2024-09-13T17:57:48.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7fb58d7fa640 1 --2- 172.21.15.111:0/2410301176 >> [v2:172.21.15.188:6816/1492113133,v1:172.21.15.188:6817/1492113133] conn(0x7fb574074120 0x7fb574076560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:48.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7fb58d7fa640 1 -- 172.21.15.111:0/2410301176 --> [v2:172.21.15.188:6816/1492113133,v1:172.21.15.188:6817/1492113133] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb574076c30 con 0x7fb574074120 2024-09-13T17:57:48.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7fb58d7fa640 1 -- 172.21.15.111:0/2410301176 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb57c019070 con 0x7fb5800047c0 2024-09-13T17:57:48.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.740+0000 7fb5950f0640 1 --2- 172.21.15.111:0/2410301176 >> [v2:172.21.15.188:6816/1492113133,v1:172.21.15.188:6817/1492113133] conn(0x7fb574074120 0x7fb574076560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.741+0000 7f59e96e0640 1 --2- 172.21.15.111:0/2425761205 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f59c806ac80 0x7f59c806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.741+0000 7f6a8ce6d640 1 --2- 172.21.15.111:0/2120811019 >> [v2:172.21.15.188:6800/1593506388,v1:172.21.15.188:6801/1593506388] conn(0x7f6a740741a0 0x7f6a740765e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.741+0000 7fec08ff9640 1 -- 172.21.15.111:0/4250213537 <== osd.5 v2:172.21.15.188:6808/3407929299 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7febf0076da0 con 0x7febf0074290 2024-09-13T17:57:48.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.741+0000 7f606d2f9640 1 --2- 172.21.15.111:0/3186799715 >> [v2:172.21.15.111:6824/3696423188,v1:172.21.15.111:6825/3696423188] conn(0x7f6054074120 0x7f6054076560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.741+0000 7f6a877fe640 1 --2- 172.21.15.111:0/2120811019 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6a7406acd0 0x7f6a7406d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6a880b4260 tx=0x7f6a78007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.741+0000 7fb5950f0640 1 --2- 172.21.15.111:0/2410301176 >> [v2:172.21.15.188:6816/1492113133,v1:172.21.15.188:6817/1492113133] conn(0x7fb574074120 0x7fb574076560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.741+0000 7f606dafa640 1 --2- 172.21.15.111:0/3186799715 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f605406acd0 0x7f605406d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f605c034000 tx=0x7f605c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.741+0000 7ff948b8f640 1 --2- 172.21.15.111:0/1319984696 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff92806acc0 0x7ff92806d180 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff934036000 tx=0x7ff934030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.741+0000 7ff949b91640 1 --2- 172.21.15.111:0/1319984696 >> [v2:172.21.15.111:6814/4024408960,v1:172.21.15.111:6816/4024408960] conn(0x7ff928074280 0x7ff9280766c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.741+0000 7f59e96e0640 1 --2- 172.21.15.111:0/2425761205 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f59c806ac80 0x7f59c806d140 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f59e0034000 tx=0x7f59e002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.741+0000 7f6a84ff9640 1 -- 172.21.15.111:0/2120811019 <== osd.6 v2:172.21.15.188:6800/1593506388 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f6a74076cb0 con 0x7f6a740741a0 2024-09-13T17:57:48.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.741+0000 7f59ea6e2640 1 --2- 172.21.15.111:0/2425761205 >> [v2:172.21.15.188:6824/2518120865,v1:172.21.15.188:6825/2518120865] conn(0x7f59c80742c0 0x7f59c8076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:48.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.742+0000 7fb58d7fa640 1 -- 172.21.15.111:0/2410301176 <== osd.4 v2:172.21.15.188:6816/1492113133 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb574076c30 con 0x7fb574074120 2024-09-13T17:57:48.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.742+0000 7f60667fc640 1 -- 172.21.15.111:0/3186799715 <== osd.3 v2:172.21.15.111:6824/3696423188 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f6054076c30 con 0x7f6054074120 2024-09-13T17:57:48.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.742+0000 7ff949b91640 1 --2- 172.21.15.111:0/1319984696 >> [v2:172.21.15.111:6814/4024408960,v1:172.21.15.111:6816/4024408960] conn(0x7ff928074280 0x7ff9280766c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.742+0000 7ff939ffb640 1 -- 172.21.15.111:0/1319984696 <== osd.0 v2:172.21.15.111:6814/4024408960 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff928076d90 con 0x7ff928074280 2024-09-13T17:57:48.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.744+0000 7f59ea6e2640 1 --2- 172.21.15.111:0/2425761205 >> [v2:172.21.15.188:6824/2518120865,v1:172.21.15.188:6825/2518120865] conn(0x7f59c80742c0 0x7f59c8076700 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:48.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.747+0000 7f59da7fc640 1 -- 172.21.15.111:0/2425761205 <== osd.7 v2:172.21.15.188:6824/2518120865 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f59c8076dd0 con 0x7f59c80742c0 2024-09-13T17:57:48.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.767+0000 7ff9277fe640 1 -- 172.21.15.111:0/1319984696 --> [v2:172.21.15.111:6814/4024408960,v1:172.21.15.111:6816/4024408960] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff94406da80 con 0x7ff928074280 2024-09-13T17:57:48.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.768+0000 7ff939ffb640 1 -- 172.21.15.111:0/1319984696 <== osd.0 v2:172.21.15.111:6814/4024408960 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff928076d90 con 0x7ff928074280 2024-09-13T17:57:48.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.768+0000 7ff94b5f4640 1 -- 172.21.15.111:0/1319984696 >> [v2:172.21.15.111:6814/4024408960,v1:172.21.15.111:6816/4024408960] conn(0x7ff928074280 msgr2=0x7ff9280766c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.768+0000 7ff94b5f4640 1 --2- 172.21.15.111:0/1319984696 >> [v2:172.21.15.111:6814/4024408960,v1:172.21.15.111:6816/4024408960] conn(0x7ff928074280 0x7ff9280766c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.768+0000 7f4b13253640 1 -- 172.21.15.111:0/2936972192 --> [v2:172.21.15.111:6808/2899699337,v1:172.21.15.111:6809/2899699337] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4b0c0cd060 con 0x7f4af0074290 2024-09-13T17:57:48.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.769+0000 7f4b097fa640 1 -- 172.21.15.111:0/2936972192 <== osd.1 v2:172.21.15.111:6808/2899699337 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4b04004070 con 0x7f4af0074290 2024-09-13T17:57:48.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.769+0000 7ff94b5f4640 1 -- 172.21.15.111:0/1319984696 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff92806acc0 msgr2=0x7ff92806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.769+0000 7ff94b5f4640 1 --2- 172.21.15.111:0/1319984696 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff92806acc0 0x7ff92806d180 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff934036000 tx=0x7ff934030000 comp rx=0 tx=0).stop 2024-09-13T17:57:48.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.769+0000 7ff94b5f4640 1 -- 172.21.15.111:0/1319984696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9440b2a20 msgr2=0x7ff9440ae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.769+0000 7ff94b5f4640 1 --2- 172.21.15.111:0/1319984696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9440b2a20 0x7ff9440ae4c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00caf0 tx=0x7ff93c002100 comp rx=0 tx=0).stop 2024-09-13T17:57:48.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.769+0000 7ff949b91640 1 -- 172.21.15.111:0/1319984696 reap_dead start 2024-09-13T17:57:48.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.769+0000 7ff94b5f4640 1 -- 172.21.15.111:0/1319984696 shutdown_connections 2024-09-13T17:57:48.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.769+0000 7ff94b5f4640 1 -- 172.21.15.111:0/1319984696 >> 172.21.15.111:0/1319984696 conn(0x7ff9440989d0 msgr2=0x7ff94415ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.770+0000 7f4af6ffd640 1 -- 172.21.15.111:0/2936972192 >> [v2:172.21.15.111:6808/2899699337,v1:172.21.15.111:6809/2899699337] conn(0x7f4af0074290 msgr2=0x7f4af00766d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.770+0000 7f4af6ffd640 1 --2- 172.21.15.111:0/2936972192 >> [v2:172.21.15.111:6808/2899699337,v1:172.21.15.111:6809/2899699337] conn(0x7f4af0074290 0x7f4af00766d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.770+0000 7f5f109a0640 1 -- 172.21.15.111:0/1285331186 --> [v2:172.21.15.111:6800/2426719794,v1:172.21.15.111:6801/2426719794] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5f0401dc00 con 0x7f5f04031860 2024-09-13T17:57:48.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.770+0000 7f4af6ffd640 1 -- 172.21.15.111:0/2936972192 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4af006acd0 msgr2=0x7f4af006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.770+0000 7f4af6ffd640 1 --2- 172.21.15.111:0/2936972192 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4af006acd0 0x7f4af006d190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0cc260 tx=0x7f4af8009040 comp rx=0 tx=0).stop 2024-09-13T17:57:48.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.770+0000 7f4af6ffd640 1 -- 172.21.15.111:0/2936972192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b0c0cd860 msgr2=0x7f4b0c158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.770+0000 7f4af6ffd640 1 --2- 172.21.15.111:0/2936972192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b0c0cd860 0x7f4b0c158170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4b00000c00 tx=0x7f4b000046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.770+0000 7f4b117f0640 1 -- 172.21.15.111:0/2936972192 reap_dead start 2024-09-13T17:57:48.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.770+0000 7f5ef7fff640 1 -- 172.21.15.111:0/1285331186 <== osd.2 v2:172.21.15.111:6800/2426719794 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5ef8005220 con 0x7f5f04031860 2024-09-13T17:57:48.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.770+0000 7f4af6ffd640 1 -- 172.21.15.111:0/2936972192 shutdown_connections 2024-09-13T17:57:48.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.770+0000 7f4af6ffd640 1 -- 172.21.15.111:0/2936972192 >> 172.21.15.111:0/2936972192 conn(0x7f4b0c089f60 msgr2=0x7f4b0c0c52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.771+0000 7f5f109a0640 1 -- 172.21.15.111:0/1285331186 >> [v2:172.21.15.111:6800/2426719794,v1:172.21.15.111:6801/2426719794] conn(0x7f5f04031860 msgr2=0x7f5f04009ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.771+0000 7f5f109a0640 1 --2- 172.21.15.111:0/1285331186 >> [v2:172.21.15.111:6800/2426719794,v1:172.21.15.111:6801/2426719794] conn(0x7f5f04031860 0x7f5f04009ab0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.771+0000 7ff94b5f4640 1 -- 172.21.15.111:0/1319984696 shutdown_connections 2024-09-13T17:57:48.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.771+0000 7f5f109a0640 1 -- 172.21.15.111:0/1285331186 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5ef006ad80 msgr2=0x7f5ef006d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.771+0000 7f5f109a0640 1 --2- 172.21.15.111:0/1285331186 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5ef006ad80 0x7f5ef006d240 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5f0403a5f0 tx=0x7f5f00008040 comp rx=0 tx=0).stop 2024-09-13T17:57:48.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.771+0000 7f5f109a0640 1 -- 172.21.15.111:0/1285331186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f0404eb20 msgr2=0x7f5f040d9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.771+0000 7f5f109a0640 1 --2- 172.21.15.111:0/1285331186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f0404eb20 0x7f5f040d9b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c02c970 tx=0x7f5f0c069bc0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.771+0000 7f5f0bfff640 1 -- 172.21.15.111:0/1285331186 reap_dead start 2024-09-13T17:57:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.771+0000 7ff94b5f4640 1 -- 172.21.15.111:0/1319984696 wait complete. 2024-09-13T17:57:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.771+0000 7f5f109a0640 1 -- 172.21.15.111:0/1285331186 shutdown_connections 2024-09-13T17:57:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.771+0000 7f5f109a0640 1 -- 172.21.15.111:0/1285331186 >> 172.21.15.111:0/1285331186 conn(0x7f5f04036050 msgr2=0x7f5f04031150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.771+0000 7f5f109a0640 1 -- 172.21.15.111:0/1285331186 shutdown_connections 2024-09-13T17:57:48.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.771+0000 7f5f109a0640 1 -- 172.21.15.111:0/1285331186 wait complete. 2024-09-13T17:57:48.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.772+0000 7f4af6ffd640 1 -- 172.21.15.111:0/2936972192 shutdown_connections 2024-09-13T17:57:48.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.772+0000 7fb596b53640 1 -- 172.21.15.111:0/2410301176 --> [v2:172.21.15.188:6816/1492113133,v1:172.21.15.188:6817/1492113133] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb59009c3f0 con 0x7fb574074120 2024-09-13T17:57:48.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.772+0000 7f6a6e7fc640 1 -- 172.21.15.111:0/2120811019 --> [v2:172.21.15.188:6800/1593506388,v1:172.21.15.188:6801/1593506388] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6a8806d9c0 con 0x7f6a740741a0 2024-09-13T17:57:48.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.772+0000 7f4af6ffd640 1 -- 172.21.15.111:0/2936972192 wait complete. 2024-09-13T17:57:48.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.772+0000 7febf67fc640 1 -- 172.21.15.111:0/4250213537 --> [v2:172.21.15.188:6808/3407929299,v1:172.21.15.188:6809/3407929299] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fec04037340 con 0x7febf0074290 2024-09-13T17:57:48.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.772+0000 7fb58d7fa640 1 -- 172.21.15.111:0/2410301176 <== osd.4 v2:172.21.15.188:6816/1492113133 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb59009c3f0 con 0x7fb574074120 2024-09-13T17:57:48.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.772+0000 7f6a84ff9640 1 -- 172.21.15.111:0/2120811019 <== osd.6 v2:172.21.15.188:6800/1593506388 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6a80007620 con 0x7f6a740741a0 2024-09-13T17:57:48.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.773+0000 7fec08ff9640 1 -- 172.21.15.111:0/4250213537 <== osd.5 v2:172.21.15.188:6808/3407929299 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fec0c04ea60 con 0x7febf0074290 2024-09-13T17:57:48.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.773+0000 7f606fd5e640 1 -- 172.21.15.111:0/3186799715 --> [v2:172.21.15.111:6824/3696423188,v1:172.21.15.111:6825/3696423188] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6048002cb0 con 0x7f6054074120 2024-09-13T17:57:48.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.773+0000 7fb596b53640 1 -- 172.21.15.111:0/2410301176 >> [v2:172.21.15.188:6816/1492113133,v1:172.21.15.188:6817/1492113133] conn(0x7fb574074120 msgr2=0x7fb574076560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.773+0000 7fb596b53640 1 --2- 172.21.15.111:0/2410301176 >> [v2:172.21.15.188:6816/1492113133,v1:172.21.15.188:6817/1492113133] conn(0x7fb574074120 0x7fb574076560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.773+0000 7fb596b53640 1 -- 172.21.15.111:0/2410301176 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb57406acf0 msgr2=0x7fb57406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.773+0000 7fb596b53640 1 --2- 172.21.15.111:0/2410301176 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb57406acf0 0x7fb57406d1b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb584032000 tx=0x7fb584005000 comp rx=0 tx=0).stop 2024-09-13T17:57:48.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.773+0000 7fb596b53640 1 -- 172.21.15.111:0/2410301176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5800047c0 msgr2=0x7fb59009af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.773+0000 7fb596b53640 1 --2- 172.21.15.111:0/2410301176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5800047c0 0x7fb59009af00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00a2e0 tx=0x7fb57c00a7a0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.773+0000 7fb5950f0640 1 -- 172.21.15.111:0/2410301176 reap_dead start 2024-09-13T17:57:48.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.773+0000 7f6a6e7fc640 1 -- 172.21.15.111:0/2120811019 >> [v2:172.21.15.188:6800/1593506388,v1:172.21.15.188:6801/1593506388] conn(0x7f6a740741a0 msgr2=0x7f6a740765e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.773+0000 7f6a6e7fc640 1 --2- 172.21.15.111:0/2120811019 >> [v2:172.21.15.188:6800/1593506388,v1:172.21.15.188:6801/1593506388] conn(0x7f6a740741a0 0x7f6a740765e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.773+0000 7f60667fc640 1 -- 172.21.15.111:0/3186799715 <== osd.3 v2:172.21.15.111:6824/3696423188 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6048002cb0 con 0x7f6054074120 2024-09-13T17:57:48.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.773+0000 7f6a6e7fc640 1 -- 172.21.15.111:0/2120811019 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6a7406acd0 msgr2=0x7f6a7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.773+0000 7f6a6e7fc640 1 --2- 172.21.15.111:0/2120811019 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6a7406acd0 0x7f6a7406d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6a880b4260 tx=0x7f6a78007e40 comp rx=0 tx=0).stop 2024-09-13T17:57:48.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.773+0000 7f6a6e7fc640 1 -- 172.21.15.111:0/2120811019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a880b0980 msgr2=0x7f6a880c10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.773+0000 7f6a6e7fc640 1 --2- 172.21.15.111:0/2120811019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a880b0980 0x7f6a880c10e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00c450 tx=0x7f6a7c00c910 comp rx=0 tx=0).stop 2024-09-13T17:57:48.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.773+0000 7f6a8ce6d640 1 -- 172.21.15.111:0/2120811019 reap_dead start 2024-09-13T17:57:48.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7fb596b53640 1 -- 172.21.15.111:0/2410301176 shutdown_connections 2024-09-13T17:57:48.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7fb596b53640 1 -- 172.21.15.111:0/2410301176 >> 172.21.15.111:0/2410301176 conn(0x7fb5900989d0 msgr2=0x7fb5900af160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7f6a6e7fc640 1 -- 172.21.15.111:0/2120811019 shutdown_connections 2024-09-13T17:57:48.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7f6a6e7fc640 1 -- 172.21.15.111:0/2120811019 >> 172.21.15.111:0/2120811019 conn(0x7f6a880afe60 msgr2=0x7f6a880ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7febf67fc640 1 -- 172.21.15.111:0/4250213537 >> [v2:172.21.15.188:6808/3407929299,v1:172.21.15.188:6809/3407929299] conn(0x7febf0074290 msgr2=0x7febf00766d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7febf67fc640 1 --2- 172.21.15.111:0/4250213537 >> [v2:172.21.15.188:6808/3407929299,v1:172.21.15.188:6809/3407929299] conn(0x7febf0074290 0x7febf00766d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7f606fd5e640 1 -- 172.21.15.111:0/3186799715 >> [v2:172.21.15.111:6824/3696423188,v1:172.21.15.111:6825/3696423188] conn(0x7f6054074120 msgr2=0x7f6054076560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7f606fd5e640 1 --2- 172.21.15.111:0/3186799715 >> [v2:172.21.15.111:6824/3696423188,v1:172.21.15.111:6825/3696423188] conn(0x7f6054074120 0x7f6054076560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7f606fd5e640 1 -- 172.21.15.111:0/3186799715 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f605406acd0 msgr2=0x7f605406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7f606fd5e640 1 --2- 172.21.15.111:0/3186799715 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f605406acd0 0x7f605406d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f605c034000 tx=0x7f605c02e000 comp rx=0 tx=0).stop 2024-09-13T17:57:48.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7f606fd5e640 1 -- 172.21.15.111:0/3186799715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f606815c760 msgr2=0x7f6068156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7f606fd5e640 1 --2- 172.21.15.111:0/3186799715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f606815c760 0x7f6068156600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f606000d800 tx=0x7f606000dcc0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7f6a6e7fc640 1 -- 172.21.15.111:0/2120811019 shutdown_connections 2024-09-13T17:57:48.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7f6a6e7fc640 1 -- 172.21.15.111:0/2120811019 wait complete. 2024-09-13T17:57:48.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7f606e2fb640 1 -- 172.21.15.111:0/3186799715 reap_dead start 2024-09-13T17:57:48.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7f606fd5e640 1 -- 172.21.15.111:0/3186799715 shutdown_connections 2024-09-13T17:57:48.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7f606fd5e640 1 -- 172.21.15.111:0/3186799715 >> 172.21.15.111:0/3186799715 conn(0x7f60680989d0 msgr2=0x7f60680ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7f59ec145640 1 -- 172.21.15.111:0/2425761205 --> [v2:172.21.15.188:6824/2518120865,v1:172.21.15.188:6825/2518120865] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f59cc002cb0 con 0x7f59c80742c0 2024-09-13T17:57:48.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7f606fd5e640 1 -- 172.21.15.111:0/3186799715 shutdown_connections 2024-09-13T17:57:48.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7febf67fc640 1 -- 172.21.15.111:0/4250213537 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7febf006ac50 msgr2=0x7febf006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7febf67fc640 1 --2- 172.21.15.111:0/4250213537 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7febf006ac50 0x7febf006d110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fec040df880 tx=0x7febf8001b70 comp rx=0 tx=0).stop 2024-09-13T17:57:48.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7febf67fc640 1 -- 172.21.15.111:0/4250213537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febfc004b90 msgr2=0x7fec040d7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7febf67fc640 1 --2- 172.21.15.111:0/4250213537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febfc004b90 0x7fec040d7b30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fec00000c00 tx=0x7fec00004730 comp rx=0 tx=0).stop 2024-09-13T17:57:48.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7fec10810640 1 -- 172.21.15.111:0/4250213537 reap_dead start 2024-09-13T17:57:48.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7febf67fc640 1 -- 172.21.15.111:0/4250213537 shutdown_connections 2024-09-13T17:57:48.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7febf67fc640 1 -- 172.21.15.111:0/4250213537 >> 172.21.15.111:0/4250213537 conn(0x7fec0401ea00 msgr2=0x7fec040e5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7f606fd5e640 1 -- 172.21.15.111:0/3186799715 wait complete. 2024-09-13T17:57:48.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7fb596b53640 1 -- 172.21.15.111:0/2410301176 shutdown_connections 2024-09-13T17:57:48.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.774+0000 7fb596b53640 1 -- 172.21.15.111:0/2410301176 wait complete. 2024-09-13T17:57:48.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.775+0000 7febf67fc640 1 -- 172.21.15.111:0/4250213537 shutdown_connections 2024-09-13T17:57:48.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.775+0000 7febf67fc640 1 -- 172.21.15.111:0/4250213537 wait complete. 2024-09-13T17:57:48.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.775+0000 7f59da7fc640 1 -- 172.21.15.111:0/2425761205 <== osd.7 v2:172.21.15.188:6824/2518120865 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f59cc002cb0 con 0x7f59c80742c0 2024-09-13T17:57:48.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.776+0000 7f59ec145640 1 -- 172.21.15.111:0/2425761205 >> [v2:172.21.15.188:6824/2518120865,v1:172.21.15.188:6825/2518120865] conn(0x7f59c80742c0 msgr2=0x7f59c8076700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.776+0000 7f59ec145640 1 --2- 172.21.15.111:0/2425761205 >> [v2:172.21.15.188:6824/2518120865,v1:172.21.15.188:6825/2518120865] conn(0x7f59c80742c0 0x7f59c8076700 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:48.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.778+0000 7f59ec145640 1 -- 172.21.15.111:0/2425761205 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f59c806ac80 msgr2=0x7f59c806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.778+0000 7f59ec145640 1 --2- 172.21.15.111:0/2425761205 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f59c806ac80 0x7f59c806d140 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f59e0034000 tx=0x7f59e002e000 comp rx=0 tx=0).stop 2024-09-13T17:57:48.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.778+0000 7f59ec145640 1 -- 172.21.15.111:0/2425761205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59e40ce320 msgr2=0x7f59e415ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:48.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.778+0000 7f59ec145640 1 --2- 172.21.15.111:0/2425761205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59e40ce320 0x7f59e415ef40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f59d400c4a0 tx=0x7f59d400c960 comp rx=0 tx=0).stop 2024-09-13T17:57:48.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.778+0000 7f59ea6e2640 1 -- 172.21.15.111:0/2425761205 reap_dead start 2024-09-13T17:57:48.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.778+0000 7f59ec145640 1 -- 172.21.15.111:0/2425761205 shutdown_connections 2024-09-13T17:57:48.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.778+0000 7f59ec145640 1 -- 172.21.15.111:0/2425761205 >> 172.21.15.111:0/2425761205 conn(0x7f59e406a2f0 msgr2=0x7f59e4162f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:48.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.778+0000 7f59ec145640 1 -- 172.21.15.111:0/2425761205 shutdown_connections 2024-09-13T17:57:48.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:48.778+0000 7f59ec145640 1 -- 172.21.15.111:0/2425761205 wait complete. 2024-09-13T17:57:48.850 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2024-09-13T17:57:48.851 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T17:57:48.852 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2024-09-13T17:57:48.852 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T17:57:48.853 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2024-09-13T17:57:48.853 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T17:57:48.854 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2024-09-13T17:57:48.854 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T17:57:48.855 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2024-09-13T17:57:48.855 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T17:57:48.856 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2024-09-13T17:57:48.856 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T17:57:48.857 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2024-09-13T17:57:48.857 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T17:57:48.858 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-09-13T17:57:48.858 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T17:57:49.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.054+0000 7f461a02b640 1 Processor -- start 2024-09-13T17:57:49.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.054+0000 7f461a02b640 1 -- start start 2024-09-13T17:57:49.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.055+0000 7fd4cd296640 1 Processor -- start 2024-09-13T17:57:49.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.055+0000 7fd4cd296640 1 -- start start 2024-09-13T17:57:49.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.055+0000 7f461a02b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f46140b0800 0x7f46140aaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.055+0000 7f461a02b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f46140ab490 0x7f46140b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.055+0000 7f461a02b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46140ad410 con 0x7f4614160a10 2024-09-13T17:57:49.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.055+0000 7f461a02b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461406ca50 con 0x7f46140b0800 2024-09-13T17:57:49.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.055+0000 7f461a02b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461406cd30 con 0x7f46140ab490 2024-09-13T17:57:49.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.055+0000 7f4612ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f46140b0800 0x7f46140aaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.056+0000 7f4612ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f46140b0800 0x7f46140aaf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54044/0 (socket says 172.21.15.111:54044) 2024-09-13T17:57:49.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.056+0000 7f4612ffd640 1 -- 172.21.15.111:0/3498718986 learned_addr learned my addr 172.21.15.111:0/3498718986 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:49.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.056+0000 7f4613fff640 1 --2- 172.21.15.111:0/3498718986 >> v2:172.21.15.188:3300/0 conn(0x7f46140ab490 0x7f46140b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.056+0000 7f4612ffd640 1 -- 172.21.15.111:0/3498718986 >> v2:172.21.15.188:3300/0 conn(0x7f46140ab490 msgr2=0x7f46140b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:49.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.056+0000 7f4612ffd640 1 --2- 172.21.15.111:0/3498718986 >> v2:172.21.15.188:3300/0 conn(0x7f46140ab490 0x7f46140b0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.056+0000 7f4612ffd640 1 -- 172.21.15.111:0/3498718986 >> v1:172.21.15.188:6789/0 conn(0x7f4614160a10 legacy=0x7f461406d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.056+0000 7f4612ffd640 1 -- 172.21.15.111:0/3498718986 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461406d010 con 0x7f46140b0800 2024-09-13T17:57:49.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.056+0000 7f4b32b65640 1 Processor -- start 2024-09-13T17:57:49.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.056+0000 7f4b32b65640 1 -- start start 2024-09-13T17:57:49.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.057+0000 7fd4c6ffd640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fd4c0005cf0 0x7fd4c00539c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:45690/0 (socket says 172.21.15.111:45690) 2024-09-13T17:57:49.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.057+0000 7fd4c6ffd640 1 -- 172.21.15.111:0/1003812496 learned_addr learned my addr 172.21.15.111:0/1003812496 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:49.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.057+0000 7fd4cd296640 1 --2- 172.21.15.111:0/1003812496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4c00eb6f0 0x7fd4c00edae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.057+0000 7fd4cd296640 1 -- 172.21.15.111:0/1003812496 --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c003cfa0 con 0x7fd4c003cb60 2024-09-13T17:57:49.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.057+0000 7fd4cd296640 1 -- 172.21.15.111:0/1003812496 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c00540d0 con 0x7fd4c0005cf0 2024-09-13T17:57:49.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.057+0000 7fd4cd296640 1 -- 172.21.15.111:0/1003812496 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c0057900 con 0x7fd4c00eb6f0 2024-09-13T17:57:49.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.057+0000 7fd4c77fe640 1 --2- 172.21.15.111:0/1003812496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4c00eb6f0 0x7fd4c00edae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.057+0000 7fae4b6aa640 1 Processor -- start 2024-09-13T17:57:49.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.057+0000 7fd4c5ffb640 1 -- 172.21.15.111:0/1003812496 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38054945 0 0) 0x7fd4c00540d0 con 0x7fd4c0005cf0 2024-09-13T17:57:49.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.057+0000 7fae4b6aa640 1 -- start start 2024-09-13T17:57:49.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.057+0000 7f1adcb18640 1 Processor -- start 2024-09-13T17:57:49.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.057+0000 7f1adcb18640 1 -- start start 2024-09-13T17:57:49.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.058+0000 7f4b32b65640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4b2c0709b0 0x7f4b2c06bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.058+0000 7f31246f0640 1 Processor -- start 2024-09-13T17:57:49.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.058+0000 7f31246f0640 1 -- start start 2024-09-13T17:57:49.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.057+0000 7fd4c5ffb640 1 -- 172.21.15.111:0/1003812496 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4b0003520 con 0x7fd4c0005cf0 2024-09-13T17:57:49.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.057+0000 7fd4c77fe640 1 -- 172.21.15.111:0/1003812496 >> v1:172.21.15.188:6789/0 conn(0x7fd4c0005cf0 legacy=0x7fd4c00539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.057+0000 7fd4c77fe640 1 -- 172.21.15.111:0/1003812496 >> v1:172.21.15.111:6789/0 conn(0x7fd4c003cb60 legacy=0x7fd4c00060d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:49.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.057+0000 7fd4c77fe640 1 -- 172.21.15.111:0/1003812496 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c0057be0 con 0x7fd4c00eb6f0 2024-09-13T17:57:49.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.058+0000 7f4b31362640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4b2c0709b0 0x7f4b2c06bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.058+0000 7f4b31362640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4b2c0709b0 0x7f4b2c06bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54060/0 (socket says 172.21.15.111:54060) 2024-09-13T17:57:49.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.058+0000 7f4b31362640 1 -- 172.21.15.111:0/1575667979 learned_addr learned my addr 172.21.15.111:0/1575667979 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:49.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.058+0000 7fd4c77fe640 1 --2- 172.21.15.111:0/1003812496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4c00eb6f0 0x7fd4c00edae0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8000b60 tx=0x7fd4c80a40e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18d3b65f8385f139 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.058+0000 7f4612ffd640 1 --2- 172.21.15.111:0/3498718986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46140b0800 0x7f46140aaf50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f461409c280 tx=0x7f460c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48223523db15d759 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.058+0000 7f46127fc640 1 -- 172.21.15.111:0/3498718986 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c031600 con 0x7f46140b0800 2024-09-13T17:57:49.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.058+0000 7f46127fc640 1 -- 172.21.15.111:0/3498718986 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460c02d050 con 0x7f46140b0800 2024-09-13T17:57:49.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.058+0000 7f46127fc640 1 -- 172.21.15.111:0/3498718986 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c031980 con 0x7f46140b0800 2024-09-13T17:57:49.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.058+0000 7fae4b6aa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fae44160a30 0x7fae440c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.058+0000 7fae4b6aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae44099230 0x7fae440adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.058+0000 7fae4b6aa640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae440c3c50 con 0x7fae440c3810 2024-09-13T17:57:49.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.058+0000 7fae4b6aa640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae440996a0 con 0x7fae44160a30 2024-09-13T17:57:49.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.058+0000 7fae4b6aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae44099890 con 0x7fae44099230 2024-09-13T17:57:49.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7fae48c45640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fae440c3810 0x7fae44098b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49760/0 (socket says 172.21.15.111:49760) 2024-09-13T17:57:49.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7fae48c45640 1 -- 172.21.15.111:0/2749450402 learned_addr learned my addr 172.21.15.111:0/2749450402 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:49.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7f0e22501640 1 Processor -- start 2024-09-13T17:57:49.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7f0e22501640 1 -- start start 2024-09-13T17:57:49.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7f4b32b65640 1 -- 172.21.15.111:0/1575667979 --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2c152cf0 con 0x7f4b2c06dff0 2024-09-13T17:57:49.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7fdbe33b7640 1 Processor -- start 2024-09-13T17:57:49.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7f4b32b65640 1 -- 172.21.15.111:0/1575667979 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2c06cd40 con 0x7f4b2c152910 2024-09-13T17:57:49.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7fae49c47640 1 --2- 172.21.15.111:0/2749450402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae44099230 0x7fae440adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7f4b32b65640 1 -- 172.21.15.111:0/1575667979 --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2c06d020 con 0x7f4b2c0709b0 2024-09-13T17:57:49.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7f461a02b640 1 -- 172.21.15.111:0/3498718986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46140b0800 msgr2=0x7f46140aaf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7f461a02b640 1 --2- 172.21.15.111:0/3498718986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46140b0800 0x7f46140aaf50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f461409c280 tx=0x7f460c02a730 comp rx=0 tx=0).stop 2024-09-13T17:57:49.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7fdbe33b7640 1 -- start start 2024-09-13T17:57:49.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7fd4c5ffb640 1 -- 172.21.15.111:0/1003812496 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80ac440 con 0x7fd4c00eb6f0 2024-09-13T17:57:49.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7fd4c5ffb640 1 -- 172.21.15.111:0/1003812496 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c80ac5a0 con 0x7fd4c00eb6f0 2024-09-13T17:57:49.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7fd4c5ffb640 1 -- 172.21.15.111:0/1003812496 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80ac950 con 0x7fd4c00eb6f0 2024-09-13T17:57:49.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7f31246f0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f311c0b2f70 0x7f311c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7f31246f0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f311c0ad490 0x7f311c0ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f31246f0640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311c158ab0 con 0x7f311c0ad8f0 2024-09-13T17:57:49.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7f461a02b640 1 -- 172.21.15.111:0/3498718986 shutdown_connections 2024-09-13T17:57:49.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7f461a02b640 1 --2- 172.21.15.111:0/3498718986 >> v2:172.21.15.188:3300/0 conn(0x7f46140ab490 0x7f46140b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7f461a02b640 1 --2- 172.21.15.111:0/3498718986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46140b0800 0x7f46140aaf50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.059+0000 7f461a02b640 1 -- 172.21.15.111:0/3498718986 >> 172.21.15.111:0/3498718986 conn(0x7f46140afc70 msgr2=0x7f46140b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:49.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f31246f0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311c0caac0 con 0x7f311c0b2f70 2024-09-13T17:57:49.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f31246f0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311c0cd610 con 0x7f311c0ad490 2024-09-13T17:57:49.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7fd4cd296640 1 -- 172.21.15.111:0/1003812496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4c00eb6f0 msgr2=0x7fd4c00edae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7fd4cd296640 1 --2- 172.21.15.111:0/1003812496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4c00eb6f0 0x7fd4c00edae0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8000b60 tx=0x7fd4c80a40e0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f1adcb18640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ad815cc20 0x7f1ad80ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f3121c8b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f311c0ad490 0x7f311c0ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f3121c8b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f311c0ad490 0x7f311c0ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:34496/0 (socket says 172.21.15.111:34496) 2024-09-13T17:57:49.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f3121c8b640 1 -- 172.21.15.111:0/2193968647 learned_addr learned my addr 172.21.15.111:0/2193968647 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:49.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f1adcb18640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad80cdde0 con 0x7f1ad80cd4d0 2024-09-13T17:57:49.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f1ad659c640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f1ad813b790 0x7f1ad815c510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:45704/0 (socket says 172.21.15.111:45704) 2024-09-13T17:57:49.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f1ad659c640 1 -- 172.21.15.111:0/4264006426 learned_addr learned my addr 172.21.15.111:0/4264006426 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:49.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f1adcb18640 1 -- 172.21.15.111:0/4264006426 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad80ce0c0 con 0x7f1ad813b790 2024-09-13T17:57:49.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f1adcb18640 1 -- 172.21.15.111:0/4264006426 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad80ce3a0 con 0x7f1ad815cc20 2024-09-13T17:57:49.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f4b31362640 1 -- 172.21.15.111:0/1575667979 >> v1:172.21.15.188:6789/0 conn(0x7f4b2c152910 legacy=0x7f4b2c0702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f4b31362640 1 -- 172.21.15.111:0/1575667979 >> v1:172.21.15.111:6789/0 conn(0x7f4b2c06dff0 legacy=0x7f4b2c06c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f4b31362640 1 -- 172.21.15.111:0/1575667979 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c06d300 con 0x7f4b2c0709b0 2024-09-13T17:57:49.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f312248c640 1 --2- 172.21.15.111:0/2193968647 >> v2:172.21.15.111:3300/0 conn(0x7f311c0b2f70 0x7f311c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f0e22501640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e1c0b1750 0x7f0e1c0d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f0e22501640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0e1c0b1b30 0x7f0e1c0d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f0e22501640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1c06c5a0 con 0x7f0e1c0d11b0 2024-09-13T17:57:49.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f0e22501640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e1c0cd320 con 0x7f0e1c0b1b30 2024-09-13T17:57:49.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f0e22501640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c0cd600 con 0x7f0e1c0b1750 2024-09-13T17:57:49.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7fae49446640 1 --2- 172.21.15.111:0/2749450402 >> v2:172.21.15.188:3300/0 conn(0x7fae44160a30 0x7fae440c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7fae49446640 1 -- 172.21.15.111:0/2749450402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae44099230 msgr2=0x7fae440adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:49.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7fae49446640 1 --2- 172.21.15.111:0/2749450402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae44099230 0x7fae440adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7fae49446640 1 -- 172.21.15.111:0/2749450402 >> v1:172.21.15.111:6789/0 conn(0x7fae440c3810 legacy=0x7fae44098b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:49.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7fae49446640 1 -- 172.21.15.111:0/2749450402 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae44099b70 con 0x7fae44160a30 2024-09-13T17:57:49.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f0e1bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e1c0b1750 0x7f0e1c0d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f1ad5d9b640 1 --2- 172.21.15.111:0/4264006426 >> v2:172.21.15.188:3300/0 conn(0x7f1ad815cc20 0x7f1ad80ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f0e1bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e1c0b1750 0x7f0e1c0d0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:34512/0 (socket says 172.21.15.111:34512) 2024-09-13T17:57:49.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7fd4cd296640 1 -- 172.21.15.111:0/1003812496 shutdown_connections 2024-09-13T17:57:49.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7fd4cd296640 1 --2- 172.21.15.111:0/1003812496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4c00eb6f0 0x7fd4c00edae0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7fd4cd296640 1 -- 172.21.15.111:0/1003812496 >> 172.21.15.111:0/1003812496 conn(0x7fd4c003a4e0 msgr2=0x7fd4c003a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:49.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7f0e1bfff640 1 -- 172.21.15.111:0/3474113868 learned_addr learned my addr 172.21.15.111:0/3474113868 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:49.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7f3121c8b640 1 -- 172.21.15.111:0/2193968647 >> v2:172.21.15.111:3300/0 conn(0x7f311c0b2f70 msgr2=0x7f311c06cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:49.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7f3121c8b640 1 --2- 172.21.15.111:0/2193968647 >> v2:172.21.15.111:3300/0 conn(0x7f311c0b2f70 0x7f311c06cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7f3121c8b640 1 -- 172.21.15.111:0/2193968647 >> v1:172.21.15.111:6789/0 conn(0x7f311c0ad8f0 legacy=0x7f311c0b1560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:57:49.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7f3121c8b640 1 -- 172.21.15.111:0/2193968647 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c0cd8f0 con 0x7f311c0ad490 2024-09-13T17:57:49.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7f0e1b7fe640 1 --2- 172.21.15.111:0/3474113868 >> v2:172.21.15.188:3300/0 conn(0x7f0e1c0b1b30 0x7f0e1c0d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7f1ad559a640 1 -- 172.21.15.111:0/4264006426 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2086068121 0 0) 0x7f1ad80cdde0 con 0x7f1ad80cd4d0 2024-09-13T17:57:49.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7f1ad559a640 1 -- 172.21.15.111:0/4264006426 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ac0003520 con 0x7f1ad80cd4d0 2024-09-13T17:57:49.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f461a02b640 1 -- 172.21.15.111:0/3498718986 shutdown_connections 2024-09-13T17:57:49.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.060+0000 7f461a02b640 1 -- 172.21.15.111:0/3498718986 wait complete. 2024-09-13T17:57:49.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7f1ad5d9b640 1 -- 172.21.15.111:0/4264006426 >> v1:172.21.15.188:6789/0 conn(0x7f1ad813b790 legacy=0x7f1ad815c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7f1ad5d9b640 1 -- 172.21.15.111:0/4264006426 >> v1:172.21.15.111:6789/0 conn(0x7f1ad80cd4d0 legacy=0x7f1ad806ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7fdbe33b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbdc0cb4d0 0x7fdbdc164db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7fdbe33b7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdbdc0ada80 0x7fdbdc0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7fdbe33b7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbdc0b3360 con 0x7fdbdc160740 2024-09-13T17:57:49.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7fdbe33b7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbdc0b15c0 con 0x7fdbdc0ada80 2024-09-13T17:57:49.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7fdbe33b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc0b18a0 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7fdbe1bb4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbdc0cb4d0 0x7fdbdc164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7fdbe1bb4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbdc0cb4d0 0x7fdbdc164db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:34528/0 (socket says 172.21.15.111:34528) 2024-09-13T17:57:49.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7fdbe1bb4640 1 -- 172.21.15.111:0/3186023062 learned_addr learned my addr 172.21.15.111:0/3186023062 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:49.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7f1ad5d9b640 1 -- 172.21.15.111:0/4264006426 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad80ce680 con 0x7f1ad815cc20 2024-09-13T17:57:49.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7fdbe2bb6640 1 --2- 172.21.15.111:0/3186023062 >> v2:172.21.15.188:3300/0 conn(0x7fdbdc0ada80 0x7fdbdc0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7f312248c640 1 --2- 172.21.15.111:0/2193968647 >> v2:172.21.15.111:3300/0 conn(0x7f311c0b2f70 0x7f311c06cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:49.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7f4b31362640 1 --2- 172.21.15.111:0/1575667979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b2c0709b0 0x7f4b2c06bfd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4b28007360 tx=0x7f4b2802c220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0104c913be716b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7f0e1bfff640 1 -- 172.21.15.111:0/3474113868 >> v2:172.21.15.188:3300/0 conn(0x7f0e1c0b1b30 msgr2=0x7f0e1c0d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:49.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7f0e1bfff640 1 --2- 172.21.15.111:0/3474113868 >> v2:172.21.15.188:3300/0 conn(0x7f0e1c0b1b30 0x7f0e1c0d22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7f0e1bfff640 1 -- 172.21.15.111:0/3474113868 >> v1:172.21.15.111:6789/0 conn(0x7f0e1c0d11b0 legacy=0x7f0e1c0ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:49.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.062+0000 7fae49446640 1 --2- 172.21.15.111:0/2749450402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44160a30 0x7fae440c32d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fae400097a0 tx=0x7fae4002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=229aec87c7d25c76 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7f0e1bfff640 1 -- 172.21.15.111:0/3474113868 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1c0cd8e0 con 0x7f0e1c0b1750 2024-09-13T17:57:49.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7fd4cd296640 1 -- 172.21.15.111:0/1003812496 shutdown_connections 2024-09-13T17:57:49.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.061+0000 7fd4cd296640 1 -- 172.21.15.111:0/1003812496 wait complete. 2024-09-13T17:57:49.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.062+0000 7f4b30b61640 1 -- 172.21.15.111:0/1575667979 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b280089e0 con 0x7f4b2c0709b0 2024-09-13T17:57:49.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.062+0000 7f4b30b61640 1 -- 172.21.15.111:0/1575667979 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b28008b40 con 0x7f4b2c0709b0 2024-09-13T17:57:49.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.062+0000 7fae3bfff640 1 -- 172.21.15.111:0/2749450402 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4002ae40 con 0x7fae44160a30 2024-09-13T17:57:49.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.062+0000 7f4b30b61640 1 -- 172.21.15.111:0/1575667979 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2800a0d0 con 0x7f4b2c0709b0 2024-09-13T17:57:49.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.062+0000 7fae3bfff640 1 -- 172.21.15.111:0/2749450402 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae40032440 con 0x7fae44160a30 2024-09-13T17:57:49.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.062+0000 7fae3bfff640 1 -- 172.21.15.111:0/2749450402 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae400327d0 con 0x7fae44160a30 2024-09-13T17:57:49.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.062+0000 7f461a02b640 1 Processor -- start 2024-09-13T17:57:49.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.062+0000 7f461a02b640 1 -- start start 2024-09-13T17:57:49.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.062+0000 7f3121c8b640 1 --2- 172.21.15.111:0/2193968647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f311c0ad490 0x7f311c0ce5c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3114009e80 tx=0x7f311402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb2a2f70307c1bca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.062+0000 7fdbe13b3640 1 -- 172.21.15.111:0/3186023062 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754431994 0 0) 0x7fdbdc0b3360 con 0x7fdbdc160740 2024-09-13T17:57:49.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.062+0000 7fdbe13b3640 1 -- 172.21.15.111:0/3186023062 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbc8003520 con 0x7fdbdc160740 2024-09-13T17:57:49.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.062+0000 7fdbe1bb4640 1 -- 172.21.15.111:0/3186023062 >> v2:172.21.15.188:3300/0 conn(0x7fdbdc0ada80 msgr2=0x7fdbdc0cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:49.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.062+0000 7fdbe1bb4640 1 --2- 172.21.15.111:0/3186023062 >> v2:172.21.15.188:3300/0 conn(0x7fdbdc0ada80 0x7fdbdc0cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.062+0000 7fdbe1bb4640 1 -- 172.21.15.111:0/3186023062 >> v1:172.21.15.111:6789/0 conn(0x7fdbdc160740 legacy=0x7fdbdc0cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.062+0000 7fdbe1bb4640 1 -- 172.21.15.111:0/3186023062 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc0cbaf0 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7f312148a640 1 -- 172.21.15.111:0/2193968647 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3114031600 con 0x7f311c0ad490 2024-09-13T17:57:49.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7f0e1bfff640 1 --2- 172.21.15.111:0/3474113868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e1c0b1750 0x7f0e1c0d0c70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009870 tx=0x7f0e0c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f8ec4578dc2772 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7f312148a640 1 -- 172.21.15.111:0/2193968647 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311402d050 con 0x7f311c0ad490 2024-09-13T17:57:49.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7f312148a640 1 -- 172.21.15.111:0/2193968647 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3114031980 con 0x7f311c0ad490 2024-09-13T17:57:49.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7f0e1affd640 1 -- 172.21.15.111:0/3474113868 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c032440 con 0x7f0e1c0b1750 2024-09-13T17:57:49.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7f1ad5d9b640 1 --2- 172.21.15.111:0/4264006426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad815cc20 0x7f1ad80ccf90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80097a0 tx=0x7f1ac802b440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfb520fa1a3e89bf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7f4b32b65640 1 -- 172.21.15.111:0/1575667979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b2c0709b0 msgr2=0x7f4b2c06bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7f4b32b65640 1 --2- 172.21.15.111:0/1575667979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b2c0709b0 0x7f4b2c06bfd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4b28007360 tx=0x7f4b2802c220 comp rx=0 tx=0).stop 2024-09-13T17:57:49.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7f1ad559a640 1 -- 172.21.15.111:0/4264006426 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8030710 con 0x7f1ad815cc20 2024-09-13T17:57:49.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7fdbe1bb4640 1 --2- 172.21.15.111:0/3186023062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbdc0cb4d0 0x7fdbdc164db0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009870 tx=0x7fdbcc02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ad9250ea0bc2300 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7fae4b6aa640 1 -- 172.21.15.111:0/2749450402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44160a30 msgr2=0x7fae440c32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7fae4b6aa640 1 --2- 172.21.15.111:0/2749450402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44160a30 0x7fae440c32d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fae400097a0 tx=0x7fae4002a130 comp rx=0 tx=0).stop 2024-09-13T17:57:49.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7f1ad559a640 1 -- 172.21.15.111:0/4264006426 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac802e050 con 0x7f1ad815cc20 2024-09-13T17:57:49.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7fd4cd296640 1 Processor -- start 2024-09-13T17:57:49.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7fae4b6aa640 1 -- 172.21.15.111:0/2749450402 shutdown_connections 2024-09-13T17:57:49.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7fae4b6aa640 1 --2- 172.21.15.111:0/2749450402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae44099230 0x7fae440adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7fae4b6aa640 1 --2- 172.21.15.111:0/2749450402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44160a30 0x7fae440c32d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7f0e1affd640 1 -- 172.21.15.111:0/3474113868 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0c0325a0 con 0x7f0e1c0b1750 2024-09-13T17:57:49.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7fae4b6aa640 1 -- 172.21.15.111:0/2749450402 >> 172.21.15.111:0/2749450402 conn(0x7fae440b1df0 msgr2=0x7fae440b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:49.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f4b32b65640 1 -- 172.21.15.111:0/1575667979 shutdown_connections 2024-09-13T17:57:49.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f4b32b65640 1 --2- 172.21.15.111:0/1575667979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b2c0709b0 0x7f4b2c06bfd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f4b32b65640 1 -- 172.21.15.111:0/1575667979 >> 172.21.15.111:0/1575667979 conn(0x7f4b2c0afc70 msgr2=0x7f4b2c0b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:49.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7fae4b6aa640 1 -- 172.21.15.111:0/2749450402 shutdown_connections 2024-09-13T17:57:49.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7f461a02b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46140ab490 0x7f461409ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7f461a02b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46140b0800 0x7f46141f9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.063+0000 7f461a02b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4614160a10 0x7f46141f9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f46137fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46140ab490 0x7f461409ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f46137fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46140ab490 0x7f461409ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:34530/0 (socket says 172.21.15.111:34530) 2024-09-13T17:57:49.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f46137fe640 1 -- 172.21.15.111:0/3048554861 learned_addr learned my addr 172.21.15.111:0/3048554861 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:49.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7fae4b6aa640 1 -- 172.21.15.111:0/2749450402 wait complete. 2024-09-13T17:57:49.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f461a02b640 1 -- 172.21.15.111:0/3048554861 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46140990c0 con 0x7f4614160a10 2024-09-13T17:57:49.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f461a02b640 1 -- 172.21.15.111:0/3048554861 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46140993a0 con 0x7f46140b0800 2024-09-13T17:57:49.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f461a02b640 1 -- 172.21.15.111:0/3048554861 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4614099680 con 0x7f46140ab490 2024-09-13T17:57:49.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f0e22501640 1 -- 172.21.15.111:0/3474113868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e1c0b1750 msgr2=0x7f0e1c0d0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f0e22501640 1 --2- 172.21.15.111:0/3474113868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e1c0b1750 0x7f0e1c0d0c70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009870 tx=0x7f0e0c02a210 comp rx=0 tx=0).stop 2024-09-13T17:57:49.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7fd4cd296640 1 -- start start 2024-09-13T17:57:49.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f1ad559a640 1 -- 172.21.15.111:0/4264006426 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8030a90 con 0x7f1ad815cc20 2024-09-13T17:57:49.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7fdbe13b3640 1 -- 172.21.15.111:0/3186023062 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc032440 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7fdbe13b3640 1 -- 172.21.15.111:0/3186023062 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc0325a0 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7fdbe13b3640 1 -- 172.21.15.111:0/3186023062 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc032930 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f31246f0640 1 -- 172.21.15.111:0/2193968647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f311c0ad490 msgr2=0x7f311c0ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f0e22501640 1 -- 172.21.15.111:0/3474113868 shutdown_connections 2024-09-13T17:57:49.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f0e22501640 1 --2- 172.21.15.111:0/3474113868 >> v2:172.21.15.188:3300/0 conn(0x7f0e1c0b1b30 0x7f0e1c0d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f0e22501640 1 --2- 172.21.15.111:0/3474113868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e1c0b1750 0x7f0e1c0d0c70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f0e22501640 1 -- 172.21.15.111:0/3474113868 >> 172.21.15.111:0/3474113868 conn(0x7f0e1c09b830 msgr2=0x7f0e1c089f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:49.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f31246f0640 1 --2- 172.21.15.111:0/2193968647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f311c0ad490 0x7f311c0ce5c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3114009e80 tx=0x7f311402a660 comp rx=0 tx=0).stop 2024-09-13T17:57:49.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f0e22501640 1 -- 172.21.15.111:0/3474113868 shutdown_connections 2024-09-13T17:57:49.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f1adcb18640 1 -- 172.21.15.111:0/4264006426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad815cc20 msgr2=0x7f1ad80ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f1adcb18640 1 --2- 172.21.15.111:0/4264006426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad815cc20 0x7f1ad80ccf90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80097a0 tx=0x7f1ac802b440 comp rx=0 tx=0).stop 2024-09-13T17:57:49.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f4612ffd640 1 --2- 172.21.15.111:0/3048554861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46140b0800 0x7f46141f9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f4613fff640 1 --2- 172.21.15.111:0/3048554861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4614160a10 0x7f46141f9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f46137fe640 1 -- 172.21.15.111:0/3048554861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46140b0800 msgr2=0x7f46141f9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f46137fe640 1 --2- 172.21.15.111:0/3048554861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46140b0800 0x7f46141f9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f46137fe640 1 -- 172.21.15.111:0/3048554861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4614160a10 msgr2=0x7f46141f9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f46137fe640 1 --2- 172.21.15.111:0/3048554861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4614160a10 0x7f46141f9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f46137fe640 1 -- 172.21.15.111:0/3048554861 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c009b30 con 0x7f46140ab490 2024-09-13T17:57:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f4613fff640 1 --2- 172.21.15.111:0/3048554861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4614160a10 0x7f46141f9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7f31246f0640 1 -- 172.21.15.111:0/2193968647 shutdown_connections 2024-09-13T17:57:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7f1adcb18640 1 -- 172.21.15.111:0/4264006426 shutdown_connections 2024-09-13T17:57:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7f1adcb18640 1 --2- 172.21.15.111:0/4264006426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad815cc20 0x7f1ad80ccf90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7f31246f0640 1 --2- 172.21.15.111:0/2193968647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f311c0ad490 0x7f311c0ce5c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7f31246f0640 1 --2- 172.21.15.111:0/2193968647 >> v2:172.21.15.111:3300/0 conn(0x7f311c0b2f70 0x7f311c06cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7f1adcb18640 1 -- 172.21.15.111:0/4264006426 >> 172.21.15.111:0/4264006426 conn(0x7f1ad8069ed0 msgr2=0x7f1ad806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:49.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7f31246f0640 1 -- 172.21.15.111:0/2193968647 >> 172.21.15.111:0/2193968647 conn(0x7f311c06a0b0 msgr2=0x7f311c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:49.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7f46137fe640 1 --2- 172.21.15.111:0/3048554861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46140ab490 0x7f461409ad80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f460800c450 tx=0x7f460800c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7f1adcb18640 1 -- 172.21.15.111:0/4264006426 shutdown_connections 2024-09-13T17:57:49.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7f31246f0640 1 -- 172.21.15.111:0/2193968647 shutdown_connections 2024-09-13T17:57:49.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7fd4cd296640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0005cf0 0x7fd4c00dbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7f31246f0640 1 -- 172.21.15.111:0/2193968647 wait complete. 2024-09-13T17:57:49.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7f1adcb18640 1 -- 172.21.15.111:0/4264006426 wait complete. 2024-09-13T17:57:49.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7f45fbfff640 1 -- 172.21.15.111:0/3048554861 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46080052d0 con 0x7f46140ab490 2024-09-13T17:57:49.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7f45fbfff640 1 -- 172.21.15.111:0/3048554861 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4608005430 con 0x7f46140ab490 2024-09-13T17:57:49.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7f45fbfff640 1 -- 172.21.15.111:0/3048554861 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608012870 con 0x7f46140ab490 2024-09-13T17:57:49.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7f461a02b640 1 -- 172.21.15.111:0/3048554861 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4614099960 con 0x7f46140ab490 2024-09-13T17:57:49.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7f461a02b640 1 -- 172.21.15.111:0/3048554861 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461409b2c0 con 0x7f46140ab490 2024-09-13T17:57:49.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7fd4cd296640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4c003cb60 0x7fd4c00e4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7fd4cd296640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4c00eb6f0 0x7fd4c00e5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7fd4cd296640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c004cca0 con 0x7fd4c0005cf0 2024-09-13T17:57:49.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7fd4cd296640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c00dc6b0 con 0x7fd4c00eb6f0 2024-09-13T17:57:49.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7fd4cd296640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c004c900 con 0x7fd4c003cb60 2024-09-13T17:57:49.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7fd4c6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0005cf0 0x7fd4c00dbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7fd4c6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0005cf0 0x7fd4c00dbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54078/0 (socket says 172.21.15.111:54078) 2024-09-13T17:57:49.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7fd4c6ffd640 1 -- 172.21.15.111:0/1141465216 learned_addr learned my addr 172.21.15.111:0/1141465216 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:49.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7fd4c67fc640 1 --2- 172.21.15.111:0/1141465216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4c003cb60 0x7fd4c00e4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7fd4c77fe640 1 --2- 172.21.15.111:0/1141465216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4c00eb6f0 0x7fd4c00e5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7fd4c6ffd640 1 -- 172.21.15.111:0/1141465216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4c003cb60 msgr2=0x7fd4c00e4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7fd4c6ffd640 1 --2- 172.21.15.111:0/1141465216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4c003cb60 0x7fd4c00e4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7fd4c6ffd640 1 -- 172.21.15.111:0/1141465216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4c00eb6f0 msgr2=0x7fd4c00e5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7fd4c6ffd640 1 --2- 172.21.15.111:0/1141465216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4c00eb6f0 0x7fd4c00e5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7fd4c6ffd640 1 -- 172.21.15.111:0/1141465216 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c8072900 con 0x7fd4c0005cf0 2024-09-13T17:57:49.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7fae4b6aa640 1 Processor -- start 2024-09-13T17:57:49.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7fae4b6aa640 1 -- start start 2024-09-13T17:57:49.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.064+0000 7f0e22501640 1 -- 172.21.15.111:0/3474113868 wait complete. 2024-09-13T17:57:49.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7f0e22501640 1 Processor -- start 2024-09-13T17:57:49.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7fdbe33b7640 1 -- 172.21.15.111:0/3186023062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbdc0cb4d0 msgr2=0x7fdbdc164db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.065+0000 7fdbe33b7640 1 --2- 172.21.15.111:0/3186023062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbdc0cb4d0 0x7fdbdc164db0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009870 tx=0x7fdbcc02a1c0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7fdbe33b7640 1 -- 172.21.15.111:0/3186023062 shutdown_connections 2024-09-13T17:57:49.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7fdbe33b7640 1 --2- 172.21.15.111:0/3186023062 >> v2:172.21.15.188:3300/0 conn(0x7fdbdc0ada80 0x7fdbdc0cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7fdbe33b7640 1 --2- 172.21.15.111:0/3186023062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbdc0cb4d0 0x7fdbdc164db0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7fdbe33b7640 1 -- 172.21.15.111:0/3186023062 >> 172.21.15.111:0/3186023062 conn(0x7fdbdc0b0100 msgr2=0x7fdbdc0b04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:49.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7fdbe33b7640 1 -- 172.21.15.111:0/3186023062 shutdown_connections 2024-09-13T17:57:49.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7fdbe33b7640 1 -- 172.21.15.111:0/3186023062 wait complete. 2024-09-13T17:57:49.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7fae4b6aa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44099230 0x7fae440c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7fae4b6aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae440c3810 0x7fae440c4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fae4b6aa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae44160a30 0x7fae440c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fae4b6aa640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae441f6e30 con 0x7fae44160a30 2024-09-13T17:57:49.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fae4b6aa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae440b5450 con 0x7fae44099230 2024-09-13T17:57:49.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fae4b6aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae440cb0c0 con 0x7fae440c3810 2024-09-13T17:57:49.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7f461a02b640 1 -- 172.21.15.111:0/3048554861 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46140aaf50 con 0x7f46140ab490 2024-09-13T17:57:49.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fae49c47640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae44160a30 0x7fae440c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7fd4c6ffd640 1 --2- 172.21.15.111:0/1141465216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0005cf0 0x7fd4c00dbdb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800c410 tx=0x7fd4b800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fae49c47640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae44160a30 0x7fae440c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54082/0 (socket says 172.21.15.111:54082) 2024-09-13T17:57:49.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fae49c47640 1 -- 172.21.15.111:0/2757367120 learned_addr learned my addr 172.21.15.111:0/2757367120 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:49.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fae49446640 1 --2- 172.21.15.111:0/2757367120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44099230 0x7fae440c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fd4af7fe640 1 -- 172.21.15.111:0/1141465216 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8005ce0 con 0x7fd4c0005cf0 2024-09-13T17:57:49.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7f1adcb18640 1 Processor -- start 2024-09-13T17:57:49.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fd4af7fe640 1 -- 172.21.15.111:0/1141465216 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b8005e40 con 0x7fd4c0005cf0 2024-09-13T17:57:49.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fd4af7fe640 1 -- 172.21.15.111:0/1141465216 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80072f0 con 0x7fd4c0005cf0 2024-09-13T17:57:49.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fd4cd296640 1 -- 172.21.15.111:0/1141465216 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c00e3870 con 0x7fd4c0005cf0 2024-09-13T17:57:49.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7f1adcb18640 1 -- start start 2024-09-13T17:57:49.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7f31246f0640 1 Processor -- start 2024-09-13T17:57:49.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fd4cd296640 1 -- 172.21.15.111:0/1141465216 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c00e3590 con 0x7fd4c0005cf0 2024-09-13T17:57:49.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7f31246f0640 1 -- start start 2024-09-13T17:57:49.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fae49446640 1 -- 172.21.15.111:0/2757367120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae440c3810 msgr2=0x7fae440c4340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T17:57:49.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fae49446640 1 --2- 172.21.15.111:0/2757367120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae440c3810 0x7fae440c4340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.066+0000 7f0e22501640 1 -- start start 2024-09-13T17:57:49.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fae49446640 1 -- 172.21.15.111:0/2757367120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae44160a30 msgr2=0x7fae440c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fae49446640 1 --2- 172.21.15.111:0/2757367120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae44160a30 0x7fae440c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7fae49446640 1 -- 172.21.15.111:0/2757367120 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae40009400 con 0x7fae44099230 2024-09-13T17:57:49.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.068+0000 7fae49446640 1 --2- 172.21.15.111:0/2757367120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44099230 0x7fae440c3e00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fae40031000 tx=0x7fae400046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.068+0000 7fae49c47640 1 --2- 172.21.15.111:0/2757367120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae44160a30 0x7fae440c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:49.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.068+0000 7fdbe33b7640 1 Processor -- start 2024-09-13T17:57:49.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.068+0000 7fdbe33b7640 1 -- start start 2024-09-13T17:57:49.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.067+0000 7f0e22501640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e1c0b1750 0x7f0e1c1ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.068+0000 7f1adcb18640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ad80cd4d0 0x7f1ad8151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f31246f0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f311c0ad490 0x7f311c071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f31246f0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f311c0ad8f0 0x7f311c158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f1adcb18640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad813b790 0x7f1ad8154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f1adcb18640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad815cc20 0x7f1ad8152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f4b32b65640 1 -- 172.21.15.111:0/1575667979 shutdown_connections 2024-09-13T17:57:49.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f45fbfff640 1 -- 172.21.15.111:0/3048554861 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f46080129d0 con 0x7f46140ab490 2024-09-13T17:57:49.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f4b32b65640 1 -- 172.21.15.111:0/1575667979 wait complete. 2024-09-13T17:57:49.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7fd4cd296640 1 -- 172.21.15.111:0/1141465216 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a8005250 con 0x7fd4c0005cf0 2024-09-13T17:57:49.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f0e1bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e1c0b1750 0x7f0e1c1ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f0e1bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e1c0b1750 0x7f0e1c1ff970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54094/0 (socket says 172.21.15.111:54094) 2024-09-13T17:57:49.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f1ad659c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ad80cd4d0 0x7f1ad8151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f1ad659c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ad80cd4d0 0x7f1ad8151a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:34558/0 (socket says 172.21.15.111:34558) 2024-09-13T17:57:49.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f0e1bfff640 1 -- 172.21.15.111:0/511730971 learned_addr learned my addr 172.21.15.111:0/511730971 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:49.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f0e22501640 1 --2- 172.21.15.111:0/511730971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e1c0d11b0 0x7f0e1c15ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f0e22501640 1 --2- 172.21.15.111:0/511730971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e1c0b2f70 0x7f0e1c1fc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f0e22501640 1 -- 172.21.15.111:0/511730971 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c200000 con 0x7f0e1c0b1750 2024-09-13T17:57:49.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7fdbe33b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbdc0ada80 0x7fdbdc161b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7fdbe33b7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbdc0cb4d0 0x7fdbdc148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7fdbe33b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbdc160740 0x7fdbdc162b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7fdbe23b5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbdc0ada80 0x7fdbdc161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7fdbe23b5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbdc0ada80 0x7fdbdc161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:49156/0 (socket says 172.21.15.111:49156) 2024-09-13T17:57:49.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7fdbe23b5640 1 -- 172.21.15.111:0/131335997 learned_addr learned my addr 172.21.15.111:0/131335997 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:49.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f0e22501640 1 -- 172.21.15.111:0/511730971 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c0b7220 con 0x7f0e1c0b2f70 2024-09-13T17:57:49.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f0e22501640 1 -- 172.21.15.111:0/511730971 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c203460 con 0x7f0e1c0d11b0 2024-09-13T17:57:49.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7f0e20a9e640 1 --2- 172.21.15.111:0/511730971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e1c0b2f70 0x7f0e1c1fc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7f0e1b7fe640 1 --2- 172.21.15.111:0/511730971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e1c0d11b0 0x7f0e1c15ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7fdbe33b7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc148a90 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7fdbe33b7640 1 -- 172.21.15.111:0/131335997 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc160e20 con 0x7fdbdc0ada80 2024-09-13T17:57:49.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7fdbe33b7640 1 -- 172.21.15.111:0/131335997 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc163530 con 0x7fdbdc160740 2024-09-13T17:57:49.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7fdbe1bb4640 1 --2- 172.21.15.111:0/131335997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbdc0cb4d0 0x7fdbdc148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f45fbfff640 1 --2- 172.21.15.111:0/3048554861 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f460006ac50 0x7f460006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f45fbfff640 1 -- 172.21.15.111:0/3048554861 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4608089a50 con 0x7f46140ab490 2024-09-13T17:57:49.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7f4612ffd640 1 --2- 172.21.15.111:0/3048554861 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f460006ac50 0x7f460006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f31246f0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f311c0b2f70 0x7f311c1591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f31246f0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c06ca70 con 0x7f311c0ad8f0 2024-09-13T17:57:49.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f31246f0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c159d30 con 0x7f311c0b2f70 2024-09-13T17:57:49.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7f31246f0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311c15a010 con 0x7f311c0ad490 2024-09-13T17:57:49.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7f3121c8b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f311c0ad8f0 0x7f311c158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7f3121c8b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f311c0ad8f0 0x7f311c158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54104/0 (socket says 172.21.15.111:54104) 2024-09-13T17:57:49.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7f3121c8b640 1 -- 172.21.15.111:0/1865313293 learned_addr learned my addr 172.21.15.111:0/1865313293 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:49.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7f3121c8b640 1 -- 172.21.15.111:0/1865313293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f311c0ad490 msgr2=0x7f311c071990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T17:57:49.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7f3121c8b640 1 --2- 172.21.15.111:0/1865313293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f311c0ad490 0x7f311c071990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7f3121c8b640 1 -- 172.21.15.111:0/1865313293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f311c0b2f70 msgr2=0x7f311c1591c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:57:49.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7f3121c8b640 1 --2- 172.21.15.111:0/1865313293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f311c0b2f70 0x7f311c1591c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7f3121c8b640 1 -- 172.21.15.111:0/1865313293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3114009b30 con 0x7f311c0ad8f0 2024-09-13T17:57:49.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7f3121c8b640 1 --2- 172.21.15.111:0/1865313293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f311c0ad8f0 0x7f311c158c80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3114009e80 tx=0x7f3114004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7fae39ffb640 1 -- 172.21.15.111:0/2757367120 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40030910 con 0x7fae44099230 2024-09-13T17:57:49.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7fae4b6aa640 1 -- 172.21.15.111:0/2757367120 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae440cb3a0 con 0x7fae44099230 2024-09-13T17:57:49.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7fae4b6aa640 1 -- 172.21.15.111:0/2757367120 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae440cb960 con 0x7fae44099230 2024-09-13T17:57:49.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7fae39ffb640 1 -- 172.21.15.111:0/2757367120 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae40030a70 con 0x7fae44099230 2024-09-13T17:57:49.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7fae39ffb640 1 -- 172.21.15.111:0/2757367120 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40030910 con 0x7fae44099230 2024-09-13T17:57:49.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7fdbe1bb4640 1 -- 172.21.15.111:0/131335997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbdc160740 msgr2=0x7fdbdc162b70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T17:57:49.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7fdbe1bb4640 1 --2- 172.21.15.111:0/131335997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbdc160740 0x7fdbdc162b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7fdbe1bb4640 1 -- 172.21.15.111:0/131335997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbdc0ada80 msgr2=0x7fdbdc161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7fdbe1bb4640 1 --2- 172.21.15.111:0/131335997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbdc0ada80 0x7fdbdc161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7f4612ffd640 1 --2- 172.21.15.111:0/3048554861 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f460006ac50 0x7f460006d110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f460c033000 tx=0x7f460c009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7f0e20a9e640 1 -- 172.21.15.111:0/511730971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e1c0d11b0 msgr2=0x7f0e1c15ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7f0e20a9e640 1 --2- 172.21.15.111:0/511730971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e1c0d11b0 0x7f0e1c15ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7f0e20a9e640 1 -- 172.21.15.111:0/511730971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e1c0b1750 msgr2=0x7f0e1c1ff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f1ad5d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad813b790 0x7f1ad8154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.069+0000 7f1ad5d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad813b790 0x7f1ad8154820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:49152/0 (socket says 172.21.15.111:49152) 2024-09-13T17:57:49.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7f1ad659c640 1 -- 172.21.15.111:0/627405051 learned_addr learned my addr 172.21.15.111:0/627405051 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:49.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7f1adcb18640 1 -- 172.21.15.111:0/627405051 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad80a76d0 con 0x7f1ad815cc20 2024-09-13T17:57:49.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7f1adcb18640 1 -- 172.21.15.111:0/627405051 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8153780 con 0x7f1ad813b790 2024-09-13T17:57:49.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7fae4b6aa640 1 -- 172.21.15.111:0/2757367120 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae440c32d0 con 0x7fae44099230 2024-09-13T17:57:49.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.070+0000 7f1adcb18640 1 -- 172.21.15.111:0/627405051 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8153a60 con 0x7f1ad80cd4d0 2024-09-13T17:57:49.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7f3112ffd640 1 -- 172.21.15.111:0/1865313293 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3114031cb0 con 0x7f311c0ad8f0 2024-09-13T17:57:49.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7f3112ffd640 1 -- 172.21.15.111:0/1865313293 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3114031e10 con 0x7f311c0ad8f0 2024-09-13T17:57:49.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7fdbe1bb4640 1 -- 172.21.15.111:0/131335997 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbcc0094d0 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7fd4af7fe640 1 -- 172.21.15.111:0/1141465216 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd4b8007510 con 0x7fd4c0005cf0 2024-09-13T17:57:49.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7f0e20a9e640 1 --2- 172.21.15.111:0/511730971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e1c0b1750 0x7f0e1c1ff970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7f0e20a9e640 1 -- 172.21.15.111:0/511730971 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c0094d0 con 0x7f0e1c0b2f70 2024-09-13T17:57:49.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7f0e1bfff640 1 --2- 172.21.15.111:0/511730971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e1c0b1750 0x7f0e1c1ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:49.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7f1ad6d9d640 1 --2- 172.21.15.111:0/627405051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad815cc20 0x7f1ad8152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7f3112ffd640 1 -- 172.21.15.111:0/1865313293 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3114049640 con 0x7f311c0ad8f0 2024-09-13T17:57:49.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7f31246f0640 1 -- 172.21.15.111:0/1865313293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311c15b720 con 0x7f311c0ad8f0 2024-09-13T17:57:49.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7f31246f0640 1 -- 172.21.15.111:0/1865313293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311c06ccc0 con 0x7f311c0ad8f0 2024-09-13T17:57:49.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.071+0000 7fdbe1bb4640 1 --2- 172.21.15.111:0/131335997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbdc0cb4d0 0x7fdbdc148550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0b19e0 tx=0x7fdbcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.072+0000 7fd4af7fe640 1 --2- 172.21.15.111:0/1141465216 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd4b406ac00 0x7fd4b406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.072+0000 7fd4af7fe640 1 -- 172.21.15.111:0/1141465216 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd4b8088bf0 con 0x7fd4c0005cf0 2024-09-13T17:57:49.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.072+0000 7f0e20a9e640 1 --2- 172.21.15.111:0/511730971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e1c0b2f70 0x7f0e1c1fc350 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400be00 tx=0x7f0e1400bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.072+0000 7f4b32b65640 1 Processor -- start 2024-09-13T17:57:49.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.072+0000 7f4b32b65640 1 -- start start 2024-09-13T17:57:49.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.072+0000 7f0e18ff9640 1 -- 172.21.15.111:0/511730971 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1400ce70 con 0x7f0e1c0b2f70 2024-09-13T17:57:49.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.072+0000 7f0e22501640 1 -- 172.21.15.111:0/511730971 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1c0b6f60 con 0x7f0e1c0b2f70 2024-09-13T17:57:49.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.072+0000 7f0e18ff9640 1 -- 172.21.15.111:0/511730971 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e14003970 con 0x7f0e1c0b2f70 2024-09-13T17:57:49.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.072+0000 7f0e18ff9640 1 -- 172.21.15.111:0/511730971 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14003d60 con 0x7f0e1c0b2f70 2024-09-13T17:57:49.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.072+0000 7f0e22501640 1 -- 172.21.15.111:0/511730971 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1c0b5770 con 0x7f0e1c0b2f70 2024-09-13T17:57:49.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.072+0000 7f1ad6d9d640 1 -- 172.21.15.111:0/627405051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ad80cd4d0 msgr2=0x7f1ad8151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.072+0000 7f1ad6d9d640 1 --2- 172.21.15.111:0/627405051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ad80cd4d0 0x7f1ad8151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.072+0000 7f1ad6d9d640 1 -- 172.21.15.111:0/627405051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad813b790 msgr2=0x7f1ad8154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.072+0000 7f1ad6d9d640 1 --2- 172.21.15.111:0/627405051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad813b790 0x7f1ad8154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.072+0000 7f1ad6d9d640 1 -- 172.21.15.111:0/627405051 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac8009400 con 0x7f1ad815cc20 2024-09-13T17:57:49.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.072+0000 7fd4c67fc640 1 --2- 172.21.15.111:0/1141465216 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd4b406ac00 0x7fd4b406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.073+0000 7f4b32b65640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b2c06dff0 0x7f4b2c1f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.073+0000 7fae39ffb640 1 -- 172.21.15.111:0/2757367120 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fae40039040 con 0x7fae44099230 2024-09-13T17:57:49.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.073+0000 7f4b32b65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b2c0709b0 0x7f4b2c14b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.073+0000 7fd4c67fc640 1 --2- 172.21.15.111:0/1141465216 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd4b406ac00 0x7fd4b406d0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc006fd0 tx=0x7fd4bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.073+0000 7f1ad6d9d640 1 --2- 172.21.15.111:0/627405051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad815cc20 0x7f1ad8152270 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000caf0 tx=0x7f1ad0002090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.073+0000 7f4b31b63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b2c06dff0 0x7f4b2c1f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.073+0000 7f45fbfff640 1 -- 172.21.15.111:0/3048554861 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4608054ab0 con 0x7f46140ab490 2024-09-13T17:57:49.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.073+0000 7f4b32b65640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b2c152910 0x7f4b2c14bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.073+0000 7f4b32b65640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c14ab30 con 0x7f4b2c152910 2024-09-13T17:57:49.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.073+0000 7f4b32b65640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c1f60e0 con 0x7f4b2c06dff0 2024-09-13T17:57:49.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.073+0000 7f4b32b65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c149c40 con 0x7f4b2c0709b0 2024-09-13T17:57:49.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.073+0000 7f4b31b63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b2c06dff0 0x7f4b2c1f5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:49160/0 (socket says 172.21.15.111:49160) 2024-09-13T17:57:49.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.073+0000 7f4b31b63640 1 -- 172.21.15.111:0/1311279333 learned_addr learned my addr 172.21.15.111:0/1311279333 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:49.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.073+0000 7f31246f0640 1 -- 172.21.15.111:0/1865313293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311c06e280 con 0x7f311c0ad8f0 2024-09-13T17:57:49.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.074+0000 7f4b31362640 1 --2- 172.21.15.111:0/1311279333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b2c0709b0 0x7f4b2c14b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.074+0000 7f4b32364640 1 --2- 172.21.15.111:0/1311279333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b2c152910 0x7f4b2c14bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.073+0000 7fae39ffb640 1 --2- 172.21.15.111:0/2757367120 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fae2806ac60 0x7fae2806d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.074+0000 7fae39ffb640 1 -- 172.21.15.111:0/2757367120 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fae400a9bd0 con 0x7fae44099230 2024-09-13T17:57:49.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.074+0000 7f4b31b63640 1 -- 172.21.15.111:0/1311279333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b2c0709b0 msgr2=0x7f4b2c14b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.074+0000 7fae48c45640 1 --2- 172.21.15.111:0/2757367120 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fae2806ac60 0x7fae2806d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.074+0000 7f4b31b63640 1 --2- 172.21.15.111:0/1311279333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b2c0709b0 0x7f4b2c14b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.074+0000 7f4b31b63640 1 -- 172.21.15.111:0/1311279333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b2c152910 msgr2=0x7f4b2c14bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.074+0000 7f4b31b63640 1 --2- 172.21.15.111:0/1311279333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b2c152910 0x7f4b2c14bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.074+0000 7f4b31b63640 1 -- 172.21.15.111:0/1311279333 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b28007040 con 0x7f4b2c06dff0 2024-09-13T17:57:49.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.073+0000 7f0e22501640 1 -- 172.21.15.111:0/511730971 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dfc005250 con 0x7f0e1c0b2f70 2024-09-13T17:57:49.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.074+0000 7f4b31b63640 1 --2- 172.21.15.111:0/1311279333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b2c06dff0 0x7f4b2c1f5ba0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00c550 tx=0x7f4b1c00ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.074+0000 7fae48c45640 1 --2- 172.21.15.111:0/2757367120 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fae2806ac60 0x7fae2806d120 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fae440b5fc0 tx=0x7fae3c004f40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.074+0000 7f0e18ff9640 1 -- 172.21.15.111:0/511730971 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0e14003ee0 con 0x7f0e1c0b2f70 2024-09-13T17:57:49.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.075+0000 7f0e18ff9640 1 --2- 172.21.15.111:0/511730971 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0e0806ac30 0x7f0e0806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.075+0000 7f0e18ff9640 1 -- 172.21.15.111:0/511730971 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0e14089510 con 0x7f0e1c0b2f70 2024-09-13T17:57:49.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.075+0000 7f0e1bfff640 1 --2- 172.21.15.111:0/511730971 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0e0806ac30 0x7f0e0806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.075+0000 7f0e1bfff640 1 --2- 172.21.15.111:0/511730971 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0e0806ac30 0x7f0e0806d0f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c000c00 tx=0x7f0e0c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.077+0000 7fdbd2ffd640 1 -- 172.21.15.111:0/131335997 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc03d8f0 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.077+0000 7f1ac6ffd640 1 -- 172.21.15.111:0/627405051 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0003b70 con 0x7f1ad815cc20 2024-09-13T17:57:49.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.077+0000 7f1adcb18640 1 -- 172.21.15.111:0/627405051 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad8153d40 con 0x7f1ad815cc20 2024-09-13T17:57:49.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.077+0000 7fdbe33b7640 1 -- 172.21.15.111:0/131335997 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbdc161070 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.077+0000 7fd4af7fe640 1 -- 172.21.15.111:0/1141465216 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fd4b8053c40 con 0x7fd4c0005cf0 2024-09-13T17:57:49.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.077+0000 7f1adcb18640 1 -- 172.21.15.111:0/627405051 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad8154300 con 0x7f1ad815cc20 2024-09-13T17:57:49.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.079+0000 7fae39ffb640 1 -- 172.21.15.111:0/2757367120 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fae40074ab0 con 0x7fae44099230 2024-09-13T17:57:49.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.077+0000 7fdbe33b7640 1 -- 172.21.15.111:0/131335997 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbdc161630 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.079+0000 7fdbd2ffd640 1 -- 172.21.15.111:0/131335997 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc03da50 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.079+0000 7f3112ffd640 1 -- 172.21.15.111:0/1865313293 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3114047070 con 0x7f311c0ad8f0 2024-09-13T17:57:49.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.079+0000 7f1adcb18640 1 -- 172.21.15.111:0/627405051 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad8154540 con 0x7f1ad815cc20 2024-09-13T17:57:49.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.079+0000 7f1ac6ffd640 1 -- 172.21.15.111:0/627405051 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad0003cd0 con 0x7f1ad815cc20 2024-09-13T17:57:49.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.079+0000 7f1ac6ffd640 1 -- 172.21.15.111:0/627405051 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0003b70 con 0x7f1ad815cc20 2024-09-13T17:57:49.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.079+0000 7f3112ffd640 1 --2- 172.21.15.111:0/1865313293 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f310006ac00 0x7f310006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.080+0000 7f4b227fc640 1 -- 172.21.15.111:0/1311279333 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c0035f0 con 0x7f4b2c06dff0 2024-09-13T17:57:49.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.080+0000 7f3112ffd640 1 -- 172.21.15.111:0/1865313293 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f311402ae40 con 0x7f311c0ad8f0 2024-09-13T17:57:49.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.080+0000 7f4b32b65640 1 -- 172.21.15.111:0/1311279333 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2c149f20 con 0x7f4b2c06dff0 2024-09-13T17:57:49.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.080+0000 7f312248c640 1 --2- 172.21.15.111:0/1865313293 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f310006ac00 0x7f310006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.080+0000 7f4b32b65640 1 -- 172.21.15.111:0/1311279333 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2c14a4e0 con 0x7f4b2c06dff0 2024-09-13T17:57:49.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.080+0000 7fdbd2ffd640 1 -- 172.21.15.111:0/131335997 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc03d8f0 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.081+0000 7f312248c640 1 --2- 172.21.15.111:0/1865313293 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f310006ac00 0x7f310006d0c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f310c00b810 tx=0x7f310c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.081+0000 7fdbd0ff9640 1 -- 172.21.15.111:0/131335997 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc4005250 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.082+0000 7fdbd2ffd640 1 -- 172.21.15.111:0/131335997 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdbcc039040 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.082+0000 7f4b17fff640 1 -- 172.21.15.111:0/1311279333 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0c005250 con 0x7f4b2c06dff0 2024-09-13T17:57:49.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.082+0000 7fdbd2ffd640 1 --2- 172.21.15.111:0/131335997 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdbc006ac60 0x7fdbc006d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.082+0000 7f1ac6ffd640 1 -- 172.21.15.111:0/627405051 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1ad00050d0 con 0x7f1ad815cc20 2024-09-13T17:57:49.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.082+0000 7fdbe23b5640 1 --2- 172.21.15.111:0/131335997 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdbc006ac60 0x7fdbc006d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.083+0000 7f4b227fc640 1 -- 172.21.15.111:0/1311279333 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1c003750 con 0x7f4b2c06dff0 2024-09-13T17:57:49.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.083+0000 7f1ac6ffd640 1 --2- 172.21.15.111:0/627405051 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ab806ac60 0x7f1ab806d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.083+0000 7f0e18ff9640 1 -- 172.21.15.111:0/511730971 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f0e140544b0 con 0x7f0e1c0b2f70 2024-09-13T17:57:49.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.083+0000 7fdbe23b5640 1 --2- 172.21.15.111:0/131335997 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdbc006ac60 0x7fdbc006d120 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8001560 tx=0x7fdbd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.083+0000 7f1ad659c640 1 --2- 172.21.15.111:0/627405051 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ab806ac60 0x7f1ab806d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.084+0000 7f4b227fc640 1 -- 172.21.15.111:0/1311279333 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c003ac0 con 0x7f4b2c06dff0 2024-09-13T17:57:49.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.084+0000 7f1ad659c640 1 --2- 172.21.15.111:0/627405051 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ab806ac60 0x7f1ab806d120 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8158500 tx=0x7f1acc007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.084+0000 7f4b227fc640 1 -- 172.21.15.111:0/1311279333 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4b1c012400 con 0x7f4b2c06dff0 2024-09-13T17:57:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.085+0000 7f4b227fc640 1 --2- 172.21.15.111:0/1311279333 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4b1006ac00 0x7f4b1006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.085+0000 7f4b227fc640 1 -- 172.21.15.111:0/1311279333 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4b1c089210 con 0x7f4b2c06dff0 2024-09-13T17:57:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.085+0000 7f4b31362640 1 --2- 172.21.15.111:0/1311279333 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4b1006ac00 0x7f4b1006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.085+0000 7f4b31362640 1 --2- 172.21.15.111:0/1311279333 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4b1006ac00 0x7f4b1006d0c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4b28034000 tx=0x7f4b28030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.087+0000 7f4b227fc640 1 -- 172.21.15.111:0/1311279333 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4b1c054320 con 0x7f4b2c06dff0 2024-09-13T17:57:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.088+0000 7fdbd2ffd640 1 -- 172.21.15.111:0/131335997 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdbcc078750 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.088+0000 7f1ac6ffd640 1 -- 172.21.15.111:0/627405051 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1ad0058c90 con 0x7f1ad815cc20 2024-09-13T17:57:49.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.089+0000 7f3112ffd640 1 -- 172.21.15.111:0/1865313293 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f311407ce60 con 0x7f311c0ad8f0 2024-09-13T17:57:49.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.097+0000 7fdbd2ffd640 1 -- 172.21.15.111:0/131335997 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fdbcc074a90 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.106+0000 7f1ac6ffd640 1 -- 172.21.15.111:0/627405051 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f1ad00548d0 con 0x7f1ad815cc20 2024-09-13T17:57:49.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.462+0000 7f461a02b640 1 -- 172.21.15.111:0/3048554861 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f46140997c0 con 0x7f46140ab490 2024-09-13T17:57:49.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.462+0000 7f45fbfff640 1 -- 172.21.15.111:0/3048554861 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f46140997c0 con 0x7f46140ab490 2024-09-13T17:57:49.465 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-09-13T17:57:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.466+0000 7f45f9ffb640 1 -- 172.21.15.111:0/3048554861 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f460006ac50 msgr2=0x7f460006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.466+0000 7f45f9ffb640 1 --2- 172.21.15.111:0/3048554861 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f460006ac50 0x7f460006d110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f460c033000 tx=0x7f460c009000 comp rx=0 tx=0).stop 2024-09-13T17:57:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.466+0000 7f45f9ffb640 1 -- 172.21.15.111:0/3048554861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46140ab490 msgr2=0x7f461409ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.466+0000 7f45f9ffb640 1 --2- 172.21.15.111:0/3048554861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46140ab490 0x7f461409ad80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f460800c450 tx=0x7f460800c910 comp rx=0 tx=0).stop 2024-09-13T17:57:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.467+0000 7f45f9ffb640 1 -- 172.21.15.111:0/3048554861 shutdown_connections 2024-09-13T17:57:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.467+0000 7f45f9ffb640 1 --2- 172.21.15.111:0/3048554861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4614160a10 0x7f46141f9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.467+0000 7f45f9ffb640 1 --2- 172.21.15.111:0/3048554861 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f460006ac50 0x7f460006d110 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.467+0000 7f45f9ffb640 1 --2- 172.21.15.111:0/3048554861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46140b0800 0x7f46141f9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.467+0000 7f45f9ffb640 1 --2- 172.21.15.111:0/3048554861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46140ab490 0x7f461409ad80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.467+0000 7f45f9ffb640 1 -- 172.21.15.111:0/3048554861 >> 172.21.15.111:0/3048554861 conn(0x7f46140afc70 msgr2=0x7f46140addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:49.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.467+0000 7f45f9ffb640 1 -- 172.21.15.111:0/3048554861 shutdown_connections 2024-09-13T17:57:49.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.467+0000 7f45f9ffb640 1 -- 172.21.15.111:0/3048554861 wait complete. 2024-09-13T17:57:49.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.467+0000 7fd4cd296640 1 -- 172.21.15.111:0/1141465216 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd4a8005500 con 0x7fd4c0005cf0 2024-09-13T17:57:49.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.468+0000 7fd4af7fe640 1 -- 172.21.15.111:0/1141465216 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd4b8017090 con 0x7fd4c0005cf0 2024-09-13T17:57:49.472 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-09-13T17:57:49.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.471+0000 7fae4b6aa640 1 -- 172.21.15.111:0/2757367120 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fae44072440 con 0x7fae44099230 2024-09-13T17:57:49.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.472+0000 7fd4ad7fa640 1 -- 172.21.15.111:0/1141465216 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd4b406ac00 msgr2=0x7fd4b406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.472+0000 7fae39ffb640 1 -- 172.21.15.111:0/2757367120 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fae40074ab0 con 0x7fae44099230 2024-09-13T17:57:49.476 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-09-13T17:57:49.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.472+0000 7fd4ad7fa640 1 --2- 172.21.15.111:0/1141465216 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd4b406ac00 0x7fd4b406d0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc006fd0 tx=0x7fd4bc008040 comp rx=0 tx=0).stop 2024-09-13T17:57:49.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.472+0000 7fd4ad7fa640 1 -- 172.21.15.111:0/1141465216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0005cf0 msgr2=0x7fd4c00dbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.472+0000 7fd4ad7fa640 1 --2- 172.21.15.111:0/1141465216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0005cf0 0x7fd4c00dbdb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800c410 tx=0x7fd4b800c8d0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.473+0000 7fd4ad7fa640 1 -- 172.21.15.111:0/1141465216 shutdown_connections 2024-09-13T17:57:49.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.473+0000 7fd4ad7fa640 1 --2- 172.21.15.111:0/1141465216 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd4b406ac00 0x7fd4b406d0c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.473+0000 7fd4ad7fa640 1 --2- 172.21.15.111:0/1141465216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4c00eb6f0 0x7fd4c00e5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.473+0000 7fd4ad7fa640 1 --2- 172.21.15.111:0/1141465216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4c003cb60 0x7fd4c00e4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.473+0000 7fd4ad7fa640 1 --2- 172.21.15.111:0/1141465216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0005cf0 0x7fd4c00dbdb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.473+0000 7fd4ad7fa640 1 -- 172.21.15.111:0/1141465216 >> 172.21.15.111:0/1141465216 conn(0x7fd4c003a4e0 msgr2=0x7fd4c00383e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:49.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.473+0000 7fd4ad7fa640 1 -- 172.21.15.111:0/1141465216 shutdown_connections 2024-09-13T17:57:49.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.473+0000 7fd4ad7fa640 1 -- 172.21.15.111:0/1141465216 wait complete. 2024-09-13T17:57:49.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.475+0000 7f31246f0640 1 -- 172.21.15.111:0/1865313293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f311c0ceed0 con 0x7f311c0ad8f0 2024-09-13T17:57:49.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.475+0000 7f0e22501640 1 -- 172.21.15.111:0/511730971 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0dfc0057d0 con 0x7f0e1c0b2f70 2024-09-13T17:57:49.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.475+0000 7fae2f7fe640 1 -- 172.21.15.111:0/2757367120 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fae2806ac60 msgr2=0x7fae2806d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.475+0000 7fae2f7fe640 1 --2- 172.21.15.111:0/2757367120 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fae2806ac60 0x7fae2806d120 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fae440b5fc0 tx=0x7fae3c004f40 comp rx=0 tx=0).stop 2024-09-13T17:57:49.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.476+0000 7fae2f7fe640 1 -- 172.21.15.111:0/2757367120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44099230 msgr2=0x7fae440c3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.476+0000 7f3112ffd640 1 -- 172.21.15.111:0/1865313293 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3114036030 con 0x7f311c0ad8f0 2024-09-13T17:57:49.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.476+0000 7fae2f7fe640 1 --2- 172.21.15.111:0/2757367120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44099230 0x7fae440c3e00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fae40031000 tx=0x7fae400046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.482 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-09-13T17:57:49.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.476+0000 7fae2f7fe640 1 -- 172.21.15.111:0/2757367120 shutdown_connections 2024-09-13T17:57:49.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.476+0000 7fae2f7fe640 1 --2- 172.21.15.111:0/2757367120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae44160a30 0x7fae440c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.476+0000 7fae2f7fe640 1 --2- 172.21.15.111:0/2757367120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae440c3810 0x7fae440c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.476+0000 7fae2f7fe640 1 --2- 172.21.15.111:0/2757367120 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fae2806ac60 0x7fae2806d120 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.476+0000 7fae2f7fe640 1 --2- 172.21.15.111:0/2757367120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44099230 0x7fae440c3e00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.476+0000 7fae2f7fe640 1 -- 172.21.15.111:0/2757367120 >> 172.21.15.111:0/2757367120 conn(0x7fae440b1df0 msgr2=0x7fae440b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:49.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.476+0000 7fae2f7fe640 1 -- 172.21.15.111:0/2757367120 shutdown_connections 2024-09-13T17:57:49.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.477+0000 7fae2f7fe640 1 -- 172.21.15.111:0/2757367120 wait complete. 2024-09-13T17:57:49.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.477+0000 7f4b17fff640 1 -- 172.21.15.111:0/1311279333 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4b0c0057d0 con 0x7f4b2c06dff0 2024-09-13T17:57:49.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.477+0000 7f0e18ff9640 1 -- 172.21.15.111:0/511730971 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0e14057ad0 con 0x7f0e1c0b2f70 2024-09-13T17:57:49.485 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-09-13T17:57:49.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.477+0000 7f4b227fc640 1 -- 172.21.15.111:0/1311279333 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4b1c0011b0 con 0x7f4b2c06dff0 2024-09-13T17:57:49.486 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-09-13T17:57:49.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.479+0000 7f31246f0640 1 -- 172.21.15.111:0/1865313293 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f310006ac00 msgr2=0x7f310006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.479+0000 7f31246f0640 1 --2- 172.21.15.111:0/1865313293 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f310006ac00 0x7f310006d0c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f310c00b810 tx=0x7f310c008040 comp rx=0 tx=0).stop 2024-09-13T17:57:49.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.479+0000 7f31246f0640 1 -- 172.21.15.111:0/1865313293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f311c0ad8f0 msgr2=0x7f311c158c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.479+0000 7f31246f0640 1 --2- 172.21.15.111:0/1865313293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f311c0ad8f0 0x7f311c158c80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3114009e80 tx=0x7f3114004a40 comp rx=0 tx=0).stop 2024-09-13T17:57:49.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.480+0000 7f31246f0640 1 -- 172.21.15.111:0/1865313293 shutdown_connections 2024-09-13T17:57:49.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.480+0000 7f31246f0640 1 --2- 172.21.15.111:0/1865313293 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f310006ac00 0x7f310006d0c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.480+0000 7f31246f0640 1 --2- 172.21.15.111:0/1865313293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f311c0b2f70 0x7f311c1591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.480+0000 7f31246f0640 1 --2- 172.21.15.111:0/1865313293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f311c0ad8f0 0x7f311c158c80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.480+0000 7f31246f0640 1 --2- 172.21.15.111:0/1865313293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f311c0ad490 0x7f311c071990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.480+0000 7f31246f0640 1 -- 172.21.15.111:0/1865313293 >> 172.21.15.111:0/1865313293 conn(0x7f311c06a0b0 msgr2=0x7f311c0addd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:49.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.480+0000 7f31246f0640 1 -- 172.21.15.111:0/1865313293 shutdown_connections 2024-09-13T17:57:49.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.481+0000 7f31246f0640 1 -- 172.21.15.111:0/1865313293 wait complete. 2024-09-13T17:57:49.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.481+0000 7f4b17fff640 1 -- 172.21.15.111:0/1311279333 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4b1006ac00 msgr2=0x7f4b1006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.481+0000 7f4b17fff640 1 --2- 172.21.15.111:0/1311279333 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4b1006ac00 0x7f4b1006d0c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4b28034000 tx=0x7f4b28030000 comp rx=0 tx=0).stop 2024-09-13T17:57:49.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.481+0000 7f4b17fff640 1 -- 172.21.15.111:0/1311279333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b2c06dff0 msgr2=0x7f4b2c1f5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.482+0000 7f0e22501640 1 -- 172.21.15.111:0/511730971 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0e0806ac30 msgr2=0x7f0e0806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.482+0000 7f0e22501640 1 --2- 172.21.15.111:0/511730971 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0e0806ac30 0x7f0e0806d0f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c000c00 tx=0x7f0e0c02e000 comp rx=0 tx=0).stop 2024-09-13T17:57:49.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.481+0000 7f4b17fff640 1 --2- 172.21.15.111:0/1311279333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b2c06dff0 0x7f4b2c1f5ba0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00c550 tx=0x7f4b1c00ca10 comp rx=0 tx=0).stop 2024-09-13T17:57:49.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.482+0000 7f0e22501640 1 -- 172.21.15.111:0/511730971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e1c0b2f70 msgr2=0x7f0e1c1fc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.482+0000 7f0e22501640 1 --2- 172.21.15.111:0/511730971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e1c0b2f70 0x7f0e1c1fc350 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400be00 tx=0x7f0e1400bef0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.481+0000 7f4b17fff640 1 -- 172.21.15.111:0/1311279333 shutdown_connections 2024-09-13T17:57:49.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.481+0000 7f4b17fff640 1 --2- 172.21.15.111:0/1311279333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b2c152910 0x7f4b2c14bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.482+0000 7f0e22501640 1 -- 172.21.15.111:0/511730971 shutdown_connections 2024-09-13T17:57:49.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.482+0000 7f0e22501640 1 --2- 172.21.15.111:0/511730971 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0e0806ac30 0x7f0e0806d0f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.482+0000 7f0e22501640 1 --2- 172.21.15.111:0/511730971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e1c0b2f70 0x7f0e1c1fc350 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.482+0000 7f0e22501640 1 --2- 172.21.15.111:0/511730971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e1c0d11b0 0x7f0e1c15ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.482+0000 7f0e22501640 1 --2- 172.21.15.111:0/511730971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e1c0b1750 0x7f0e1c1ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.482+0000 7f0e22501640 1 -- 172.21.15.111:0/511730971 >> 172.21.15.111:0/511730971 conn(0x7f0e1c09b830 msgr2=0x7f0e1c0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:49.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.481+0000 7f4b17fff640 1 --2- 172.21.15.111:0/1311279333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b2c0709b0 0x7f4b2c14b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.481+0000 7f4b17fff640 1 --2- 172.21.15.111:0/1311279333 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4b1006ac00 0x7f4b1006d0c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.481+0000 7f4b17fff640 1 --2- 172.21.15.111:0/1311279333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b2c06dff0 0x7f4b2c1f5ba0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.482+0000 7f4b17fff640 1 -- 172.21.15.111:0/1311279333 >> 172.21.15.111:0/1311279333 conn(0x7f4b2c0afc70 msgr2=0x7f4b2c0ce400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:49.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.482+0000 7f4b17fff640 1 -- 172.21.15.111:0/1311279333 shutdown_connections 2024-09-13T17:57:49.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.482+0000 7f4b17fff640 1 -- 172.21.15.111:0/1311279333 wait complete. 2024-09-13T17:57:49.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.482+0000 7f0e22501640 1 -- 172.21.15.111:0/511730971 shutdown_connections 2024-09-13T17:57:49.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.482+0000 7f0e22501640 1 -- 172.21.15.111:0/511730971 wait complete. 2024-09-13T17:57:49.495 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2024-09-13T17:57:49.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.486+0000 7fdbd0ff9640 1 -- 172.21.15.111:0/131335997 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fdbc4005500 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.486+0000 7fdbd2ffd640 1 -- 172.21.15.111:0/131335997 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdbcc074a90 con 0x7fdbdc0cb4d0 2024-09-13T17:57:49.496 INFO:teuthology.orchestra.run.smithi111.stdout:55834574856 2024-09-13T17:57:49.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.488+0000 7fdbd0ff9640 1 -- 172.21.15.111:0/131335997 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdbc006ac60 msgr2=0x7fdbc006d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.488+0000 7fdbd0ff9640 1 --2- 172.21.15.111:0/131335997 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdbc006ac60 0x7fdbc006d120 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8001560 tx=0x7fdbd8009040 comp rx=0 tx=0).stop 2024-09-13T17:57:49.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.488+0000 7fdbd0ff9640 1 -- 172.21.15.111:0/131335997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbdc0cb4d0 msgr2=0x7fdbdc148550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.488+0000 7fdbd0ff9640 1 --2- 172.21.15.111:0/131335997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbdc0cb4d0 0x7fdbdc148550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0b19e0 tx=0x7fdbcc0046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.489+0000 7fdbd0ff9640 1 -- 172.21.15.111:0/131335997 shutdown_connections 2024-09-13T17:57:49.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.489+0000 7fdbd0ff9640 1 --2- 172.21.15.111:0/131335997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbdc160740 0x7fdbdc162b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.489+0000 7fdbd0ff9640 1 --2- 172.21.15.111:0/131335997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbdc0cb4d0 0x7fdbdc148550 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.489+0000 7fdbd0ff9640 1 --2- 172.21.15.111:0/131335997 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdbc006ac60 0x7fdbc006d120 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.489+0000 7fdbd0ff9640 1 --2- 172.21.15.111:0/131335997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbdc0ada80 0x7fdbdc161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.489+0000 7fdbd0ff9640 1 -- 172.21.15.111:0/131335997 >> 172.21.15.111:0/131335997 conn(0x7fdbdc0b0100 msgr2=0x7fdbdc0ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:49.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.489+0000 7fdbd0ff9640 1 -- 172.21.15.111:0/131335997 shutdown_connections 2024-09-13T17:57:49.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.489+0000 7fdbd0ff9640 1 -- 172.21.15.111:0/131335997 wait complete. 2024-09-13T17:57:49.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.495+0000 7f1adcb18640 1 -- 172.21.15.111:0/627405051 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1ad8154540 con 0x7f1ad815cc20 2024-09-13T17:57:49.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.495+0000 7f1ac6ffd640 1 -- 172.21.15.111:0/627405051 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1ad8154540 con 0x7f1ad815cc20 2024-09-13T17:57:49.500 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2024-09-13T17:57:49.500 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-09-13T17:57:49.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.498+0000 7f1adcb18640 1 -- 172.21.15.111:0/627405051 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ab806ac60 msgr2=0x7f1ab806d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.498+0000 7f1adcb18640 1 --2- 172.21.15.111:0/627405051 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ab806ac60 0x7f1ab806d120 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8158500 tx=0x7f1acc007d30 comp rx=0 tx=0).stop 2024-09-13T17:57:49.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.498+0000 7f1adcb18640 1 -- 172.21.15.111:0/627405051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad815cc20 msgr2=0x7f1ad8152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:49.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.498+0000 7f1adcb18640 1 --2- 172.21.15.111:0/627405051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad815cc20 0x7f1ad8152270 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000caf0 tx=0x7f1ad0002090 comp rx=0 tx=0).stop 2024-09-13T17:57:49.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.498+0000 7f1adcb18640 1 -- 172.21.15.111:0/627405051 shutdown_connections 2024-09-13T17:57:49.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.498+0000 7f1adcb18640 1 --2- 172.21.15.111:0/627405051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad815cc20 0x7f1ad8152270 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.498+0000 7f1adcb18640 1 --2- 172.21.15.111:0/627405051 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1ab806ac60 0x7f1ab806d120 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.498+0000 7f1adcb18640 1 --2- 172.21.15.111:0/627405051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad813b790 0x7f1ad8154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.498+0000 7f1adcb18640 1 --2- 172.21.15.111:0/627405051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ad80cd4d0 0x7f1ad8151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:49.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.498+0000 7f1adcb18640 1 -- 172.21.15.111:0/627405051 >> 172.21.15.111:0/627405051 conn(0x7f1ad8069ed0 msgr2=0x7f1ad815ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:49.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.498+0000 7f1adcb18640 1 -- 172.21.15.111:0/627405051 shutdown_connections 2024-09-13T17:57:49.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:49.498+0000 7f1adcb18640 1 -- 172.21.15.111:0/627405051 wait complete. 2024-09-13T17:57:49.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2024-09-13T17:57:49.506 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.3 2024-09-13T17:57:49.507 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2024-09-13T17:57:49.508 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.7 2024-09-13T17:57:49.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-09-13T17:57:49.513 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2024-09-13T17:57:50.496 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T17:57:50.500 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T17:57:50.505 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T17:57:50.506 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T17:57:50.508 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T17:57:50.509 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T17:57:50.510 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T17:57:50.514 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T17:57:50.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.647+0000 7f132a50f640 1 Processor -- start 2024-09-13T17:57:50.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.647+0000 7f132a50f640 1 -- start start 2024-09-13T17:57:50.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.648+0000 7f132a50f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13240cfb30 0x7f132406b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.648+0000 7f132a50f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f132406b590 0x7f13240c3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.648+0000 7f132a50f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f13240c3980 0x7f13240afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.648+0000 7f132a50f640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1324099a30 con 0x7f132406b590 2024-09-13T17:57:50.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.648+0000 7f132a50f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13240c3d90 con 0x7f13240c3980 2024-09-13T17:57:50.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.648+0000 7f132a50f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13240cff10 con 0x7f13240cfb30 2024-09-13T17:57:50.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.649+0000 7f1323fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13240cfb30 0x7f132406b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.649+0000 7f1328aac640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f13240c3980 0x7f13240afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.649+0000 7f1328aac640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f13240c3980 0x7f13240afdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:49174/0 (socket says 172.21.15.111:49174) 2024-09-13T17:57:50.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.649+0000 7f1328aac640 1 -- 172.21.15.111:0/994518578 learned_addr learned my addr 172.21.15.111:0/994518578 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:50.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.649+0000 7f13237fe640 1 --2- 172.21.15.111:0/994518578 >> v2:172.21.15.111:3300/0 conn(0x7f132406b590 0x7f13240c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.649+0000 7f1328aac640 1 -- 172.21.15.111:0/994518578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13240cfb30 msgr2=0x7f132406b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:50.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.649+0000 7f1328aac640 1 --2- 172.21.15.111:0/994518578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13240cfb30 0x7f132406b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.649+0000 7f1328aac640 1 -- 172.21.15.111:0/994518578 >> v2:172.21.15.111:3300/0 conn(0x7f132406b590 msgr2=0x7f13240c3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:50.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.649+0000 7f1328aac640 1 --2- 172.21.15.111:0/994518578 >> v2:172.21.15.111:3300/0 conn(0x7f132406b590 0x7f13240c3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.649+0000 7f1328aac640 1 -- 172.21.15.111:0/994518578 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1324098db0 con 0x7f13240c3980 2024-09-13T17:57:50.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.650+0000 7f13237fe640 1 --2- 172.21.15.111:0/994518578 >> v2:172.21.15.111:3300/0 conn(0x7f132406b590 0x7f13240c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:50.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.652+0000 7f1328aac640 1 --2- 172.21.15.111:0/994518578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13240c3980 0x7f13240afdb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1324072200 tx=0x7f131c04c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed746736347a3d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.652+0000 7f1322ffd640 1 -- 172.21.15.111:0/994518578 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c053600 con 0x7f13240c3980 2024-09-13T17:57:50.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.652+0000 7f1322ffd640 1 -- 172.21.15.111:0/994518578 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131c04f050 con 0x7f13240c3980 2024-09-13T17:57:50.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.653+0000 7f132a50f640 1 -- 172.21.15.111:0/994518578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13240c3980 msgr2=0x7f13240afdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.653+0000 7f132a50f640 1 --2- 172.21.15.111:0/994518578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13240c3980 0x7f13240afdb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1324072200 tx=0x7f131c04c6d0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.653+0000 7f132a50f640 1 -- 172.21.15.111:0/994518578 shutdown_connections 2024-09-13T17:57:50.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.653+0000 7f132a50f640 1 --2- 172.21.15.111:0/994518578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13240c3980 0x7f13240afdb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.653+0000 7f132a50f640 1 --2- 172.21.15.111:0/994518578 >> v2:172.21.15.111:3300/0 conn(0x7f132406b590 0x7f13240c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.653+0000 7f132a50f640 1 --2- 172.21.15.111:0/994518578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13240cfb30 0x7f132406b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.653+0000 7f132a50f640 1 -- 172.21.15.111:0/994518578 >> 172.21.15.111:0/994518578 conn(0x7f13240b1f60 msgr2=0x7f13240b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:50.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.654+0000 7f132a50f640 1 -- 172.21.15.111:0/994518578 shutdown_connections 2024-09-13T17:57:50.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.654+0000 7f132a50f640 1 -- 172.21.15.111:0/994518578 wait complete. 2024-09-13T17:57:50.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.656+0000 7f132a50f640 1 Processor -- start 2024-09-13T17:57:50.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.656+0000 7f132a50f640 1 -- start start 2024-09-13T17:57:50.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.657+0000 7f132a50f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f132406b590 0x7f132409d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.658+0000 7f1323fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f132406b590 0x7f132409d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.658+0000 7f1323fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f132406b590 0x7f132409d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54136/0 (socket says 172.21.15.111:54136) 2024-09-13T17:57:50.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.658+0000 7f1323fff640 1 -- 172.21.15.111:0/2629714960 learned_addr learned my addr 172.21.15.111:0/2629714960 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:50.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.658+0000 7f132a50f640 1 --2- 172.21.15.111:0/2629714960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13240cfb30 0x7f13241d9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.658+0000 7f132a50f640 1 --2- 172.21.15.111:0/2629714960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13241d9cf0 0x7f132409a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.658+0000 7f132a50f640 1 -- 172.21.15.111:0/2629714960 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132409e010 con 0x7f132406b590 2024-09-13T17:57:50.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.658+0000 7f132a50f640 1 -- 172.21.15.111:0/2629714960 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132409da50 con 0x7f13241d9cf0 2024-09-13T17:57:50.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.658+0000 7f132a50f640 1 -- 172.21.15.111:0/2629714960 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132409dd30 con 0x7f13240cfb30 2024-09-13T17:57:50.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.658+0000 7f13237fe640 1 --2- 172.21.15.111:0/2629714960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13240cfb30 0x7f13241d9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.659+0000 7f1323fff640 1 -- 172.21.15.111:0/2629714960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13240cfb30 msgr2=0x7f13241d9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.659+0000 7f1323fff640 1 --2- 172.21.15.111:0/2629714960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13240cfb30 0x7f13241d9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.659+0000 7f1323fff640 1 -- 172.21.15.111:0/2629714960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13241d9cf0 msgr2=0x7f132409a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.659+0000 7f1323fff640 1 --2- 172.21.15.111:0/2629714960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13241d9cf0 0x7f132409a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.659+0000 7f1323fff640 1 -- 172.21.15.111:0/2629714960 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131c009b30 con 0x7f132406b590 2024-09-13T17:57:50.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.660+0000 7f1323fff640 1 --2- 172.21.15.111:0/2629714960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f132406b590 0x7f132409d370 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f131400ebe0 tx=0x7f131400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.661+0000 7f1320ff9640 1 -- 172.21.15.111:0/2629714960 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131400ed90 con 0x7f132406b590 2024-09-13T17:57:50.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.661+0000 7f132a50f640 1 -- 172.21.15.111:0/2629714960 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13240c51c0 con 0x7f132406b590 2024-09-13T17:57:50.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.661+0000 7f132a50f640 1 -- 172.21.15.111:0/2629714960 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132409b390 con 0x7f132406b590 2024-09-13T17:57:50.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.664+0000 7f1320ff9640 1 -- 172.21.15.111:0/2629714960 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131400cc30 con 0x7f132406b590 2024-09-13T17:57:50.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.664+0000 7f1320ff9640 1 -- 172.21.15.111:0/2629714960 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1314013470 con 0x7f132406b590 2024-09-13T17:57:50.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.664+0000 7f1320ff9640 1 -- 172.21.15.111:0/2629714960 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f13140135d0 con 0x7f132406b590 2024-09-13T17:57:50.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.665+0000 7f1320ff9640 1 --2- 172.21.15.111:0/2629714960 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f131006ad80 0x7f131006d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.665+0000 7f13237fe640 1 --2- 172.21.15.111:0/2629714960 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f131006ad80 0x7f131006d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.665+0000 7f1320ff9640 1 -- 172.21.15.111:0/2629714960 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1314089920 con 0x7f132406b590 2024-09-13T17:57:50.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.666+0000 7f13237fe640 1 --2- 172.21.15.111:0/2629714960 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f131006ad80 0x7f131006d240 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13240c3d60 tx=0x7f1318008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.666+0000 7f132a50f640 1 -- 172.21.15.111:0/2629714960 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1304005250 con 0x7f132406b590 2024-09-13T17:57:50.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.676+0000 7f1320ff9640 1 -- 172.21.15.111:0/2629714960 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f13140547f0 con 0x7f132406b590 2024-09-13T17:57:50.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.706+0000 7fba86a61640 1 Processor -- start 2024-09-13T17:57:50.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.706+0000 7fba86a61640 1 -- start start 2024-09-13T17:57:50.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.705+0000 7f162e5e2640 1 Processor -- start 2024-09-13T17:57:50.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.705+0000 7f162e5e2640 1 -- start start 2024-09-13T17:57:50.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.707+0000 7f162e5e2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1628169600 0x7f162816b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.707+0000 7f162e5e2640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16280adb50 con 0x7f16280ae880 2024-09-13T17:57:50.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.707+0000 7f162e5e2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16280af870 con 0x7f16280c1280 2024-09-13T17:57:50.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.707+0000 7f162e5e2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16280adda0 con 0x7f1628169600 2024-09-13T17:57:50.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.707+0000 7f16277fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f16280ae880 0x7f16280aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49818/0 (socket says 172.21.15.111:49818) 2024-09-13T17:57:50.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.707+0000 7f16277fe640 1 -- 172.21.15.111:0/3018963915 learned_addr learned my addr 172.21.15.111:0/3018963915 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:50.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.706+0000 7f1d64a5a640 1 Processor -- start 2024-09-13T17:57:50.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.707+0000 7f1d64a5a640 1 -- start start 2024-09-13T17:57:50.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.708+0000 7fba86a61640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba8015a610 0x7fba800b2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.708+0000 7fba86a61640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fba8015e170 0x7fba8015e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.708+0000 7fba86a61640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba800cc200 con 0x7fba80099790 2024-09-13T17:57:50.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.708+0000 7fba86a61640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba8015eb40 con 0x7fba8015e170 2024-09-13T17:57:50.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.708+0000 7fba86a61640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba800989d0 con 0x7fba8015a610 2024-09-13T17:57:50.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.708+0000 7f1626ffd640 1 -- 172.21.15.111:0/3018963915 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866220899 0 0) 0x7f16280adb50 con 0x7f16280ae880 2024-09-13T17:57:50.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.708+0000 7f1626ffd640 1 -- 172.21.15.111:0/3018963915 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1610003520 con 0x7f16280ae880 2024-09-13T17:57:50.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.707+0000 7f32d007d640 1 Processor -- start 2024-09-13T17:57:50.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.707+0000 7f32d007d640 1 -- start start 2024-09-13T17:57:50.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.708+0000 7f32d007d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f32c80c1280 0x7f32c80c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.708+0000 7f162cb7f640 1 --2- 172.21.15.111:0/3018963915 >> v2:172.21.15.188:3300/0 conn(0x7f1628169600 0x7f162816b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.708+0000 7f7cb5ad7640 1 Processor -- start 2024-09-13T17:57:50.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.708+0000 7f32d007d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c80adb50 con 0x7f32c80ae880 2024-09-13T17:57:50.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.708+0000 7f32d007d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c80c1ba0 con 0x7f32c80ab140 2024-09-13T17:57:50.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.708+0000 7f32d007d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c80af870 con 0x7f32c80c1280 2024-09-13T17:57:50.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.708+0000 7f32cde19640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f32c80c1280 0x7f32c80c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.708+0000 7f32cde19640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f32c80c1280 0x7f32c80c1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54152/0 (socket says 172.21.15.111:54152) 2024-09-13T17:57:50.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7f1d64a5a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1d6015bbd0 0x7f1d6015cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7f1d64a5a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1d600cc590 0x7f1d600adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7f1d64a5a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d60096ee0 con 0x7f1d60097370 2024-09-13T17:57:50.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7fba84ffe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fba8015e170 0x7fba8015e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7fba84ffe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fba8015e170 0x7fba8015e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54150/0 (socket says 172.21.15.111:54150) 2024-09-13T17:57:50.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7fba84ffe640 1 -- 172.21.15.111:0/3973336853 learned_addr learned my addr 172.21.15.111:0/3973336853 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:50.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7f1d64a5a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6015d0f0 con 0x7f1d600cc590 2024-09-13T17:57:50.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7fba7ffff640 1 --2- 172.21.15.111:0/3973336853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba8015a610 0x7fba800b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7f1d64a5a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6015bfb0 con 0x7f1d6015bbd0 2024-09-13T17:57:50.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7f1d5e59c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1d6015bbd0 0x7f1d6015cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7f1d5e59c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1d6015bbd0 0x7f1d6015cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:49206/0 (socket says 172.21.15.111:49206) 2024-09-13T17:57:50.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7f1d5e59c640 1 -- 172.21.15.111:0/88434114 learned_addr learned my addr 172.21.15.111:0/88434114 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:50.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7f1d5ed9d640 1 --2- 172.21.15.111:0/88434114 >> v2:172.21.15.111:3300/0 conn(0x7f1d600cc590 0x7f1d600adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7f1d5e59c640 1 -- 172.21.15.111:0/88434114 >> v2:172.21.15.111:3300/0 conn(0x7f1d600cc590 msgr2=0x7f1d600adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:50.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.708+0000 7f7cb5ad7640 1 -- start start 2024-09-13T17:57:50.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.710+0000 7f7cb5ad7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7ca8005a20 0x7f7ca8053f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.710+0000 7f7cb5ad7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7ca80e70e0 0x7f7ca8005e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.710+0000 7f7cb5ad7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca80520a0 con 0x7f7ca804eb20 2024-09-13T17:57:50.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.710+0000 7f7cb5ad7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca8006340 con 0x7f7ca80e70e0 2024-09-13T17:57:50.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.710+0000 7f7cb5ad7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca80522f0 con 0x7f7ca8005a20 2024-09-13T17:57:50.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.710+0000 7f7caeffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7ca80e70e0 0x7f7ca8005e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.710+0000 7f7caeffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7ca80e70e0 0x7f7ca8005e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54158/0 (socket says 172.21.15.111:54158) 2024-09-13T17:57:50.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.710+0000 7f7caeffd640 1 -- 172.21.15.111:0/1789727548 learned_addr learned my addr 172.21.15.111:0/1789727548 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:50.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.710+0000 7f7caf7fe640 1 --2- 172.21.15.111:0/1789727548 >> v2:172.21.15.188:3300/0 conn(0x7f7ca8005a20 0x7f7ca8053f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.708+0000 7f32cde19640 1 -- 172.21.15.111:0/1415979346 learned_addr learned my addr 172.21.15.111:0/1415979346 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:50.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.710+0000 7f32cde19640 1 -- 172.21.15.111:0/1415979346 >> v1:172.21.15.188:6789/0 conn(0x7f32c80ab140 legacy=0x7f32c80ab540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.710+0000 7f32cde19640 1 -- 172.21.15.111:0/1415979346 >> v1:172.21.15.111:6789/0 conn(0x7f32c80ae880 legacy=0x7f32c80aedd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:57:50.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.710+0000 7f32cde19640 1 -- 172.21.15.111:0/1415979346 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c80adda0 con 0x7f32c80c1280 2024-09-13T17:57:50.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7f162cb7f640 1 -- 172.21.15.111:0/3018963915 >> v1:172.21.15.188:6789/0 conn(0x7f16280c1280 legacy=0x7f16280c1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:50.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7f162cb7f640 1 -- 172.21.15.111:0/3018963915 >> v1:172.21.15.111:6789/0 conn(0x7f16280ae880 legacy=0x7f16280aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7f162cb7f640 1 -- 172.21.15.111:0/3018963915 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16280ae080 con 0x7f1628169600 2024-09-13T17:57:50.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7f1d5e59c640 1 --2- 172.21.15.111:0/88434114 >> v2:172.21.15.111:3300/0 conn(0x7f1d600cc590 0x7f1d600adc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7f1d5e59c640 1 -- 172.21.15.111:0/88434114 >> v1:172.21.15.111:6789/0 conn(0x7f1d60097370 legacy=0x7f1d600cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.709+0000 7f1d5e59c640 1 -- 172.21.15.111:0/88434114 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d600962d0 con 0x7f1d6015bbd0 2024-09-13T17:57:50.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.711+0000 7f32cde19640 1 --2- 172.21.15.111:0/1415979346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32c80c1280 0x7f32c80c1660 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009870 tx=0x7f32c402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77818480d240131e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.711+0000 7f1d5e59c640 1 --2- 172.21.15.111:0/88434114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d6015bbd0 0x7f1d6015cbb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1d60072590 tx=0x7f1d5802eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae6339b8af3ad8e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.711+0000 7f32cce17640 1 -- 172.21.15.111:0/1415979346 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c4032440 con 0x7f32c80c1280 2024-09-13T17:57:50.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.710+0000 7fba84ffe640 1 -- 172.21.15.111:0/3973336853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba8015a610 msgr2=0x7fba800b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:50.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.710+0000 7fba84ffe640 1 --2- 172.21.15.111:0/3973336853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba8015a610 0x7fba800b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.710+0000 7fba84ffe640 1 -- 172.21.15.111:0/3973336853 >> v1:172.21.15.111:6789/0 conn(0x7fba80099790 legacy=0x7fba800cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:50.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.710+0000 7fba84ffe640 1 -- 172.21.15.111:0/3973336853 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba80098cb0 con 0x7fba8015e170 2024-09-13T17:57:50.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.711+0000 7f1d5d59a640 1 -- 172.21.15.111:0/88434114 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5802f9f0 con 0x7f1d6015bbd0 2024-09-13T17:57:50.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.711+0000 7f1d5d59a640 1 -- 172.21.15.111:0/88434114 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d58033070 con 0x7f1d6015bbd0 2024-09-13T17:57:50.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.711+0000 7fba84ffe640 1 --2- 172.21.15.111:0/3973336853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba8015e170 0x7fba8015e600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fba70009870 tx=0x7fba7002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5773d3c78e6bfc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.711+0000 7f7caf7fe640 1 -- 172.21.15.111:0/1789727548 >> v2:172.21.15.111:3300/0 conn(0x7f7ca80e70e0 msgr2=0x7f7ca8005e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:50.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.711+0000 7f7caf7fe640 1 --2- 172.21.15.111:0/1789727548 >> v2:172.21.15.111:3300/0 conn(0x7f7ca80e70e0 0x7f7ca8005e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.711+0000 7f7caf7fe640 1 -- 172.21.15.111:0/1789727548 >> v1:172.21.15.188:6789/0 conn(0x7f7ca804eb20 legacy=0x7f7ca801c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.711+0000 7f7caf7fe640 1 -- 172.21.15.111:0/1789727548 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca8006c40 con 0x7f7ca8005a20 2024-09-13T17:57:50.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7fba7effd640 1 -- 172.21.15.111:0/3973336853 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7002ae90 con 0x7fba8015e170 2024-09-13T17:57:50.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.711+0000 7f356ffff640 1 Processor -- start 2024-09-13T17:57:50.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.711+0000 7f356ffff640 1 -- start start 2024-09-13T17:57:50.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.711+0000 7f32cce17640 1 -- 172.21.15.111:0/1415979346 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c40325a0 con 0x7f32c80c1280 2024-09-13T17:57:50.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.711+0000 7f32cce17640 1 -- 172.21.15.111:0/1415979346 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c4032930 con 0x7f32c80c1280 2024-09-13T17:57:50.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7fba7effd640 1 -- 172.21.15.111:0/3973336853 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba70032440 con 0x7fba8015e170 2024-09-13T17:57:50.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f7caf7fe640 1 --2- 172.21.15.111:0/1789727548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ca8005a20 0x7f7ca8053f10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8004930 tx=0x7f7ca402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57a489d188176b8e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f1d5d59a640 1 -- 172.21.15.111:0/88434114 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5802fdb0 con 0x7f1d6015bbd0 2024-09-13T17:57:50.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f356ffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f35700b0f70 0x7f35700b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f356ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35700b1900 0x7f357006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f356ffff640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35700723b0 con 0x7f35700973c0 2024-09-13T17:57:50.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f356ffff640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35700977a0 con 0x7f35700b0f70 2024-09-13T17:57:50.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f356ffff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357006c440 con 0x7f35700b1900 2024-09-13T17:57:50.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f162cb7f640 1 --2- 172.21.15.111:0/3018963915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1628169600 0x7f162816b9f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1620009d90 tx=0x7f162002a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd8cc4e6d375ef65 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f1d64a5a640 1 -- 172.21.15.111:0/88434114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d6015bbd0 msgr2=0x7f1d6015cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f1d64a5a640 1 --2- 172.21.15.111:0/88434114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d6015bbd0 0x7f1d6015cbb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1d60072590 tx=0x7f1d5802eca0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f7cae7fc640 1 -- 172.21.15.111:0/1789727548 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4032440 con 0x7f7ca8005a20 2024-09-13T17:57:50.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f1d64a5a640 1 -- 172.21.15.111:0/88434114 shutdown_connections 2024-09-13T17:57:50.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f7cae7fc640 1 -- 172.21.15.111:0/1789727548 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca40325a0 con 0x7f7ca8005a20 2024-09-13T17:57:50.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f1d64a5a640 1 --2- 172.21.15.111:0/88434114 >> v2:172.21.15.111:3300/0 conn(0x7f1d600cc590 0x7f1d600adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f1d64a5a640 1 --2- 172.21.15.111:0/88434114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d6015bbd0 0x7f1d6015cbb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f1d64a5a640 1 -- 172.21.15.111:0/88434114 >> 172.21.15.111:0/88434114 conn(0x7f1d600afe60 msgr2=0x7f1d600b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:50.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f1d64a5a640 1 -- 172.21.15.111:0/88434114 shutdown_connections 2024-09-13T17:57:50.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f1d64a5a640 1 -- 172.21.15.111:0/88434114 wait complete. 2024-09-13T17:57:50.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f356effd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f35700973c0 0x7f35700cd780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49830/0 (socket says 172.21.15.111:49830) 2024-09-13T17:57:50.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7fba86a61640 1 -- 172.21.15.111:0/3973336853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba8015e170 msgr2=0x7fba8015e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.712+0000 7f356effd640 1 -- 172.21.15.111:0/1503096616 learned_addr learned my addr 172.21.15.111:0/1503096616 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:50.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f356f7fe640 1 --2- 172.21.15.111:0/1503096616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35700b1900 0x7f357006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f356e7fc640 1 --2- 172.21.15.111:0/1503096616 >> v2:172.21.15.188:3300/0 conn(0x7f35700b0f70 0x7f35700b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f356dffb640 1 -- 172.21.15.111:0/1503096616 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3771339029 0 0) 0x7f35700723b0 con 0x7f35700973c0 2024-09-13T17:57:50.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f32d007d640 1 -- 172.21.15.111:0/1415979346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32c80c1280 msgr2=0x7f32c80c1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f32d007d640 1 --2- 172.21.15.111:0/1415979346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32c80c1280 0x7f32c80c1660 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009870 tx=0x7f32c402a2a0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f356dffb640 1 -- 172.21.15.111:0/1503096616 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3558003520 con 0x7f35700973c0 2024-09-13T17:57:50.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f32d007d640 1 -- 172.21.15.111:0/1415979346 shutdown_connections 2024-09-13T17:57:50.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f32d007d640 1 --2- 172.21.15.111:0/1415979346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32c80c1280 0x7f32c80c1660 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f32d007d640 1 -- 172.21.15.111:0/1415979346 >> 172.21.15.111:0/1415979346 conn(0x7f32c80afe60 msgr2=0x7f32c80b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:50.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7fba86a61640 1 --2- 172.21.15.111:0/3973336853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba8015e170 0x7fba8015e600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fba70009870 tx=0x7fba7002a180 comp rx=0 tx=0).stop 2024-09-13T17:57:50.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f7cae7fc640 1 -- 172.21.15.111:0/1789727548 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4032910 con 0x7f7ca8005a20 2024-09-13T17:57:50.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f7cb5ad7640 1 -- 172.21.15.111:0/1789727548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ca8005a20 msgr2=0x7f7ca8053f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f7cb5ad7640 1 --2- 172.21.15.111:0/1789727548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ca8005a20 0x7f7ca8053f10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8004930 tx=0x7f7ca402a250 comp rx=0 tx=0).stop 2024-09-13T17:57:50.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7fba86a61640 1 -- 172.21.15.111:0/3973336853 shutdown_connections 2024-09-13T17:57:50.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7fba86a61640 1 --2- 172.21.15.111:0/3973336853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba8015e170 0x7fba8015e600 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7fba86a61640 1 --2- 172.21.15.111:0/3973336853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba8015a610 0x7fba800b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7fba86a61640 1 -- 172.21.15.111:0/3973336853 >> 172.21.15.111:0/3973336853 conn(0x7fba800b1df0 msgr2=0x7fba800b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:50.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f356dffb640 1 -- 172.21.15.111:0/1503096616 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3601589391 0 0) 0x7f3558003520 con 0x7f35700973c0 2024-09-13T17:57:50.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f356dffb640 1 -- 172.21.15.111:0/1503096616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35700b1900 msgr2=0x7f357006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:50.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f356dffb640 1 --2- 172.21.15.111:0/1503096616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35700b1900 0x7f357006c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f7cb5ad7640 1 -- 172.21.15.111:0/1789727548 shutdown_connections 2024-09-13T17:57:50.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f7cb5ad7640 1 --2- 172.21.15.111:0/1789727548 >> v2:172.21.15.111:3300/0 conn(0x7f7ca80e70e0 0x7f7ca8005e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f7cb5ad7640 1 --2- 172.21.15.111:0/1789727548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ca8005a20 0x7f7ca8053f10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f7cb5ad7640 1 -- 172.21.15.111:0/1789727548 >> 172.21.15.111:0/1789727548 conn(0x7f7ca8036050 msgr2=0x7f7ca8036460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:50.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f356dffb640 1 -- 172.21.15.111:0/1503096616 >> v2:172.21.15.188:3300/0 conn(0x7f35700b0f70 msgr2=0x7f35700b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:50.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f356dffb640 1 --2- 172.21.15.111:0/1503096616 >> v2:172.21.15.188:3300/0 conn(0x7f35700b0f70 0x7f35700b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f356dffb640 1 -- 172.21.15.111:0/1503096616 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35700720d0 con 0x7f35700973c0 2024-09-13T17:57:50.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f1626ffd640 1 -- 172.21.15.111:0/3018963915 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162002aea0 con 0x7f1628169600 2024-09-13T17:57:50.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f1626ffd640 1 -- 172.21.15.111:0/3018963915 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162002d050 con 0x7f1628169600 2024-09-13T17:57:50.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f1626ffd640 1 -- 172.21.15.111:0/3018963915 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16200079a0 con 0x7f1628169600 2024-09-13T17:57:50.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7fba86a61640 1 -- 172.21.15.111:0/3973336853 shutdown_connections 2024-09-13T17:57:50.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f356e7fc640 1 --2- 172.21.15.111:0/1503096616 >> v2:172.21.15.188:3300/0 conn(0x7f35700b0f70 0x7f35700b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:50.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.713+0000 7f7cb5ad7640 1 -- 172.21.15.111:0/1789727548 shutdown_connections 2024-09-13T17:57:50.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f7cb5ad7640 1 -- 172.21.15.111:0/1789727548 wait complete. 2024-09-13T17:57:50.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f356dffb640 1 -- 172.21.15.111:0/1503096616 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f35600019f0 con 0x7f35700973c0 2024-09-13T17:57:50.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f356dffb640 1 -- 172.21.15.111:0/1503096616 >> v1:172.21.15.111:6789/0 conn(0x7f35700973c0 legacy=0x7f35700cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f356dffb640 1 --2- 172.21.15.111:0/1503096616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3558003970 0x7f3558003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f356dffb640 1 --2- 172.21.15.111:0/1503096616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35700b1900 0x7f35580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f356dffb640 1 --2- 172.21.15.111:0/1503096616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3558004d00 0x7f3558005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f162e5e2640 1 -- 172.21.15.111:0/3018963915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1628169600 msgr2=0x7f162816b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f162e5e2640 1 --2- 172.21.15.111:0/3018963915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1628169600 0x7f162816b9f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1620009d90 tx=0x7f162002a6c0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f162e5e2640 1 -- 172.21.15.111:0/3018963915 shutdown_connections 2024-09-13T17:57:50.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f162e5e2640 1 --2- 172.21.15.111:0/3018963915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1628169600 0x7f162816b9f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f162e5e2640 1 -- 172.21.15.111:0/3018963915 >> 172.21.15.111:0/3018963915 conn(0x7f16280afe60 msgr2=0x7f16280b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:50.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f356effd640 1 --2- 172.21.15.111:0/1503096616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3558003970 0x7f3558003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f356dffb640 1 -- 172.21.15.111:0/1503096616 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357006c440 con 0x7f3558003970 2024-09-13T17:57:50.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f356e7fc640 1 --2- 172.21.15.111:0/1503096616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35700b1900 0x7f35580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f356dffb640 1 -- 172.21.15.111:0/1503096616 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355800ba80 con 0x7f3558004d00 2024-09-13T17:57:50.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f32d007d640 1 -- 172.21.15.111:0/1415979346 shutdown_connections 2024-09-13T17:57:50.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f32d007d640 1 -- 172.21.15.111:0/1415979346 wait complete. 2024-09-13T17:57:50.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7fba86a61640 1 -- 172.21.15.111:0/3973336853 wait complete. 2024-09-13T17:57:50.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f356dffb640 1 -- 172.21.15.111:0/1503096616 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355800bc40 con 0x7f35700b1900 2024-09-13T17:57:50.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.714+0000 7f356f7fe640 1 --2- 172.21.15.111:0/1503096616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3558004d00 0x7f3558005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.715+0000 7f162e5e2640 1 -- 172.21.15.111:0/3018963915 shutdown_connections 2024-09-13T17:57:50.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.715+0000 7f7cb5ad7640 1 Processor -- start 2024-09-13T17:57:50.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.715+0000 7f162e5e2640 1 -- 172.21.15.111:0/3018963915 wait complete. 2024-09-13T17:57:50.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.715+0000 7f7cb5ad7640 1 -- start start 2024-09-13T17:57:50.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.716+0000 7f1d64a5a640 1 Processor -- start 2024-09-13T17:57:50.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.716+0000 7f356effd640 1 -- 172.21.15.111:0/1503096616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35700b1900 msgr2=0x7f35580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:50.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.716+0000 7f356effd640 1 --2- 172.21.15.111:0/1503096616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35700b1900 0x7f35580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.716+0000 7f356effd640 1 -- 172.21.15.111:0/1503096616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3558004d00 msgr2=0x7f3558005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:50.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.716+0000 7f356effd640 1 --2- 172.21.15.111:0/1503096616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3558004d00 0x7f3558005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.716+0000 7f356effd640 1 -- 172.21.15.111:0/1503096616 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3558003520 con 0x7f3558003970 2024-09-13T17:57:50.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.715+0000 7fba86a61640 1 Processor -- start 2024-09-13T17:57:50.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.716+0000 7fba86a61640 1 -- start start 2024-09-13T17:57:50.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.716+0000 7f32d007d640 1 Processor -- start 2024-09-13T17:57:50.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.716+0000 7f32d007d640 1 -- start start 2024-09-13T17:57:50.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.716+0000 7f1d64a5a640 1 -- start start 2024-09-13T17:57:50.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f7cb5ad7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ca804eb20 0x7f7ca817be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f162e5e2640 1 Processor -- start 2024-09-13T17:57:50.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f7cb5ad7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ca80e70e0 0x7f7ca8048000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f7cb5ad7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ca8048540 0x7f7ca804f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f7cb5ad7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca817c3d0 con 0x7f7ca80e70e0 2024-09-13T17:57:50.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f7cb5ad7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca804aeb0 con 0x7f7ca8048540 2024-09-13T17:57:50.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f7cb5ad7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8049e10 con 0x7f7ca804eb20 2024-09-13T17:57:50.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f7caffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ca8048540 0x7f7ca804f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f7caffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ca8048540 0x7f7ca804f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:49246/0 (socket says 172.21.15.111:49246) 2024-09-13T17:57:50.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f7caffff640 1 -- 172.21.15.111:0/459104015 learned_addr learned my addr 172.21.15.111:0/459104015 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:50.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f7caeffd640 1 --2- 172.21.15.111:0/459104015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ca80e70e0 0x7f7ca8048000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f162e5e2640 1 -- start start 2024-09-13T17:57:50.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f356effd640 1 --2- 172.21.15.111:0/1503096616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3558003970 0x7f3558003d90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f356000a760 tx=0x7f35580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b9d54e81e02fa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f1d64a5a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d60097370 0x7f1d600c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f7caf7fe640 1 --2- 172.21.15.111:0/459104015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ca804eb20 0x7f7ca817be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f1d64a5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d600cc590 0x7f1d600bb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f1d64a5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d6015bbd0 0x7f1d601548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f1d64a5a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d600b2e10 con 0x7f1d60097370 2024-09-13T17:57:50.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f1d64a5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d60095a40 con 0x7f1d6015bbd0 2024-09-13T17:57:50.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f1d64a5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d60154450 con 0x7f1d600cc590 2024-09-13T17:57:50.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f7caffff640 1 -- 172.21.15.111:0/459104015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ca804eb20 msgr2=0x7f7ca817be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f32d007d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32c80ab140 0x7f32c80999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f32d007d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32c80ae880 0x7f32c81e84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f32d007d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32c80c1280 0x7f32c8154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f32cde19640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32c80ab140 0x7f32c80999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7fba86a61640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba80099790 0x7fba800c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7fba86a61640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba8015a610 0x7fba800c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7fba86a61640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba8015e170 0x7fba800c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7fba86a61640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba800c3e30 con 0x7fba8015e170 2024-09-13T17:57:50.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f32cde19640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32c80ab140 0x7f32c80999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:49264/0 (socket says 172.21.15.111:49264) 2024-09-13T17:57:50.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f32cd618640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32c80ae880 0x7f32c81e84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f32cd618640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32c80ae880 0x7f32c81e84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:34672/0 (socket says 172.21.15.111:34672) 2024-09-13T17:57:50.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f32d007d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81e68c0 con 0x7f32c80c1280 2024-09-13T17:57:50.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f32d007d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81e5cd0 con 0x7f32c80ab140 2024-09-13T17:57:50.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f32d007d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81e5f20 con 0x7f32c80ae880 2024-09-13T17:57:50.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f32cde19640 1 -- 172.21.15.111:0/3006024453 learned_addr learned my addr 172.21.15.111:0/3006024453 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:50.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7fba86a61640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba800b51b0 con 0x7fba80099790 2024-09-13T17:57:50.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f7caffff640 1 --2- 172.21.15.111:0/459104015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ca804eb20 0x7f7ca817be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f7caffff640 1 -- 172.21.15.111:0/459104015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ca80e70e0 msgr2=0x7f7ca8048000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f7caffff640 1 --2- 172.21.15.111:0/459104015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ca80e70e0 0x7f7ca8048000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7f7caffff640 1 -- 172.21.15.111:0/459104015 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca40093a0 con 0x7f7ca8048540 2024-09-13T17:57:50.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f1d5ed9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d6015bbd0 0x7f1d601548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f1d5ed9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d6015bbd0 0x7f1d601548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:49266/0 (socket says 172.21.15.111:49266) 2024-09-13T17:57:50.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f1d5ed9d640 1 -- 172.21.15.111:0/1387165094 learned_addr learned my addr 172.21.15.111:0/1387165094 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:50.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f7caf7fe640 1 --2- 172.21.15.111:0/459104015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ca804eb20 0x7f7ca817be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:50.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7fba86a61640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba800b5490 con 0x7fba8015a610 2024-09-13T17:57:50.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7fba7ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba80099790 0x7fba800c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7fba7ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba80099790 0x7fba800c2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:49254/0 (socket says 172.21.15.111:49254) 2024-09-13T17:57:50.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7fba7ffff640 1 -- 172.21.15.111:0/4151516486 learned_addr learned my addr 172.21.15.111:0/4151516486 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:50.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7fba7f7fe640 1 --2- 172.21.15.111:0/4151516486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba8015a610 0x7fba800c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.717+0000 7fba84ffe640 1 --2- 172.21.15.111:0/4151516486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba8015e170 0x7fba800c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f7caeffd640 1 --2- 172.21.15.111:0/459104015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ca80e70e0 0x7f7ca8048000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:50.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f32ce61a640 1 --2- 172.21.15.111:0/3006024453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32c80c1280 0x7f32c8154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7fa26759e640 1 Processor -- start 2024-09-13T17:57:50.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7fba7ffff640 1 -- 172.21.15.111:0/4151516486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba8015a610 msgr2=0x7fba800c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7fba7ffff640 1 --2- 172.21.15.111:0/4151516486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba8015a610 0x7fba800c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7fba7ffff640 1 -- 172.21.15.111:0/4151516486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba8015e170 msgr2=0x7fba800c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7fba7ffff640 1 --2- 172.21.15.111:0/4151516486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba8015e170 0x7fba800c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7fba7ffff640 1 -- 172.21.15.111:0/4151516486 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba700094d0 con 0x7fba80099790 2024-09-13T17:57:50.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7fa26759e640 1 -- start start 2024-09-13T17:57:50.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f1d5e59c640 1 --2- 172.21.15.111:0/1387165094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d60097370 0x7f1d600c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f1d5ed9d640 1 -- 172.21.15.111:0/1387165094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d600cc590 msgr2=0x7f1d600bb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f1d5dd9b640 1 --2- 172.21.15.111:0/1387165094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d600cc590 0x7f1d600bb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f1d5ed9d640 1 --2- 172.21.15.111:0/1387165094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d600cc590 0x7f1d600bb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f1d5ed9d640 1 -- 172.21.15.111:0/1387165094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d60097370 msgr2=0x7f1d600c1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f1d5ed9d640 1 --2- 172.21.15.111:0/1387165094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d60097370 0x7f1d600c1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f7caffff640 1 --2- 172.21.15.111:0/459104015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ca8048540 0x7f7ca804f830 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0065a00 tx=0x7f7cb0067190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7fba7ffff640 1 --2- 172.21.15.111:0/4151516486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba80099790 0x7fba800c2a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba78007e60 tx=0x7fba7800e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f1d5ed9d640 1 -- 172.21.15.111:0/1387165094 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d58007790 con 0x7f1d6015bbd0 2024-09-13T17:57:50.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f1d5dd9b640 1 --2- 172.21.15.111:0/1387165094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d600cc590 0x7f1d600bb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:50.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f1d5e59c640 1 --2- 172.21.15.111:0/1387165094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d60097370 0x7f1d600c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:50.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f7c9bfff640 1 -- 172.21.15.111:0/459104015 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb006f070 con 0x7f7ca8048540 2024-09-13T17:57:50.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f7c9bfff640 1 -- 172.21.15.111:0/459104015 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb0073520 con 0x7f7ca8048540 2024-09-13T17:57:50.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f7cb5ad7640 1 -- 172.21.15.111:0/459104015 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca804a0f0 con 0x7f7ca8048540 2024-09-13T17:57:50.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f356dffb640 1 -- 172.21.15.111:0/1503096616 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356002b990 con 0x7f3558003970 2024-09-13T17:57:50.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.719+0000 7f356dffb640 1 -- 172.21.15.111:0/1503096616 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356002bb10 con 0x7f3558003970 2024-09-13T17:57:50.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.719+0000 7f162e5e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16280ae880 0x7f162815c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.719+0000 7f162e5e2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16280c1280 0x7f1628201a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.719+0000 7f32cd618640 1 -- 172.21.15.111:0/3006024453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32c80ab140 msgr2=0x7f32c80999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.719+0000 7fba7cff9640 1 -- 172.21.15.111:0/4151516486 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78014620 con 0x7fba80099790 2024-09-13T17:57:50.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.719+0000 7fba7cff9640 1 -- 172.21.15.111:0/4151516486 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7800ee90 con 0x7fba80099790 2024-09-13T17:57:50.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.719+0000 7fba7cff9640 1 -- 172.21.15.111:0/4151516486 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba780149a0 con 0x7fba80099790 2024-09-13T17:57:50.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.719+0000 7f162e5e2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1628169600 0x7f162815eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.719+0000 7f32cd618640 1 --2- 172.21.15.111:0/3006024453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32c80ab140 0x7f32c80999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.719+0000 7f32cd618640 1 -- 172.21.15.111:0/3006024453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32c80c1280 msgr2=0x7f32c8154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.719+0000 7f32cd618640 1 --2- 172.21.15.111:0/3006024453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32c80c1280 0x7f32c8154ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.719+0000 7f32cd618640 1 -- 172.21.15.111:0/3006024453 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c40094d0 con 0x7f32c80ae880 2024-09-13T17:57:50.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.719+0000 7f32ce61a640 1 --2- 172.21.15.111:0/3006024453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32c80c1280 0x7f32c8154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:50.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.718+0000 7f7cb5ad7640 1 -- 172.21.15.111:0/459104015 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca804a6b0 con 0x7f7ca8048540 2024-09-13T17:57:50.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.719+0000 7f7c9bfff640 1 -- 172.21.15.111:0/459104015 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0092420 con 0x7f7ca8048540 2024-09-13T17:57:50.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.719+0000 7f32cd618640 1 --2- 172.21.15.111:0/3006024453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32c80ae880 0x7f32c81e84d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f32c000be20 tx=0x7f32c0009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.719+0000 7f162cb7f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1628169600 0x7f162815eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.719+0000 7f356dffb640 1 -- 172.21.15.111:0/1503096616 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3560034470 con 0x7f3558003970 2024-09-13T17:57:50.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f162cb7f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1628169600 0x7f162815eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54216/0 (socket says 172.21.15.111:54216) 2024-09-13T17:57:50.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f16277fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16280c1280 0x7f1628201a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f356ffff640 1 -- 172.21.15.111:0/1503096616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3558003970 msgr2=0x7f3558003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f1d5ed9d640 1 --2- 172.21.15.111:0/1387165094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d6015bbd0 0x7f1d601548e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000e960 tx=0x7f1d5000ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f356ffff640 1 --2- 172.21.15.111:0/1503096616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3558003970 0x7f3558003d90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f356000a760 tx=0x7f35580057f0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f356f7fe640 1 -- 172.21.15.111:0/1503096616 reap_dead start 2024-09-13T17:57:50.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f16277fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16280c1280 0x7f1628201a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:34684/0 (socket says 172.21.15.111:34684) 2024-09-13T17:57:50.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f7c99ffb640 1 -- 172.21.15.111:0/459104015 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca8047b10 con 0x7f7ca8048540 2024-09-13T17:57:50.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f356ffff640 1 -- 172.21.15.111:0/1503096616 shutdown_connections 2024-09-13T17:57:50.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f356ffff640 1 -- 172.21.15.111:0/1503096616 >> 172.21.15.111:0/1503096616 conn(0x7f3570070440 msgr2=0x7f35700afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:50.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7fba86a61640 1 -- 172.21.15.111:0/4151516486 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba800b5770 con 0x7fba80099790 2024-09-13T17:57:50.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f356ffff640 1 -- 172.21.15.111:0/1503096616 shutdown_connections 2024-09-13T17:57:50.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f356ffff640 1 -- 172.21.15.111:0/1503096616 wait complete. 2024-09-13T17:57:50.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7fa26759e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa260036430 0x7fa26000a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7fa26759e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa26004ef70 0x7fa26000bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f1627fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16280ae880 0x7f162815c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f1627fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16280ae880 0x7f162815c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:49274/0 (socket says 172.21.15.111:49274) 2024-09-13T17:57:50.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f162e5e2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162815f5f0 con 0x7f1628169600 2024-09-13T17:57:50.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f162e5e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1628201fd0 con 0x7f16280ae880 2024-09-13T17:57:50.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f162e5e2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162815f310 con 0x7f16280c1280 2024-09-13T17:57:50.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f16277fe640 1 -- 172.21.15.111:0/2909140432 learned_addr learned my addr 172.21.15.111:0/2909140432 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:50.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7fa26759e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa2600550c0 0x7fa260036810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7fa26759e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26000c0c0 con 0x7fa2600550c0 2024-09-13T17:57:50.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7fa266d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa2600550c0 0x7fa260036810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7fa266d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa2600550c0 0x7fa260036810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54218/0 (socket says 172.21.15.111:54218) 2024-09-13T17:57:50.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7fa266d9d640 1 -- 172.21.15.111:0/415597878 learned_addr learned my addr 172.21.15.111:0/415597878 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:50.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7fa26759e640 1 -- 172.21.15.111:0/415597878 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa260007890 con 0x7fa26004ef70 2024-09-13T17:57:50.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7fa26759e640 1 -- 172.21.15.111:0/415597878 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600067f0 con 0x7fa260036430 2024-09-13T17:57:50.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7fba86a61640 1 -- 172.21.15.111:0/4151516486 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba800c2f50 con 0x7fba80099790 2024-09-13T17:57:50.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7fba86a61640 1 -- 172.21.15.111:0/4151516486 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba800c42c0 con 0x7fba80099790 2024-09-13T17:57:50.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7f7c9bfff640 1 -- 172.21.15.111:0/459104015 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7cb008f020 con 0x7f7ca8048540 2024-09-13T17:57:50.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.722+0000 7f7c9bfff640 1 --2- 172.21.15.111:0/459104015 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7c9406ac50 0x7f7c9406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.722+0000 7f7c9bfff640 1 -- 172.21.15.111:0/459104015 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7cb0102da0 con 0x7f7ca8048540 2024-09-13T17:57:50.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.722+0000 7f7caf7fe640 1 --2- 172.21.15.111:0/459104015 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7c9406ac50 0x7f7c9406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f1d4effd640 1 -- 172.21.15.111:0/1387165094 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d50009690 con 0x7f1d6015bbd0 2024-09-13T17:57:50.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f1d4effd640 1 -- 172.21.15.111:0/1387165094 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5000c980 con 0x7f1d6015bbd0 2024-09-13T17:57:50.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f1d4effd640 1 -- 172.21.15.111:0/1387165094 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5000cd30 con 0x7f1d6015bbd0 2024-09-13T17:57:50.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7f1d64a5a640 1 -- 172.21.15.111:0/1387165094 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d601559a0 con 0x7f1d6015bbd0 2024-09-13T17:57:50.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7f1d64a5a640 1 -- 172.21.15.111:0/1387165094 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d601540e0 con 0x7f1d6015bbd0 2024-09-13T17:57:50.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f32be7fc640 1 -- 172.21.15.111:0/3006024453 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0014440 con 0x7f32c80ae880 2024-09-13T17:57:50.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f32be7fc640 1 -- 172.21.15.111:0/3006024453 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c00145a0 con 0x7f32c80ae880 2024-09-13T17:57:50.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.720+0000 7f32be7fc640 1 -- 172.21.15.111:0/3006024453 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0014930 con 0x7f32c80ae880 2024-09-13T17:57:50.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7f32d007d640 1 -- 172.21.15.111:0/3006024453 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c81e6200 con 0x7f32c80ae880 2024-09-13T17:57:50.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.722+0000 7f32d007d640 1 -- 172.21.15.111:0/3006024453 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c81e8a10 con 0x7f32c80ae880 2024-09-13T17:57:50.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7f16277fe640 1 -- 172.21.15.111:0/2909140432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16280ae880 msgr2=0x7f162815c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7f16277fe640 1 --2- 172.21.15.111:0/2909140432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16280ae880 0x7f162815c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7f16277fe640 1 -- 172.21.15.111:0/2909140432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1628169600 msgr2=0x7f162815eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7f16277fe640 1 --2- 172.21.15.111:0/2909140432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1628169600 0x7f162815eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7f162cb7f640 1 --2- 172.21.15.111:0/2909140432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1628169600 0x7f162815eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-13T17:57:50.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7f16277fe640 1 -- 172.21.15.111:0/2909140432 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1620009a70 con 0x7f16280c1280 2024-09-13T17:57:50.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7f1627fff640 1 --2- 172.21.15.111:0/2909140432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16280ae880 0x7f162815c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-13T17:57:50.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.722+0000 7f16277fe640 1 --2- 172.21.15.111:0/2909140432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16280c1280 0x7f1628201a90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f161800c430 tx=0x7f161800c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.722+0000 7f1624ff9640 1 -- 172.21.15.111:0/2909140432 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1618005ce0 con 0x7f16280c1280 2024-09-13T17:57:50.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.722+0000 7f162e5e2640 1 -- 172.21.15.111:0/2909140432 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16280b17d0 con 0x7f16280c1280 2024-09-13T17:57:50.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.722+0000 7f162e5e2640 1 -- 172.21.15.111:0/2909140432 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16280b1d90 con 0x7f16280c1280 2024-09-13T17:57:50.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.722+0000 7f1624ff9640 1 -- 172.21.15.111:0/2909140432 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1618005e60 con 0x7f16280c1280 2024-09-13T17:57:50.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.722+0000 7f1624ff9640 1 -- 172.21.15.111:0/2909140432 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1618007310 con 0x7f16280c1280 2024-09-13T17:57:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7fa26659c640 1 --2- 172.21.15.111:0/415597878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa260036430 0x7fa26000a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.721+0000 7fa265d9b640 1 --2- 172.21.15.111:0/415597878 >> v2:172.21.15.188:3300/0 conn(0x7fa26004ef70 0x7fa26000bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.722+0000 7fa265d9b640 1 -- 172.21.15.111:0/415597878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa260036430 msgr2=0x7fa26000a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.722+0000 7fa265d9b640 1 --2- 172.21.15.111:0/415597878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa260036430 0x7fa26000a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.722+0000 7fa265d9b640 1 -- 172.21.15.111:0/415597878 >> v2:172.21.15.111:3300/0 conn(0x7fa2600550c0 msgr2=0x7fa260036810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.722+0000 7fa265d9b640 1 --2- 172.21.15.111:0/415597878 >> v2:172.21.15.111:3300/0 conn(0x7fa2600550c0 0x7fa260036810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.722+0000 7fa265d9b640 1 -- 172.21.15.111:0/415597878 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa260006d50 con 0x7fa26004ef70 2024-09-13T17:57:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.723+0000 7fa266d9d640 1 --2- 172.21.15.111:0/415597878 >> v2:172.21.15.111:3300/0 conn(0x7fa2600550c0 0x7fa260036810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:50.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.722+0000 7f356ffff640 1 Processor -- start 2024-09-13T17:57:50.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.723+0000 7f356ffff640 1 -- start start 2024-09-13T17:57:50.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.723+0000 7f7caf7fe640 1 --2- 172.21.15.111:0/459104015 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7c9406ac50 0x7f7c9406d110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4034000 tx=0x7f7ca402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.723+0000 7f32d007d640 1 -- 172.21.15.111:0/3006024453 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c81e6a00 con 0x7f32c80ae880 2024-09-13T17:57:50.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.723+0000 7f1d64a5a640 1 -- 172.21.15.111:0/1387165094 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d601545e0 con 0x7f1d6015bbd0 2024-09-13T17:57:50.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.723+0000 7fa265d9b640 1 --2- 172.21.15.111:0/415597878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa26004ef70 0x7fa26000bb80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa26000f060 tx=0x7fa258029ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6389f5265e8b193 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.724+0000 7f32be7fc640 1 -- 172.21.15.111:0/3006024453 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f32c0014a90 con 0x7f32c80ae880 2024-09-13T17:57:50.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.724+0000 7f162e5e2640 1 -- 172.21.15.111:0/2909140432 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1608005250 con 0x7f16280c1280 2024-09-13T17:57:50.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.724+0000 7f356ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3558003970 0x7f357008cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.724+0000 7f32be7fc640 1 --2- 172.21.15.111:0/3006024453 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f32ac06ac50 0x7f32ac06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.724+0000 7f1624ff9640 1 -- 172.21.15.111:0/2909140432 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1618007470 con 0x7f16280c1280 2024-09-13T17:57:50.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.724+0000 7f356ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3570166010 0x7f3570166490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.724+0000 7f356ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357008ff30 0x7f35700903e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.724+0000 7f32be7fc640 1 -- 172.21.15.111:0/3006024453 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f32c0089740 con 0x7f32c80ae880 2024-09-13T17:57:50.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.724+0000 7f356ffff640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35702032a0 con 0x7f3570166010 2024-09-13T17:57:50.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.724+0000 7f32cde19640 1 --2- 172.21.15.111:0/3006024453 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f32ac06ac50 0x7f32ac06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.724+0000 7f356ffff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3570090eb0 con 0x7f3558003970 2024-09-13T17:57:50.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.724+0000 7f356ffff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35701669d0 con 0x7f357008ff30 2024-09-13T17:57:50.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.724+0000 7f356effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3558003970 0x7f357008cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.724+0000 7f356effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3558003970 0x7f357008cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:49298/0 (socket says 172.21.15.111:49298) 2024-09-13T17:57:50.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.724+0000 7f356effd640 1 -- 172.21.15.111:0/3197311759 learned_addr learned my addr 172.21.15.111:0/3197311759 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:50.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.725+0000 7f356e7fc640 1 --2- 172.21.15.111:0/3197311759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3570166010 0x7f3570166490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.725+0000 7f356f7fe640 1 --2- 172.21.15.111:0/3197311759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357008ff30 0x7f35700903e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.725+0000 7f1624ff9640 1 --2- 172.21.15.111:0/2909140432 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f161406ac00 0x7f161406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.725+0000 7f1624ff9640 1 -- 172.21.15.111:0/2909140432 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1618088c80 con 0x7f16280c1280 2024-09-13T17:57:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.725+0000 7f1627fff640 1 --2- 172.21.15.111:0/2909140432 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f161406ac00 0x7f161406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.725+0000 7f356effd640 1 -- 172.21.15.111:0/3197311759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357008ff30 msgr2=0x7f35700903e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.725+0000 7f356effd640 1 --2- 172.21.15.111:0/3197311759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357008ff30 0x7f35700903e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.725+0000 7f356effd640 1 -- 172.21.15.111:0/3197311759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3570166010 msgr2=0x7f3570166490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.725+0000 7f356effd640 1 --2- 172.21.15.111:0/3197311759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3570166010 0x7f3570166490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.725+0000 7f356effd640 1 -- 172.21.15.111:0/3197311759 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3570090920 con 0x7f3558003970 2024-09-13T17:57:50.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.725+0000 7f356effd640 1 --2- 172.21.15.111:0/3197311759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3558003970 0x7f357008cb00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f356000a880 tx=0x7f3560001cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.725+0000 7f32cde19640 1 --2- 172.21.15.111:0/3006024453 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f32ac06ac50 0x7f32ac06d110 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f32c40090b0 tx=0x7f32c402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.725+0000 7f1627fff640 1 --2- 172.21.15.111:0/2909140432 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f161406ac00 0x7f161406d0c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f161c003eb0 tx=0x7f161c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.727+0000 7f7c9bfff640 1 -- 172.21.15.111:0/459104015 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f7cb00cdd20 con 0x7f7ca8048540 2024-09-13T17:57:50.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.728+0000 7f35577fe640 1 -- 172.21.15.111:0/3197311759 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3560034680 con 0x7f3558003970 2024-09-13T17:57:50.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.728+0000 7fa26559a640 1 -- 172.21.15.111:0/415597878 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25802ad40 con 0x7fa26004ef70 2024-09-13T17:57:50.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.728+0000 7f356ffff640 1 -- 172.21.15.111:0/3197311759 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3570090bd0 con 0x7f3558003970 2024-09-13T17:57:50.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.728+0000 7fa26559a640 1 -- 172.21.15.111:0/415597878 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa258032440 con 0x7fa26004ef70 2024-09-13T17:57:50.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.728+0000 7fa26559a640 1 -- 172.21.15.111:0/415597878 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25802aea0 con 0x7fa26004ef70 2024-09-13T17:57:50.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.728+0000 7f356ffff640 1 -- 172.21.15.111:0/3197311759 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35701643d0 con 0x7f3558003970 2024-09-13T17:57:50.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.728+0000 7f35577fe640 1 -- 172.21.15.111:0/3197311759 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35600347e0 con 0x7f3558003970 2024-09-13T17:57:50.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.728+0000 7f35577fe640 1 -- 172.21.15.111:0/3197311759 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3560034b70 con 0x7f3558003970 2024-09-13T17:57:50.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.729+0000 7fba7cff9640 1 -- 172.21.15.111:0/4151516486 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fba78014b00 con 0x7fba80099790 2024-09-13T17:57:50.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.729+0000 7fa26759e640 1 -- 172.21.15.111:0/415597878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa26004ef70 msgr2=0x7fa26000bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.729+0000 7fa26759e640 1 --2- 172.21.15.111:0/415597878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa26004ef70 0x7fa26000bb80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa26000f060 tx=0x7fa258029ff0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.729+0000 7f1d4effd640 1 -- 172.21.15.111:0/1387165094 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1d50013470 con 0x7f1d6015bbd0 2024-09-13T17:57:50.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.729+0000 7fa26759e640 1 -- 172.21.15.111:0/415597878 shutdown_connections 2024-09-13T17:57:50.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.729+0000 7fa26759e640 1 --2- 172.21.15.111:0/415597878 >> v2:172.21.15.111:3300/0 conn(0x7fa2600550c0 0x7fa260036810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.729+0000 7fa26759e640 1 --2- 172.21.15.111:0/415597878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa26004ef70 0x7fa26000bb80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.729+0000 7fa26759e640 1 --2- 172.21.15.111:0/415597878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa260036430 0x7fa26000a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.729+0000 7fa26759e640 1 -- 172.21.15.111:0/415597878 >> 172.21.15.111:0/415597878 conn(0x7fa26001dd40 msgr2=0x7fa2600549b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:50.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.729+0000 7fba7cff9640 1 --2- 172.21.15.111:0/4151516486 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fba6c06ac50 0x7fba6c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.729+0000 7fa26759e640 1 -- 172.21.15.111:0/415597878 shutdown_connections 2024-09-13T17:57:50.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.729+0000 7fa26759e640 1 -- 172.21.15.111:0/415597878 wait complete. 2024-09-13T17:57:50.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.729+0000 7f356ffff640 1 -- 172.21.15.111:0/3197311759 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3550005250 con 0x7f3558003970 2024-09-13T17:57:50.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.730+0000 7fba7cff9640 1 -- 172.21.15.111:0/4151516486 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fba7808a9e0 con 0x7fba80099790 2024-09-13T17:57:50.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.730+0000 7fba7f7fe640 1 --2- 172.21.15.111:0/4151516486 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fba6c06ac50 0x7fba6c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.730+0000 7f1d4effd640 1 --2- 172.21.15.111:0/1387165094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1d4006ac50 0x7f1d4006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.730+0000 7f1d4effd640 1 -- 172.21.15.111:0/1387165094 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1d5008a020 con 0x7f1d6015bbd0 2024-09-13T17:57:50.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.730+0000 7f1d5e59c640 1 --2- 172.21.15.111:0/1387165094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1d4006ac50 0x7f1d4006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.730+0000 7fba7f7fe640 1 --2- 172.21.15.111:0/4151516486 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fba6c06ac50 0x7fba6c06d110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba8015e960 tx=0x7fba74007e30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.731+0000 7f1d5e59c640 1 --2- 172.21.15.111:0/1387165094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1d4006ac50 0x7f1d4006d110 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1d5803a000 tx=0x7f1d58035000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.731+0000 7fa26759e640 1 Processor -- start 2024-09-13T17:57:50.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.731+0000 7fa26759e640 1 -- start start 2024-09-13T17:57:50.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa26759e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa260036430 0x7fa2600dc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa26759e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa26004ef70 0x7fa2600e05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa26759e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2600550c0 0x7fa2600e0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa26759e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600da7e0 con 0x7fa26004ef70 2024-09-13T17:57:50.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa26759e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600db920 con 0x7fa260036430 2024-09-13T17:57:50.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa26759e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600da980 con 0x7fa2600550c0 2024-09-13T17:57:50.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa265d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa26004ef70 0x7fa2600e05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa26659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa260036430 0x7fa2600dc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa265d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa26004ef70 0x7fa2600e05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54238/0 (socket says 172.21.15.111:54238) 2024-09-13T17:57:50.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa265d9b640 1 -- 172.21.15.111:0/4291856609 learned_addr learned my addr 172.21.15.111:0/4291856609 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:50.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa266d9d640 1 --2- 172.21.15.111:0/4291856609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2600550c0 0x7fa2600e0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa26659c640 1 -- 172.21.15.111:0/4291856609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2600550c0 msgr2=0x7fa2600e0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa26659c640 1 --2- 172.21.15.111:0/4291856609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2600550c0 0x7fa2600e0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa26659c640 1 -- 172.21.15.111:0/4291856609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa26004ef70 msgr2=0x7fa2600e05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:50.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa26659c640 1 --2- 172.21.15.111:0/4291856609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa26004ef70 0x7fa2600e05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:50.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa26659c640 1 -- 172.21.15.111:0/4291856609 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa258009400 con 0x7fa260036430 2024-09-13T17:57:50.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa265d9b640 1 --2- 172.21.15.111:0/4291856609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa26004ef70 0x7fa2600e05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:50.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.732+0000 7fa266d9d640 1 --2- 172.21.15.111:0/4291856609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2600550c0 0x7fa2600e0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:50.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.733+0000 7fa26659c640 1 --2- 172.21.15.111:0/4291856609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa260036430 0x7fa2600dc6a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00e8a0 tx=0x7fa25c00eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.733+0000 7f32be7fc640 1 -- 172.21.15.111:0/3006024453 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f32c81e6a00 con 0x7f32c80ae880 2024-09-13T17:57:50.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.738+0000 7fba7cff9640 1 -- 172.21.15.111:0/4151516486 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fba78055af0 con 0x7fba80099790 2024-09-13T17:57:50.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.743+0000 7f1624ff9640 1 -- 172.21.15.111:0/2909140432 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f1618053b70 con 0x7f16280c1280 2024-09-13T17:57:50.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.745+0000 7f1d4effd640 1 -- 172.21.15.111:0/1387165094 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f1d50054f60 con 0x7f1d6015bbd0 2024-09-13T17:57:50.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.745+0000 7fa256ffd640 1 -- 172.21.15.111:0/4291856609 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c00ee90 con 0x7fa260036430 2024-09-13T17:57:50.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.745+0000 7fa256ffd640 1 -- 172.21.15.111:0/4291856609 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25c00c940 con 0x7fa260036430 2024-09-13T17:57:50.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.746+0000 7fa256ffd640 1 -- 172.21.15.111:0/4291856609 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c00ccb0 con 0x7fa260036430 2024-09-13T17:57:50.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.746+0000 7fa26759e640 1 -- 172.21.15.111:0/4291856609 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2600dac40 con 0x7fa260036430 2024-09-13T17:57:50.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.746+0000 7fa26759e640 1 -- 172.21.15.111:0/4291856609 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2600dbea0 con 0x7fa260036430 2024-09-13T17:57:50.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.746+0000 7f35577fe640 1 -- 172.21.15.111:0/3197311759 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3560031070 con 0x7f3558003970 2024-09-13T17:57:50.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.747+0000 7f35577fe640 1 --2- 172.21.15.111:0/3197311759 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f355c06ac30 0x7f355c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.747+0000 7f35577fe640 1 -- 172.21.15.111:0/3197311759 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f35600aa3c0 con 0x7f3558003970 2024-09-13T17:57:50.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.748+0000 7f356e7fc640 1 --2- 172.21.15.111:0/3197311759 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f355c06ac30 0x7f355c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.748+0000 7fa254ff9640 1 -- 172.21.15.111:0/4291856609 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa240005250 con 0x7fa260036430 2024-09-13T17:57:50.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.749+0000 7f356e7fc640 1 --2- 172.21.15.111:0/3197311759 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f355c06ac30 0x7f355c06d0f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f35700b1780 tx=0x7f3564008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:50.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.749+0000 7f35577fe640 1 -- 172.21.15.111:0/3197311759 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f35600753d0 con 0x7f3558003970 2024-09-13T17:57:50.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.753+0000 7fa256ffd640 1 -- 172.21.15.111:0/4291856609 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa25c005990 con 0x7fa260036430 2024-09-13T17:57:50.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.754+0000 7fa256ffd640 1 --2- 172.21.15.111:0/4291856609 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa24806acd0 0x7fa24806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:50.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.754+0000 7fa256ffd640 1 -- 172.21.15.111:0/4291856609 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa25c089940 con 0x7fa260036430 2024-09-13T17:57:50.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.754+0000 7fa256ffd640 1 -- 172.21.15.111:0/4291856609 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fa25c0b6a60 con 0x7fa260036430 2024-09-13T17:57:50.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.756+0000 7fa265d9b640 1 --2- 172.21.15.111:0/4291856609 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa24806acd0 0x7fa24806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:50.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:50.765+0000 7fa265d9b640 1 --2- 172.21.15.111:0/4291856609 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa24806acd0 0x7fa24806d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa26017b510 tx=0x7fa25802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:51.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.043+0000 7f132a50f640 1 -- 172.21.15.111:0/2629714960 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1304005500 con 0x7f132406b590 2024-09-13T17:57:51.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.044+0000 7f1320ff9640 1 -- 172.21.15.111:0/2629714960 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1314017090 con 0x7f132406b590 2024-09-13T17:57:51.047 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2024-09-13T17:57:51.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.047+0000 7f130a7fc640 1 -- 172.21.15.111:0/2629714960 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f131006ad80 msgr2=0x7f131006d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.047+0000 7f130a7fc640 1 --2- 172.21.15.111:0/2629714960 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f131006ad80 0x7f131006d240 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13240c3d60 tx=0x7f1318008040 comp rx=0 tx=0).stop 2024-09-13T17:57:51.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.047+0000 7f130a7fc640 1 -- 172.21.15.111:0/2629714960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f132406b590 msgr2=0x7f132409d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.048+0000 7f130a7fc640 1 --2- 172.21.15.111:0/2629714960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f132406b590 0x7f132409d370 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f131400ebe0 tx=0x7f131400c520 comp rx=0 tx=0).stop 2024-09-13T17:57:51.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.048+0000 7f130a7fc640 1 -- 172.21.15.111:0/2629714960 shutdown_connections 2024-09-13T17:57:51.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.048+0000 7f130a7fc640 1 --2- 172.21.15.111:0/2629714960 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f131006ad80 0x7f131006d240 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.048+0000 7f130a7fc640 1 --2- 172.21.15.111:0/2629714960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13241d9cf0 0x7f132409a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.048+0000 7f130a7fc640 1 --2- 172.21.15.111:0/2629714960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13240cfb30 0x7f13241d9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.048+0000 7f130a7fc640 1 --2- 172.21.15.111:0/2629714960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f132406b590 0x7f132409d370 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.048+0000 7f130a7fc640 1 -- 172.21.15.111:0/2629714960 >> 172.21.15.111:0/2629714960 conn(0x7f13240b1f60 msgr2=0x7f13240aeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:51.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.049+0000 7f130a7fc640 1 -- 172.21.15.111:0/2629714960 shutdown_connections 2024-09-13T17:57:51.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.049+0000 7f130a7fc640 1 -- 172.21.15.111:0/2629714960 wait complete. 2024-09-13T17:57:51.068 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.2 2024-09-13T17:57:51.068 DEBUG:teuthology.parallel:result is None 2024-09-13T17:57:51.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.086+0000 7f32d007d640 1 -- 172.21.15.111:0/3006024453 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f32c81e6a00 con 0x7f32c80ae880 2024-09-13T17:57:51.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.087+0000 7f32be7fc640 1 -- 172.21.15.111:0/3006024453 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f32c81e6a00 con 0x7f32c80ae880 2024-09-13T17:57:51.090 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2024-09-13T17:57:51.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.089+0000 7f32d007d640 1 -- 172.21.15.111:0/3006024453 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f32ac06ac50 msgr2=0x7f32ac06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.089+0000 7f32d007d640 1 --2- 172.21.15.111:0/3006024453 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f32ac06ac50 0x7f32ac06d110 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f32c40090b0 tx=0x7f32c402e000 comp rx=0 tx=0).stop 2024-09-13T17:57:51.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.089+0000 7f32d007d640 1 -- 172.21.15.111:0/3006024453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32c80ae880 msgr2=0x7f32c81e84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.089+0000 7f32d007d640 1 --2- 172.21.15.111:0/3006024453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32c80ae880 0x7f32c81e84d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f32c000be20 tx=0x7f32c0009f70 comp rx=0 tx=0).stop 2024-09-13T17:57:51.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.089+0000 7f32d007d640 1 -- 172.21.15.111:0/3006024453 shutdown_connections 2024-09-13T17:57:51.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.089+0000 7f32d007d640 1 --2- 172.21.15.111:0/3006024453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32c80c1280 0x7f32c8154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.089+0000 7f32d007d640 1 --2- 172.21.15.111:0/3006024453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32c80ae880 0x7f32c81e84d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.089+0000 7f32d007d640 1 --2- 172.21.15.111:0/3006024453 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f32ac06ac50 0x7f32ac06d110 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.089+0000 7f32d007d640 1 --2- 172.21.15.111:0/3006024453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32c80ab140 0x7f32c80999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.089+0000 7f32d007d640 1 -- 172.21.15.111:0/3006024453 >> 172.21.15.111:0/3006024453 conn(0x7f32c80afe60 msgr2=0x7f32c80ab540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:51.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.090+0000 7f32d007d640 1 -- 172.21.15.111:0/3006024453 shutdown_connections 2024-09-13T17:57:51.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.090+0000 7f32d007d640 1 -- 172.21.15.111:0/3006024453 wait complete. 2024-09-13T17:57:51.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.096+0000 7f7c99ffb640 1 -- 172.21.15.111:0/459104015 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7ca803a230 con 0x7f7ca8048540 2024-09-13T17:57:51.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.097+0000 7f7c9bfff640 1 -- 172.21.15.111:0/459104015 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7cb0097070 con 0x7f7ca8048540 2024-09-13T17:57:51.100 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2024-09-13T17:57:51.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.100+0000 7f7c99ffb640 1 -- 172.21.15.111:0/459104015 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7c9406ac50 msgr2=0x7f7c9406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.100+0000 7f7c99ffb640 1 --2- 172.21.15.111:0/459104015 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7c9406ac50 0x7f7c9406d110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4034000 tx=0x7f7ca402e000 comp rx=0 tx=0).stop 2024-09-13T17:57:51.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.100+0000 7f7c99ffb640 1 -- 172.21.15.111:0/459104015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ca8048540 msgr2=0x7f7ca804f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.100+0000 7f7c99ffb640 1 --2- 172.21.15.111:0/459104015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ca8048540 0x7f7ca804f830 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0065a00 tx=0x7f7cb0067190 comp rx=0 tx=0).stop 2024-09-13T17:57:51.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.101+0000 7f7c99ffb640 1 -- 172.21.15.111:0/459104015 shutdown_connections 2024-09-13T17:57:51.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.101+0000 7f7c99ffb640 1 --2- 172.21.15.111:0/459104015 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7c9406ac50 0x7f7c9406d110 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.101+0000 7f7c99ffb640 1 --2- 172.21.15.111:0/459104015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ca8048540 0x7f7ca804f830 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.101+0000 7f7c99ffb640 1 --2- 172.21.15.111:0/459104015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ca80e70e0 0x7f7ca8048000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.101+0000 7f7c99ffb640 1 --2- 172.21.15.111:0/459104015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ca804eb20 0x7f7ca817be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.101+0000 7f7c99ffb640 1 -- 172.21.15.111:0/459104015 >> 172.21.15.111:0/459104015 conn(0x7f7ca8036050 msgr2=0x7f7ca80351c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:51.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.101+0000 7f7c99ffb640 1 -- 172.21.15.111:0/459104015 shutdown_connections 2024-09-13T17:57:51.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.101+0000 7f7c99ffb640 1 -- 172.21.15.111:0/459104015 wait complete. 2024-09-13T17:57:51.105 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.7 2024-09-13T17:57:51.105 DEBUG:teuthology.parallel:result is None 2024-09-13T17:57:51.118 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.6 2024-09-13T17:57:51.118 DEBUG:teuthology.parallel:result is None 2024-09-13T17:57:51.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.118+0000 7fba86a61640 1 -- 172.21.15.111:0/4151516486 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fba800c42c0 con 0x7fba80099790 2024-09-13T17:57:51.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.119+0000 7fba7cff9640 1 -- 172.21.15.111:0/4151516486 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba800c42c0 con 0x7fba80099790 2024-09-13T17:57:51.122 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2024-09-13T17:57:51.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.122+0000 7fba86a61640 1 -- 172.21.15.111:0/4151516486 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fba6c06ac50 msgr2=0x7fba6c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.122+0000 7fba86a61640 1 --2- 172.21.15.111:0/4151516486 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fba6c06ac50 0x7fba6c06d110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba8015e960 tx=0x7fba74007e30 comp rx=0 tx=0).stop 2024-09-13T17:57:51.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.122+0000 7fba86a61640 1 -- 172.21.15.111:0/4151516486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba80099790 msgr2=0x7fba800c2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.122+0000 7fba86a61640 1 --2- 172.21.15.111:0/4151516486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba80099790 0x7fba800c2a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba78007e60 tx=0x7fba7800e4b0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.122+0000 7fba86a61640 1 -- 172.21.15.111:0/4151516486 shutdown_connections 2024-09-13T17:57:51.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.122+0000 7fba86a61640 1 --2- 172.21.15.111:0/4151516486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba8015e170 0x7fba800c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.122+0000 7fba86a61640 1 --2- 172.21.15.111:0/4151516486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba8015a610 0x7fba800c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.122+0000 7fba86a61640 1 --2- 172.21.15.111:0/4151516486 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fba6c06ac50 0x7fba6c06d110 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.122+0000 7fba86a61640 1 --2- 172.21.15.111:0/4151516486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba80099790 0x7fba800c2a10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.122+0000 7fba86a61640 1 -- 172.21.15.111:0/4151516486 >> 172.21.15.111:0/4151516486 conn(0x7fba800b1df0 msgr2=0x7fba800afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:51.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.122+0000 7fba86a61640 1 -- 172.21.15.111:0/4151516486 shutdown_connections 2024-09-13T17:57:51.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.122+0000 7fba86a61640 1 -- 172.21.15.111:0/4151516486 wait complete. 2024-09-13T17:57:51.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.128+0000 7f356ffff640 1 -- 172.21.15.111:0/3197311759 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f35500057d0 con 0x7f3558003970 2024-09-13T17:57:51.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.129+0000 7f35577fe640 1 -- 172.21.15.111:0/3197311759 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3560045090 con 0x7f3558003970 2024-09-13T17:57:51.131 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2024-09-13T17:57:51.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.131+0000 7f356ffff640 1 -- 172.21.15.111:0/3197311759 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f355c06ac30 msgr2=0x7f355c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.131+0000 7f356ffff640 1 --2- 172.21.15.111:0/3197311759 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f355c06ac30 0x7f355c06d0f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f35700b1780 tx=0x7f3564008040 comp rx=0 tx=0).stop 2024-09-13T17:57:51.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.131+0000 7f356ffff640 1 -- 172.21.15.111:0/3197311759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3558003970 msgr2=0x7f357008cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.131+0000 7f356ffff640 1 --2- 172.21.15.111:0/3197311759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3558003970 0x7f357008cb00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f356000a880 tx=0x7f3560001cb0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.131+0000 7f356ffff640 1 -- 172.21.15.111:0/3197311759 shutdown_connections 2024-09-13T17:57:51.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.131+0000 7f356ffff640 1 --2- 172.21.15.111:0/3197311759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357008ff30 0x7f35700903e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.131+0000 7f356ffff640 1 --2- 172.21.15.111:0/3197311759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3570166010 0x7f3570166490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.131+0000 7f356ffff640 1 --2- 172.21.15.111:0/3197311759 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f355c06ac30 0x7f355c06d0f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.131+0000 7f356ffff640 1 --2- 172.21.15.111:0/3197311759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3558003970 0x7f357008cb00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.131+0000 7f356ffff640 1 -- 172.21.15.111:0/3197311759 >> 172.21.15.111:0/3197311759 conn(0x7f3570070440 msgr2=0x7f35700ae680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:51.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.131+0000 7f356ffff640 1 -- 172.21.15.111:0/3197311759 shutdown_connections 2024-09-13T17:57:51.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.131+0000 7f356ffff640 1 -- 172.21.15.111:0/3197311759 wait complete. 2024-09-13T17:57:51.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.132+0000 7fa254ff9640 1 -- 172.21.15.111:0/4291856609 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa2400057d0 con 0x7fa260036430 2024-09-13T17:57:51.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.132+0000 7fa256ffd640 1 -- 172.21.15.111:0/4291856609 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa25c021020 con 0x7fa260036430 2024-09-13T17:57:51.135 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2024-09-13T17:57:51.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.133+0000 7f1d64a5a640 1 -- 172.21.15.111:0/1387165094 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1d60097790 con 0x7f1d6015bbd0 2024-09-13T17:57:51.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.134+0000 7f162e5e2640 1 -- 172.21.15.111:0/2909140432 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f16080057d0 con 0x7f16280c1280 2024-09-13T17:57:51.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.134+0000 7f1d4effd640 1 -- 172.21.15.111:0/1387165094 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1d50021090 con 0x7f1d6015bbd0 2024-09-13T17:57:51.137 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2024-09-13T17:57:51.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.134+0000 7f1624ff9640 1 -- 172.21.15.111:0/2909140432 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1618017020 con 0x7f16280c1280 2024-09-13T17:57:51.138 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2024-09-13T17:57:51.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.135+0000 7fa254ff9640 1 -- 172.21.15.111:0/4291856609 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa24806acd0 msgr2=0x7fa24806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.135+0000 7fa254ff9640 1 --2- 172.21.15.111:0/4291856609 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa24806acd0 0x7fa24806d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa26017b510 tx=0x7fa25802e000 comp rx=0 tx=0).stop 2024-09-13T17:57:51.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.135+0000 7fa254ff9640 1 -- 172.21.15.111:0/4291856609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa260036430 msgr2=0x7fa2600dc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.135+0000 7fa254ff9640 1 --2- 172.21.15.111:0/4291856609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa260036430 0x7fa2600dc6a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00e8a0 tx=0x7fa25c00eca0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.135+0000 7fa254ff9640 1 -- 172.21.15.111:0/4291856609 shutdown_connections 2024-09-13T17:57:51.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.135+0000 7fa254ff9640 1 --2- 172.21.15.111:0/4291856609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2600550c0 0x7fa2600e0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.135+0000 7fa254ff9640 1 --2- 172.21.15.111:0/4291856609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa26004ef70 0x7fa2600e05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.135+0000 7fa254ff9640 1 --2- 172.21.15.111:0/4291856609 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa24806acd0 0x7fa24806d190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.135+0000 7fa254ff9640 1 --2- 172.21.15.111:0/4291856609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa260036430 0x7fa2600dc6a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.135+0000 7fa254ff9640 1 -- 172.21.15.111:0/4291856609 >> 172.21.15.111:0/4291856609 conn(0x7fa26001dd40 msgr2=0x7fa260034410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:51.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.136+0000 7fa254ff9640 1 -- 172.21.15.111:0/4291856609 shutdown_connections 2024-09-13T17:57:51.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.136+0000 7fa254ff9640 1 -- 172.21.15.111:0/4291856609 wait complete. 2024-09-13T17:57:51.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.136+0000 7f1d64a5a640 1 -- 172.21.15.111:0/1387165094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1d4006ac50 msgr2=0x7f1d4006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.136+0000 7f1d64a5a640 1 --2- 172.21.15.111:0/1387165094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1d4006ac50 0x7f1d4006d110 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1d5803a000 tx=0x7f1d58035000 comp rx=0 tx=0).stop 2024-09-13T17:57:51.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.136+0000 7f1d64a5a640 1 -- 172.21.15.111:0/1387165094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d6015bbd0 msgr2=0x7f1d601548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.136+0000 7f1d64a5a640 1 --2- 172.21.15.111:0/1387165094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d6015bbd0 0x7f1d601548e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000e960 tx=0x7f1d5000ee20 comp rx=0 tx=0).stop 2024-09-13T17:57:51.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.136+0000 7f1d64a5a640 1 -- 172.21.15.111:0/1387165094 shutdown_connections 2024-09-13T17:57:51.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.136+0000 7f1d64a5a640 1 --2- 172.21.15.111:0/1387165094 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1d4006ac50 0x7f1d4006d110 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.136+0000 7f1d64a5a640 1 --2- 172.21.15.111:0/1387165094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d6015bbd0 0x7f1d601548e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.136+0000 7f1d64a5a640 1 --2- 172.21.15.111:0/1387165094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d600cc590 0x7f1d600bb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.136+0000 7f1d64a5a640 1 --2- 172.21.15.111:0/1387165094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d60097370 0x7f1d600c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.136+0000 7f1d64a5a640 1 -- 172.21.15.111:0/1387165094 >> 172.21.15.111:0/1387165094 conn(0x7f1d600afe60 msgr2=0x7f1d600adc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:51.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.136+0000 7f1d64a5a640 1 -- 172.21.15.111:0/1387165094 shutdown_connections 2024-09-13T17:57:51.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.136+0000 7f1d64a5a640 1 -- 172.21.15.111:0/1387165094 wait complete. 2024-09-13T17:57:51.142 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.1 2024-09-13T17:57:51.143 DEBUG:teuthology.parallel:result is None 2024-09-13T17:57:51.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.138+0000 7f162e5e2640 1 -- 172.21.15.111:0/2909140432 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f161406ac00 msgr2=0x7f161406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.138+0000 7f162e5e2640 1 --2- 172.21.15.111:0/2909140432 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f161406ac00 0x7f161406d0c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f161c003eb0 tx=0x7f161c009040 comp rx=0 tx=0).stop 2024-09-13T17:57:51.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.138+0000 7f162e5e2640 1 -- 172.21.15.111:0/2909140432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16280c1280 msgr2=0x7f1628201a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.138+0000 7f162e5e2640 1 --2- 172.21.15.111:0/2909140432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16280c1280 0x7f1628201a90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f161800c430 tx=0x7f161800c8f0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.138+0000 7f162e5e2640 1 -- 172.21.15.111:0/2909140432 shutdown_connections 2024-09-13T17:57:51.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.138+0000 7f162e5e2640 1 --2- 172.21.15.111:0/2909140432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1628169600 0x7f162815eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.138+0000 7f162e5e2640 1 --2- 172.21.15.111:0/2909140432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16280c1280 0x7f1628201a90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.138+0000 7f162e5e2640 1 --2- 172.21.15.111:0/2909140432 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f161406ac00 0x7f161406d0c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.138+0000 7f162e5e2640 1 --2- 172.21.15.111:0/2909140432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16280ae880 0x7f162815c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.138+0000 7f162e5e2640 1 -- 172.21.15.111:0/2909140432 >> 172.21.15.111:0/2909140432 conn(0x7f16280afe60 msgr2=0x7f162815bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:51.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.138+0000 7f162e5e2640 1 -- 172.21.15.111:0/2909140432 shutdown_connections 2024-09-13T17:57:51.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.138+0000 7f162e5e2640 1 -- 172.21.15.111:0/2909140432 wait complete. 2024-09-13T17:57:51.149 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.0 2024-09-13T17:57:51.150 DEBUG:teuthology.parallel:result is None 2024-09-13T17:57:51.152 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.4 2024-09-13T17:57:51.152 DEBUG:teuthology.parallel:result is None 2024-09-13T17:57:51.157 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.3 2024-09-13T17:57:51.158 DEBUG:teuthology.parallel:result is None 2024-09-13T17:57:51.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.5 2024-09-13T17:57:51.158 DEBUG:teuthology.parallel:result is None 2024-09-13T17:57:51.158 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T17:57:51.158 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:57:51.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.292+0000 7f69cfe9c640 1 Processor -- start 2024-09-13T17:57:51.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.292+0000 7f69cfe9c640 1 -- start start 2024-09-13T17:57:51.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.293+0000 7f69cfe9c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f69c815a2e0 0x7f69c80c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.293+0000 7f69cfe9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c80c9f80 0x7f69c80cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.293+0000 7f69cfe9c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c80c8920 con 0x7f69c80cb220 2024-09-13T17:57:51.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.293+0000 7f69cfe9c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c80cb650 con 0x7f69c815a2e0 2024-09-13T17:57:51.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.293+0000 7f69cfe9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c815a6c0 con 0x7f69c80c9f80 2024-09-13T17:57:51.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.293+0000 7f69cdc38640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f69c815a2e0 0x7f69c80c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.293+0000 7f69cdc38640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f69c815a2e0 0x7f69c80c9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54242/0 (socket says 172.21.15.111:54242) 2024-09-13T17:57:51.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.293+0000 7f69cdc38640 1 -- 172.21.15.111:0/3725529666 learned_addr learned my addr 172.21.15.111:0/3725529666 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:51.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.293+0000 7f69cd437640 1 --2- 172.21.15.111:0/3725529666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c80c9f80 0x7f69c80cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.293+0000 7f69cdc38640 1 -- 172.21.15.111:0/3725529666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c80c9f80 msgr2=0x7f69c80cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:51.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.294+0000 7f69cdc38640 1 --2- 172.21.15.111:0/3725529666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c80c9f80 0x7f69c80cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.294+0000 7f69cdc38640 1 -- 172.21.15.111:0/3725529666 >> v1:172.21.15.188:6789/0 conn(0x7f69c80cb220 legacy=0x7f69c80ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.294+0000 7f69cdc38640 1 -- 172.21.15.111:0/3725529666 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c80ca500 con 0x7f69c815a2e0 2024-09-13T17:57:51.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.294+0000 7f69cd437640 1 --2- 172.21.15.111:0/3725529666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c80c9f80 0x7f69c80cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:51.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.294+0000 7f69cdc38640 1 --2- 172.21.15.111:0/3725529666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69c815a2e0 0x7f69c80c9a40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009870 tx=0x7f69c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b1de9ff65a065b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:51.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.294+0000 7f69ccc36640 1 -- 172.21.15.111:0/3725529666 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c4032440 con 0x7f69c815a2e0 2024-09-13T17:57:51.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.294+0000 7f69ccc36640 1 -- 172.21.15.111:0/3725529666 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c40325a0 con 0x7f69c815a2e0 2024-09-13T17:57:51.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.294+0000 7f69ccc36640 1 -- 172.21.15.111:0/3725529666 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c4032930 con 0x7f69c815a2e0 2024-09-13T17:57:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.295+0000 7f69cfe9c640 1 -- 172.21.15.111:0/3725529666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69c815a2e0 msgr2=0x7f69c80c9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.295+0000 7f69cfe9c640 1 --2- 172.21.15.111:0/3725529666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69c815a2e0 0x7f69c80c9a40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009870 tx=0x7f69c402a270 comp rx=0 tx=0).stop 2024-09-13T17:57:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.295+0000 7f69cfe9c640 1 -- 172.21.15.111:0/3725529666 shutdown_connections 2024-09-13T17:57:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.295+0000 7f69cfe9c640 1 --2- 172.21.15.111:0/3725529666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c80c9f80 0x7f69c80cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.295+0000 7f69cfe9c640 1 --2- 172.21.15.111:0/3725529666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69c815a2e0 0x7f69c80c9a40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.295+0000 7f69cfe9c640 1 -- 172.21.15.111:0/3725529666 >> 172.21.15.111:0/3725529666 conn(0x7f69c8069ed0 msgr2=0x7f69c806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.295+0000 7f69cfe9c640 1 -- 172.21.15.111:0/3725529666 shutdown_connections 2024-09-13T17:57:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.295+0000 7f69cfe9c640 1 -- 172.21.15.111:0/3725529666 wait complete. 2024-09-13T17:57:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.296+0000 7f69cfe9c640 1 Processor -- start 2024-09-13T17:57:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.296+0000 7f69cfe9c640 1 -- start start 2024-09-13T17:57:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69cfe9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c80c9f80 0x7f69c815a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69cfe9c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69c80cb220 0x7f69c815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69cfe9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c815a2e0 0x7f69c815d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69cfe9c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c815bdf0 con 0x7f69c80cb220 2024-09-13T17:57:51.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69cfe9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c815b170 con 0x7f69c815a2e0 2024-09-13T17:57:51.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69cfe9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c815b450 con 0x7f69c80c9f80 2024-09-13T17:57:51.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69cd437640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69c80cb220 0x7f69c815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69cd437640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69c80cb220 0x7f69c815ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54252/0 (socket says 172.21.15.111:54252) 2024-09-13T17:57:51.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69cd437640 1 -- 172.21.15.111:0/2659794754 learned_addr learned my addr 172.21.15.111:0/2659794754 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:51.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69cdc38640 1 --2- 172.21.15.111:0/2659794754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c80c9f80 0x7f69c815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69ce439640 1 --2- 172.21.15.111:0/2659794754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c815a2e0 0x7f69c815d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69cd437640 1 -- 172.21.15.111:0/2659794754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c80c9f80 msgr2=0x7f69c815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69cd437640 1 --2- 172.21.15.111:0/2659794754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c80c9f80 0x7f69c815a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69cd437640 1 -- 172.21.15.111:0/2659794754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c815a2e0 msgr2=0x7f69c815d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69cd437640 1 --2- 172.21.15.111:0/2659794754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c815a2e0 0x7f69c815d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69cd437640 1 -- 172.21.15.111:0/2659794754 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c40094d0 con 0x7f69c80cb220 2024-09-13T17:57:51.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69cdc38640 1 --2- 172.21.15.111:0/2659794754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c80c9f80 0x7f69c815a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:51.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69ce439640 1 --2- 172.21.15.111:0/2659794754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c815a2e0 0x7f69c815d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:51.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69cd437640 1 --2- 172.21.15.111:0/2659794754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69c80cb220 0x7f69c815ccb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f69b800e730 tx=0x7f69b800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:51.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.297+0000 7f69be7fc640 1 -- 172.21.15.111:0/2659794754 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8004230 con 0x7f69c80cb220 2024-09-13T17:57:51.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.298+0000 7f69be7fc640 1 -- 172.21.15.111:0/2659794754 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b8004390 con 0x7f69c80cb220 2024-09-13T17:57:51.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.298+0000 7f69cfe9c640 1 -- 172.21.15.111:0/2659794754 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c815b730 con 0x7f69c80cb220 2024-09-13T17:57:51.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.298+0000 7f69be7fc640 1 -- 172.21.15.111:0/2659794754 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b800c8f0 con 0x7f69c80cb220 2024-09-13T17:57:51.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.298+0000 7f69cfe9c640 1 -- 172.21.15.111:0/2659794754 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c815ad40 con 0x7f69c80cb220 2024-09-13T17:57:51.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.299+0000 7f69cfe9c640 1 -- 172.21.15.111:0/2659794754 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c80c9630 con 0x7f69c80cb220 2024-09-13T17:57:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.299+0000 7f69be7fc640 1 -- 172.21.15.111:0/2659794754 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f69b800ca50 con 0x7f69c80cb220 2024-09-13T17:57:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.299+0000 7f69be7fc640 1 --2- 172.21.15.111:0/2659794754 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f69ac06ac00 0x7f69ac06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.300+0000 7f69be7fc640 1 -- 172.21.15.111:0/2659794754 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f69b8089280 con 0x7f69c80cb220 2024-09-13T17:57:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.300+0000 7f69cdc38640 1 --2- 172.21.15.111:0/2659794754 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f69ac06ac00 0x7f69ac06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.300+0000 7f69cdc38640 1 --2- 172.21.15.111:0/2659794754 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f69ac06ac00 0x7f69ac06d0c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f69c4034000 tx=0x7f69c402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:51.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.304+0000 7f69be7fc640 1 -- 172.21.15.111:0/2659794754 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f69b8054170 con 0x7f69c80cb220 2024-09-13T17:57:51.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.462+0000 7f69cfe9c640 1 -- 172.21.15.111:0/2659794754 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69c80c9130 con 0x7f69ac06ac00 2024-09-13T17:57:51.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.470+0000 7f69be7fc640 1 -- 172.21.15.111:0/2659794754 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345188 (secure 0 0 0) 0x7f69c4042090 con 0x7f69ac06ac00 2024-09-13T17:57:51.477 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:51.479 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-09-13T17:57:51.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.474+0000 7f69cfe9c640 1 -- 172.21.15.111:0/2659794754 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f69ac06ac00 msgr2=0x7f69ac06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.474+0000 7f69cfe9c640 1 --2- 172.21.15.111:0/2659794754 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f69ac06ac00 0x7f69ac06d0c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f69c4034000 tx=0x7f69c402e000 comp rx=0 tx=0).stop 2024-09-13T17:57:51.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.474+0000 7f69cfe9c640 1 -- 172.21.15.111:0/2659794754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69c80cb220 msgr2=0x7f69c815ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.474+0000 7f69cfe9c640 1 --2- 172.21.15.111:0/2659794754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69c80cb220 0x7f69c815ccb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f69b800e730 tx=0x7f69b800ebf0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.474+0000 7f69cfe9c640 1 -- 172.21.15.111:0/2659794754 shutdown_connections 2024-09-13T17:57:51.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.474+0000 7f69cfe9c640 1 --2- 172.21.15.111:0/2659794754 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f69ac06ac00 0x7f69ac06d0c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.474+0000 7f69cfe9c640 1 --2- 172.21.15.111:0/2659794754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c815a2e0 0x7f69c815d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.474+0000 7f69cfe9c640 1 --2- 172.21.15.111:0/2659794754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69c80cb220 0x7f69c815ccb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.474+0000 7f69cfe9c640 1 --2- 172.21.15.111:0/2659794754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c80c9f80 0x7f69c815a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.474+0000 7f69cfe9c640 1 -- 172.21.15.111:0/2659794754 >> 172.21.15.111:0/2659794754 conn(0x7f69c8069ed0 msgr2=0x7f69c81e2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:51.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.474+0000 7f69cfe9c640 1 -- 172.21.15.111:0/2659794754 shutdown_connections 2024-09-13T17:57:51.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.474+0000 7f69cfe9c640 1 -- 172.21.15.111:0/2659794754 wait complete. 2024-09-13T17:57:51.487 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-13T17:57:50.271362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3204,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3280896,"data_stored":1987143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002630"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727037+0000","last_change":"2024-09-13T17:57:25.378960+0000","last_active":"2024-09-13T17:57:29.727037+0000","last_peered":"2024-09-13T17:57:29.727037+0000","last_clean":"2024-09-13T17:57:29.727037+0000","last_became_active":"2024-09-13T17:57:25.378681+0000","last_became_peered":"2024-09-13T17:57:25.378681+0000","last_unstale":"2024-09-13T17:57:29.727037+0000","last_undegraded":"2024-09-13T17:57:29.727037+0000","last_fullsized":"2024-09-13T17:57:29.727037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721168+0000","last_change":"2024-09-13T17:57:25.734706+0000","last_active":"2024-09-13T17:57:29.721168+0000","last_peered":"2024-09-13T17:57:29.721168+0000","last_clean":"2024-09-13T17:57:29.721168+0000","last_became_active":"2024-09-13T17:57:25.734432+0000","last_became_peered":"2024-09-13T17:57:25.734432+0000","last_unstale":"2024-09-13T17:57:29.721168+0000","last_undegraded":"2024-09-13T17:57:29.721168+0000","last_fullsized":"2024-09-13T17:57:29.721168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:12.960061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.730686+0000","last_change":"2024-09-13T17:57:29.729380+0000","last_active":"2024-09-13T17:57:29.730686+0000","last_peered":"2024-09-13T17:57:29.730686+0000","last_clean":"2024-09-13T17:57:29.730686+0000","last_became_active":"2024-09-13T17:57:29.729000+0000","last_became_peered":"2024-09-13T17:57:29.729000+0000","last_unstale":"2024-09-13T17:57:29.730686+0000","last_undegraded":"2024-09-13T17:57:29.730686+0000","last_fullsized":"2024-09-13T17:57:29.730686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:31.285328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721336+0000","last_change":"2024-09-13T17:57:25.382416+0000","last_active":"2024-09-13T17:57:29.721336+0000","last_peered":"2024-09-13T17:57:29.721336+0000","last_clean":"2024-09-13T17:57:29.721336+0000","last_became_active":"2024-09-13T17:57:25.382154+0000","last_became_peered":"2024-09-13T17:57:25.382154+0000","last_unstale":"2024-09-13T17:57:29.721336+0000","last_undegraded":"2024-09-13T17:57:29.721336+0000","last_fullsized":"2024-09-13T17:57:29.721336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725491+0000","last_change":"2024-09-13T17:57:25.731759+0000","last_active":"2024-09-13T17:57:29.725491+0000","last_peered":"2024-09-13T17:57:29.725491+0000","last_clean":"2024-09-13T17:57:29.725491+0000","last_became_active":"2024-09-13T17:57:25.731494+0000","last_became_peered":"2024-09-13T17:57:25.731494+0000","last_unstale":"2024-09-13T17:57:29.725491+0000","last_undegraded":"2024-09-13T17:57:29.725491+0000","last_fullsized":"2024-09-13T17:57:29.725491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.751270+0000","last_change":"2024-09-13T17:57:29.750278+0000","last_active":"2024-09-13T17:57:29.751270+0000","last_peered":"2024-09-13T17:57:29.751270+0000","last_clean":"2024-09-13T17:57:29.751270+0000","last_became_active":"2024-09-13T17:57:29.749874+0000","last_became_peered":"2024-09-13T17:57:29.749874+0000","last_unstale":"2024-09-13T17:57:29.751270+0000","last_undegraded":"2024-09-13T17:57:29.751270+0000","last_fullsized":"2024-09-13T17:57:29.751270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:03.749305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725223+0000","last_change":"2024-09-13T17:57:25.377660+0000","last_active":"2024-09-13T17:57:29.725223+0000","last_peered":"2024-09-13T17:57:29.725223+0000","last_clean":"2024-09-13T17:57:29.725223+0000","last_became_active":"2024-09-13T17:57:25.377386+0000","last_became_peered":"2024-09-13T17:57:25.377386+0000","last_unstale":"2024-09-13T17:57:29.725223+0000","last_undegraded":"2024-09-13T17:57:29.725223+0000","last_fullsized":"2024-09-13T17:57:29.725223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:38.038738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725142+0000","last_change":"2024-09-13T17:57:25.728949+0000","last_active":"2024-09-13T17:57:29.725142+0000","last_peered":"2024-09-13T17:57:29.725142+0000","last_clean":"2024-09-13T17:57:29.725142+0000","last_became_active":"2024-09-13T17:57:25.728675+0000","last_became_peered":"2024-09-13T17:57:25.728675+0000","last_unstale":"2024-09-13T17:57:29.725142+0000","last_undegraded":"2024-09-13T17:57:29.725142+0000","last_fullsized":"2024-09-13T17:57:29.725142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749006+0000","last_change":"2024-09-13T17:57:29.745555+0000","last_active":"2024-09-13T17:57:29.749006+0000","last_peered":"2024-09-13T17:57:29.749006+0000","last_clean":"2024-09-13T17:57:29.749006+0000","last_became_active":"2024-09-13T17:57:29.745172+0000","last_became_peered":"2024-09-13T17:57:29.745172+0000","last_unstale":"2024-09-13T17:57:29.749006+0000","last_undegraded":"2024-09-13T17:57:29.749006+0000","last_fullsized":"2024-09-13T17:57:29.749006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:37.101711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725546+0000","last_change":"2024-09-13T17:57:25.375351+0000","last_active":"2024-09-13T17:57:29.725546+0000","last_peered":"2024-09-13T17:57:29.725546+0000","last_clean":"2024-09-13T17:57:29.725546+0000","last_became_active":"2024-09-13T17:57:25.374550+0000","last_became_peered":"2024-09-13T17:57:25.374550+0000","last_unstale":"2024-09-13T17:57:29.725546+0000","last_undegraded":"2024-09-13T17:57:29.725546+0000","last_fullsized":"2024-09-13T17:57:29.725546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:11:57.892225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725579+0000","last_change":"2024-09-13T17:57:25.729527+0000","last_active":"2024-09-13T17:57:29.725579+0000","last_peered":"2024-09-13T17:57:29.725579+0000","last_clean":"2024-09-13T17:57:29.725579+0000","last_became_active":"2024-09-13T17:57:25.729225+0000","last_became_peered":"2024-09-13T17:57:25.729225+0000","last_unstale":"2024-09-13T17:57:29.725579+0000","last_undegraded":"2024-09-13T17:57:29.725579+0000","last_fullsized":"2024-09-13T17:57:29.725579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:27.969805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.748980+0000","last_change":"2024-09-13T17:57:29.745532+0000","last_active":"2024-09-13T17:57:29.748980+0000","last_peered":"2024-09-13T17:57:29.748980+0000","last_clean":"2024-09-13T17:57:29.748980+0000","last_became_active":"2024-09-13T17:57:29.745130+0000","last_became_peered":"2024-09-13T17:57:29.745130+0000","last_unstale":"2024-09-13T17:57:29.748980+0000","last_undegraded":"2024-09-13T17:57:29.748980+0000","last_fullsized":"2024-09-13T17:57:29.748980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:54.172556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728175+0000","last_change":"2024-09-13T17:57:25.735067+0000","last_active":"2024-09-13T17:57:29.728175+0000","last_peered":"2024-09-13T17:57:29.728175+0000","last_clean":"2024-09-13T17:57:29.728175+0000","last_became_active":"2024-09-13T17:57:25.734807+0000","last_became_peered":"2024-09-13T17:57:25.734807+0000","last_unstale":"2024-09-13T17:57:29.728175+0000","last_undegraded":"2024-09-13T17:57:29.728175+0000","last_fullsized":"2024-09-13T17:57:29.728175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:19.536367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725323+0000","last_change":"2024-09-13T17:57:25.374934+0000","last_active":"2024-09-13T17:57:29.725323+0000","last_peered":"2024-09-13T17:57:29.725323+0000","last_clean":"2024-09-13T17:57:29.725323+0000","last_became_active":"2024-09-13T17:57:25.374307+0000","last_became_peered":"2024-09-13T17:57:25.374307+0000","last_unstale":"2024-09-13T17:57:29.725323+0000","last_undegraded":"2024-09-13T17:57:29.725323+0000","last_fullsized":"2024-09-13T17:57:29.725323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:15.663292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.748917+0000","last_change":"2024-09-13T17:57:29.743757+0000","last_active":"2024-09-13T17:57:29.748917+0000","last_peered":"2024-09-13T17:57:29.748917+0000","last_clean":"2024-09-13T17:57:29.748917+0000","last_became_active":"2024-09-13T17:57:29.743362+0000","last_became_peered":"2024-09-13T17:57:29.743362+0000","last_unstale":"2024-09-13T17:57:29.748917+0000","last_undegraded":"2024-09-13T17:57:29.748917+0000","last_fullsized":"2024-09-13T17:57:29.748917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:31.285328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720769+0000","last_change":"2024-09-13T17:57:25.720127+0000","last_active":"2024-09-13T17:57:29.720769+0000","last_peered":"2024-09-13T17:57:29.720769+0000","last_clean":"2024-09-13T17:57:29.720769+0000","last_became_active":"2024-09-13T17:57:25.719766+0000","last_became_peered":"2024-09-13T17:57:25.719766+0000","last_unstale":"2024-09-13T17:57:29.720769+0000","last_undegraded":"2024-09-13T17:57:29.720769+0000","last_fullsized":"2024-09-13T17:57:29.720769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.730752+0000","last_change":"2024-09-13T17:57:25.381085+0000","last_active":"2024-09-13T17:57:29.730752+0000","last_peered":"2024-09-13T17:57:29.730752+0000","last_clean":"2024-09-13T17:57:29.730752+0000","last_became_active":"2024-09-13T17:57:25.380825+0000","last_became_peered":"2024-09-13T17:57:25.380825+0000","last_unstale":"2024-09-13T17:57:29.730752+0000","last_undegraded":"2024-09-13T17:57:29.730752+0000","last_fullsized":"2024-09-13T17:57:29.730752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:15.663292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749031+0000","last_change":"2024-09-13T17:57:29.746426+0000","last_active":"2024-09-13T17:57:29.749031+0000","last_peered":"2024-09-13T17:57:29.749031+0000","last_clean":"2024-09-13T17:57:29.749031+0000","last_became_active":"2024-09-13T17:57:29.745582+0000","last_became_peered":"2024-09-13T17:57:29.745582+0000","last_unstale":"2024-09-13T17:57:29.749031+0000","last_undegraded":"2024-09-13T17:57:29.749031+0000","last_fullsized":"2024-09-13T17:57:29.749031+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:53.991862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721690+0000","last_change":"2024-09-13T17:57:25.724108+0000","last_active":"2024-09-13T17:57:29.721690+0000","last_peered":"2024-09-13T17:57:29.721690+0000","last_clean":"2024-09-13T17:57:29.721690+0000","last_became_active":"2024-09-13T17:57:25.723857+0000","last_became_peered":"2024-09-13T17:57:25.723857+0000","last_unstale":"2024-09-13T17:57:29.721690+0000","last_undegraded":"2024-09-13T17:57:29.721690+0000","last_fullsized":"2024-09-13T17:57:29.721690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:19.536367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729185+0000","last_change":"2024-09-13T17:57:25.374954+0000","last_active":"2024-09-13T17:57:29.729185+0000","last_peered":"2024-09-13T17:57:29.729185+0000","last_clean":"2024-09-13T17:57:29.729185+0000","last_became_active":"2024-09-13T17:57:25.374721+0000","last_became_peered":"2024-09-13T17:57:25.374721+0000","last_unstale":"2024-09-13T17:57:29.729185+0000","last_undegraded":"2024-09-13T17:57:29.729185+0000","last_fullsized":"2024-09-13T17:57:29.729185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:34.109316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749541+0000","last_change":"2024-09-13T17:57:29.744520+0000","last_active":"2024-09-13T17:57:29.749541+0000","last_peered":"2024-09-13T17:57:29.749541+0000","last_clean":"2024-09-13T17:57:29.749541+0000","last_became_active":"2024-09-13T17:57:29.744143+0000","last_became_peered":"2024-09-13T17:57:29.744143+0000","last_unstale":"2024-09-13T17:57:29.749541+0000","last_undegraded":"2024-09-13T17:57:29.749541+0000","last_fullsized":"2024-09-13T17:57:29.749541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:18.526568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.733595+0000","last_change":"2024-09-13T17:57:25.718903+0000","last_active":"2024-09-13T17:57:29.733595+0000","last_peered":"2024-09-13T17:57:29.733595+0000","last_clean":"2024-09-13T17:57:29.733595+0000","last_became_active":"2024-09-13T17:57:25.718483+0000","last_became_peered":"2024-09-13T17:57:25.718483+0000","last_unstale":"2024-09-13T17:57:29.733595+0000","last_undegraded":"2024-09-13T17:57:29.733595+0000","last_fullsized":"2024-09-13T17:57:29.733595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:59.252453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725956+0000","last_change":"2024-09-13T17:57:25.378628+0000","last_active":"2024-09-13T17:57:29.725956+0000","last_peered":"2024-09-13T17:57:29.725956+0000","last_clean":"2024-09-13T17:57:29.725956+0000","last_became_active":"2024-09-13T17:57:25.378359+0000","last_became_peered":"2024-09-13T17:57:25.378359+0000","last_unstale":"2024-09-13T17:57:29.725956+0000","last_undegraded":"2024-09-13T17:57:29.725956+0000","last_fullsized":"2024-09-13T17:57:29.725956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:00.725292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749048+0000","last_change":"2024-09-13T17:57:29.746604+0000","last_active":"2024-09-13T17:57:29.749048+0000","last_peered":"2024-09-13T17:57:29.749048+0000","last_clean":"2024-09-13T17:57:29.749048+0000","last_became_active":"2024-09-13T17:57:29.746216+0000","last_became_peered":"2024-09-13T17:57:29.746216+0000","last_unstale":"2024-09-13T17:57:29.749048+0000","last_undegraded":"2024-09-13T17:57:29.749048+0000","last_fullsized":"2024-09-13T17:57:29.749048+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:18.526568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725404+0000","last_change":"2024-09-13T17:57:25.722540+0000","last_active":"2024-09-13T17:57:29.725404+0000","last_peered":"2024-09-13T17:57:29.725404+0000","last_clean":"2024-09-13T17:57:29.725404+0000","last_became_active":"2024-09-13T17:57:25.722301+0000","last_became_peered":"2024-09-13T17:57:25.722301+0000","last_unstale":"2024-09-13T17:57:29.725404+0000","last_undegraded":"2024-09-13T17:57:29.725404+0000","last_fullsized":"2024-09-13T17:57:29.725404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:34.117674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728850+0000","last_change":"2024-09-13T17:57:25.380087+0000","last_active":"2024-09-13T17:57:29.728850+0000","last_peered":"2024-09-13T17:57:29.728850+0000","last_clean":"2024-09-13T17:57:29.728850+0000","last_became_active":"2024-09-13T17:57:25.379808+0000","last_became_peered":"2024-09-13T17:57:25.379808+0000","last_unstale":"2024-09-13T17:57:29.728850+0000","last_undegraded":"2024-09-13T17:57:29.728850+0000","last_fullsized":"2024-09-13T17:57:29.728850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:38.038738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.743674+0000","last_change":"2024-09-13T17:57:29.740242+0000","last_active":"2024-09-13T17:57:29.743674+0000","last_peered":"2024-09-13T17:57:29.743674+0000","last_clean":"2024-09-13T17:57:29.743674+0000","last_became_active":"2024-09-13T17:57:29.739815+0000","last_became_peered":"2024-09-13T17:57:29.739815+0000","last_unstale":"2024-09-13T17:57:29.743674+0000","last_undegraded":"2024-09-13T17:57:29.743674+0000","last_fullsized":"2024-09-13T17:57:29.743674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:45.306949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720934+0000","last_change":"2024-09-13T17:57:25.718065+0000","last_active":"2024-09-13T17:57:29.720934+0000","last_peered":"2024-09-13T17:57:29.720934+0000","last_clean":"2024-09-13T17:57:29.720934+0000","last_became_active":"2024-09-13T17:57:25.717759+0000","last_became_peered":"2024-09-13T17:57:25.717759+0000","last_unstale":"2024-09-13T17:57:29.720934+0000","last_undegraded":"2024-09-13T17:57:29.720934+0000","last_fullsized":"2024-09-13T17:57:29.720934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.814140+0000","last_change":"2024-09-13T17:57:25.377272+0000","last_active":"2024-09-13T17:57:29.814140+0000","last_peered":"2024-09-13T17:57:29.814140+0000","last_clean":"2024-09-13T17:57:29.814140+0000","last_became_active":"2024-09-13T17:57:25.376938+0000","last_became_peered":"2024-09-13T17:57:25.376938+0000","last_unstale":"2024-09-13T17:57:29.814140+0000","last_undegraded":"2024-09-13T17:57:29.814140+0000","last_fullsized":"2024-09-13T17:57:29.814140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:00.725292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.746741+0000","last_change":"2024-09-13T17:57:29.743363+0000","last_active":"2024-09-13T17:57:29.746741+0000","last_peered":"2024-09-13T17:57:29.746741+0000","last_clean":"2024-09-13T17:57:29.746741+0000","last_became_active":"2024-09-13T17:57:29.738263+0000","last_became_peered":"2024-09-13T17:57:29.738263+0000","last_unstale":"2024-09-13T17:57:29.746741+0000","last_undegraded":"2024-09-13T17:57:29.746741+0000","last_fullsized":"2024-09-13T17:57:29.746741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:42:39.016490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729043+0000","last_change":"2024-09-13T17:57:25.717635+0000","last_active":"2024-09-13T17:57:29.729043+0000","last_peered":"2024-09-13T17:57:29.729043+0000","last_clean":"2024-09-13T17:57:29.729043+0000","last_became_active":"2024-09-13T17:57:25.717360+0000","last_became_peered":"2024-09-13T17:57:25.717360+0000","last_unstale":"2024-09-13T17:57:29.729043+0000","last_undegraded":"2024-09-13T17:57:29.729043+0000","last_fullsized":"2024-09-13T17:57:29.729043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725395+0000","last_change":"2024-09-13T17:57:25.378289+0000","last_active":"2024-09-13T17:57:29.725395+0000","last_peered":"2024-09-13T17:57:29.725395+0000","last_clean":"2024-09-13T17:57:29.725395+0000","last_became_active":"2024-09-13T17:57:25.378038+0000","last_became_peered":"2024-09-13T17:57:25.378038+0000","last_unstale":"2024-09-13T17:57:29.725395+0000","last_undegraded":"2024-09-13T17:57:29.725395+0000","last_fullsized":"2024-09-13T17:57:29.725395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:58.248434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.746870+0000","last_change":"2024-09-13T17:57:29.741347+0000","last_active":"2024-09-13T17:57:29.746870+0000","last_peered":"2024-09-13T17:57:29.746870+0000","last_clean":"2024-09-13T17:57:29.746870+0000","last_became_active":"2024-09-13T17:57:29.740980+0000","last_became_peered":"2024-09-13T17:57:29.740980+0000","last_unstale":"2024-09-13T17:57:29.746870+0000","last_undegraded":"2024-09-13T17:57:29.746870+0000","last_fullsized":"2024-09-13T17:57:29.746870+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:53.991862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725663+0000","last_change":"2024-09-13T17:57:25.728135+0000","last_active":"2024-09-13T17:57:29.725663+0000","last_peered":"2024-09-13T17:57:29.725663+0000","last_clean":"2024-09-13T17:57:29.725663+0000","last_became_active":"2024-09-13T17:57:25.727903+0000","last_became_peered":"2024-09-13T17:57:25.727903+0000","last_unstale":"2024-09-13T17:57:29.725663+0000","last_undegraded":"2024-09-13T17:57:29.725663+0000","last_fullsized":"2024-09-13T17:57:29.725663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:48.163930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729130+0000","last_change":"2024-09-13T17:57:25.375187+0000","last_active":"2024-09-13T17:57:29.729130+0000","last_peered":"2024-09-13T17:57:29.729130+0000","last_clean":"2024-09-13T17:57:29.729130+0000","last_became_active":"2024-09-13T17:57:25.374295+0000","last_became_peered":"2024-09-13T17:57:25.374295+0000","last_unstale":"2024-09-13T17:57:29.729130+0000","last_undegraded":"2024-09-13T17:57:29.729130+0000","last_fullsized":"2024-09-13T17:57:29.729130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:58.248434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742560+0000","last_change":"2024-09-13T17:57:29.740383+0000","last_active":"2024-09-13T17:57:29.742560+0000","last_peered":"2024-09-13T17:57:29.742560+0000","last_clean":"2024-09-13T17:57:29.742560+0000","last_became_active":"2024-09-13T17:57:29.739542+0000","last_became_peered":"2024-09-13T17:57:29.739542+0000","last_unstale":"2024-09-13T17:57:29.742560+0000","last_undegraded":"2024-09-13T17:57:29.742560+0000","last_fullsized":"2024-09-13T17:57:29.742560+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:13.633882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.734117+0000","last_change":"2024-09-13T17:57:25.717973+0000","last_active":"2024-09-13T17:57:29.734117+0000","last_peered":"2024-09-13T17:57:29.734117+0000","last_clean":"2024-09-13T17:57:29.734117+0000","last_became_active":"2024-09-13T17:57:25.717702+0000","last_became_peered":"2024-09-13T17:57:25.717702+0000","last_unstale":"2024-09-13T17:57:29.734117+0000","last_undegraded":"2024-09-13T17:57:29.734117+0000","last_fullsized":"2024-09-13T17:57:29.734117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:35.113334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725035+0000","last_change":"2024-09-13T17:57:25.373814+0000","last_active":"2024-09-13T17:57:29.725035+0000","last_peered":"2024-09-13T17:57:29.725035+0000","last_clean":"2024-09-13T17:57:29.725035+0000","last_became_active":"2024-09-13T17:57:25.373556+0000","last_became_peered":"2024-09-13T17:57:25.373556+0000","last_unstale":"2024-09-13T17:57:29.725035+0000","last_undegraded":"2024-09-13T17:57:29.725035+0000","last_fullsized":"2024-09-13T17:57:29.725035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:04.441662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742509+0000","last_change":"2024-09-13T17:57:29.739908+0000","last_active":"2024-09-13T17:57:29.742509+0000","last_peered":"2024-09-13T17:57:29.742509+0000","last_clean":"2024-09-13T17:57:29.742509+0000","last_became_active":"2024-09-13T17:57:29.739070+0000","last_became_peered":"2024-09-13T17:57:29.739070+0000","last_unstale":"2024-09-13T17:57:29.742509+0000","last_undegraded":"2024-09-13T17:57:29.742509+0000","last_fullsized":"2024-09-13T17:57:29.742509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:37.101711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729849+0000","last_change":"2024-09-13T17:57:25.718810+0000","last_active":"2024-09-13T17:57:29.729849+0000","last_peered":"2024-09-13T17:57:29.729849+0000","last_clean":"2024-09-13T17:57:29.729849+0000","last_became_active":"2024-09-13T17:57:25.718576+0000","last_became_peered":"2024-09-13T17:57:25.718576+0000","last_unstale":"2024-09-13T17:57:29.729849+0000","last_undegraded":"2024-09-13T17:57:29.729849+0000","last_fullsized":"2024-09-13T17:57:29.729849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:13.197645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732373+0000","last_change":"2024-09-13T17:57:25.380336+0000","last_active":"2024-09-13T17:57:29.732373+0000","last_peered":"2024-09-13T17:57:29.732373+0000","last_clean":"2024-09-13T17:57:29.732373+0000","last_became_active":"2024-09-13T17:57:25.380044+0000","last_became_peered":"2024-09-13T17:57:25.380044+0000","last_unstale":"2024-09-13T17:57:29.732373+0000","last_undegraded":"2024-09-13T17:57:29.732373+0000","last_fullsized":"2024-09-13T17:57:29.732373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:41.189680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.746550+0000","last_change":"2024-09-13T17:57:29.741637+0000","last_active":"2024-09-13T17:57:29.746550+0000","last_peered":"2024-09-13T17:57:29.746550+0000","last_clean":"2024-09-13T17:57:29.746550+0000","last_became_active":"2024-09-13T17:57:29.741260+0000","last_became_peered":"2024-09-13T17:57:29.741260+0000","last_unstale":"2024-09-13T17:57:29.746550+0000","last_undegraded":"2024-09-13T17:57:29.746550+0000","last_fullsized":"2024-09-13T17:57:29.746550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:53.991862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724599+0000","last_change":"2024-09-13T17:57:25.718276+0000","last_active":"2024-09-13T17:57:29.724599+0000","last_peered":"2024-09-13T17:57:29.724599+0000","last_clean":"2024-09-13T17:57:29.724599+0000","last_became_active":"2024-09-13T17:57:25.718000+0000","last_became_peered":"2024-09-13T17:57:25.718000+0000","last_unstale":"2024-09-13T17:57:29.724599+0000","last_undegraded":"2024-09-13T17:57:29.724599+0000","last_fullsized":"2024-09-13T17:57:29.724599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:05.710729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724849+0000","last_change":"2024-09-13T17:57:25.375143+0000","last_active":"2024-09-13T17:57:29.724849+0000","last_peered":"2024-09-13T17:57:29.724849+0000","last_clean":"2024-09-13T17:57:29.724849+0000","last_became_active":"2024-09-13T17:57:25.374871+0000","last_became_peered":"2024-09-13T17:57:25.374871+0000","last_unstale":"2024-09-13T17:57:29.724849+0000","last_undegraded":"2024-09-13T17:57:29.724849+0000","last_fullsized":"2024-09-13T17:57:29.724849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:34.109316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.738027+0000","last_change":"2024-09-13T17:57:29.733428+0000","last_active":"2024-09-13T17:57:29.738027+0000","last_peered":"2024-09-13T17:57:29.738027+0000","last_clean":"2024-09-13T17:57:29.738027+0000","last_became_active":"2024-09-13T17:57:29.732952+0000","last_became_peered":"2024-09-13T17:57:29.732952+0000","last_unstale":"2024-09-13T17:57:29.738027+0000","last_undegraded":"2024-09-13T17:57:29.738027+0000","last_fullsized":"2024-09-13T17:57:29.738027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:56:53.973670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720518+0000","last_change":"2024-09-13T17:57:25.732656+0000","last_active":"2024-09-13T17:57:29.720518+0000","last_peered":"2024-09-13T17:57:29.720518+0000","last_clean":"2024-09-13T17:57:29.720518+0000","last_became_active":"2024-09-13T17:57:25.732308+0000","last_became_peered":"2024-09-13T17:57:25.732308+0000","last_unstale":"2024-09-13T17:57:29.720518+0000","last_undegraded":"2024-09-13T17:57:29.720518+0000","last_fullsized":"2024-09-13T17:57:29.720518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:34.797142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724840+0000","last_change":"2024-09-13T17:57:25.374372+0000","last_active":"2024-09-13T17:57:29.724840+0000","last_peered":"2024-09-13T17:57:29.724840+0000","last_clean":"2024-09-13T17:57:29.724840+0000","last_became_active":"2024-09-13T17:57:25.374110+0000","last_became_peered":"2024-09-13T17:57:25.374110+0000","last_unstale":"2024-09-13T17:57:29.724840+0000","last_undegraded":"2024-09-13T17:57:29.724840+0000","last_fullsized":"2024-09-13T17:57:29.724840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:38.038738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724941+0000","last_change":"2024-09-13T17:57:15.560506+0000","last_active":"2024-09-13T17:57:29.724941+0000","last_peered":"2024-09-13T17:57:29.724941+0000","last_clean":"2024-09-13T17:57:29.724941+0000","last_became_active":"2024-09-13T17:57:15.559612+0000","last_became_peered":"2024-09-13T17:57:15.559612+0000","last_unstale":"2024-09-13T17:57:29.724941+0000","last_undegraded":"2024-09-13T17:57:29.724941+0000","last_fullsized":"2024-09-13T17:57:29.724941+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_clean_scrub_stamp":"2024-09-13T17:57:14.317378+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:16:27.663312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.743992+0000","last_change":"2024-09-13T17:57:29.739891+0000","last_active":"2024-09-13T17:57:29.743992+0000","last_peered":"2024-09-13T17:57:29.743992+0000","last_clean":"2024-09-13T17:57:29.743992+0000","last_became_active":"2024-09-13T17:57:29.739331+0000","last_became_peered":"2024-09-13T17:57:29.739331+0000","last_unstale":"2024-09-13T17:57:29.743992+0000","last_undegraded":"2024-09-13T17:57:29.743992+0000","last_fullsized":"2024-09-13T17:57:29.743992+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:45.306949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725127+0000","last_change":"2024-09-13T17:57:25.725987+0000","last_active":"2024-09-13T17:57:29.725127+0000","last_peered":"2024-09-13T17:57:29.725127+0000","last_clean":"2024-09-13T17:57:29.725127+0000","last_became_active":"2024-09-13T17:57:25.725695+0000","last_became_peered":"2024-09-13T17:57:25.725695+0000","last_unstale":"2024-09-13T17:57:29.725127+0000","last_undegraded":"2024-09-13T17:57:29.725127+0000","last_fullsized":"2024-09-13T17:57:29.725127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:48.163930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732467+0000","last_change":"2024-09-13T17:57:25.376170+0000","last_active":"2024-09-13T17:57:29.732467+0000","last_peered":"2024-09-13T17:57:29.732467+0000","last_clean":"2024-09-13T17:57:29.732467+0000","last_became_active":"2024-09-13T17:57:25.375798+0000","last_became_peered":"2024-09-13T17:57:25.375798+0000","last_unstale":"2024-09-13T17:57:29.732467+0000","last_undegraded":"2024-09-13T17:57:29.732467+0000","last_fullsized":"2024-09-13T17:57:29.732467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:08.374958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744268+0000","last_change":"2024-09-13T17:57:29.741819+0000","last_active":"2024-09-13T17:57:29.744268+0000","last_peered":"2024-09-13T17:57:29.744268+0000","last_clean":"2024-09-13T17:57:29.744268+0000","last_became_active":"2024-09-13T17:57:29.741410+0000","last_became_peered":"2024-09-13T17:57:29.741410+0000","last_unstale":"2024-09-13T17:57:29.744268+0000","last_undegraded":"2024-09-13T17:57:29.744268+0000","last_fullsized":"2024-09-13T17:57:29.744268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:15.299874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725178+0000","last_change":"2024-09-13T17:57:25.729870+0000","last_active":"2024-09-13T17:57:29.725178+0000","last_peered":"2024-09-13T17:57:29.725178+0000","last_clean":"2024-09-13T17:57:29.725178+0000","last_became_active":"2024-09-13T17:57:25.728383+0000","last_became_peered":"2024-09-13T17:57:25.728383+0000","last_unstale":"2024-09-13T17:57:29.725178+0000","last_undegraded":"2024-09-13T17:57:29.725178+0000","last_fullsized":"2024-09-13T17:57:29.725178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:51.804478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.730112+0000","last_change":"2024-09-13T17:57:25.373843+0000","last_active":"2024-09-13T17:57:29.730112+0000","last_peered":"2024-09-13T17:57:29.730112+0000","last_clean":"2024-09-13T17:57:29.730112+0000","last_became_active":"2024-09-13T17:57:25.373498+0000","last_became_peered":"2024-09-13T17:57:25.373498+0000","last_unstale":"2024-09-13T17:57:29.730112+0000","last_undegraded":"2024-09-13T17:57:29.730112+0000","last_fullsized":"2024-09-13T17:57:29.730112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:11:57.892225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742650+0000","last_change":"2024-09-13T17:57:29.735791+0000","last_active":"2024-09-13T17:57:29.742650+0000","last_peered":"2024-09-13T17:57:29.742650+0000","last_clean":"2024-09-13T17:57:29.742650+0000","last_became_active":"2024-09-13T17:57:29.735444+0000","last_became_peered":"2024-09-13T17:57:29.735444+0000","last_unstale":"2024-09-13T17:57:29.742650+0000","last_undegraded":"2024-09-13T17:57:29.742650+0000","last_fullsized":"2024-09-13T17:57:29.742650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:13.633882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725552+0000","last_change":"2024-09-13T17:57:25.724498+0000","last_active":"2024-09-13T17:57:29.725552+0000","last_peered":"2024-09-13T17:57:29.725552+0000","last_clean":"2024-09-13T17:57:29.725552+0000","last_became_active":"2024-09-13T17:57:25.724159+0000","last_became_peered":"2024-09-13T17:57:25.724159+0000","last_unstale":"2024-09-13T17:57:29.725552+0000","last_undegraded":"2024-09-13T17:57:29.725552+0000","last_fullsized":"2024-09-13T17:57:29.725552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:34.797142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732295+0000","last_change":"2024-09-13T17:57:25.380132+0000","last_active":"2024-09-13T17:57:29.732295+0000","last_peered":"2024-09-13T17:57:29.732295+0000","last_clean":"2024-09-13T17:57:29.732295+0000","last_became_active":"2024-09-13T17:57:25.379862+0000","last_became_peered":"2024-09-13T17:57:25.379862+0000","last_unstale":"2024-09-13T17:57:29.732295+0000","last_undegraded":"2024-09-13T17:57:29.732295+0000","last_fullsized":"2024-09-13T17:57:29.732295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:43:20.708183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.740512+0000","last_change":"2024-09-13T17:57:29.738502+0000","last_active":"2024-09-13T17:57:29.740512+0000","last_peered":"2024-09-13T17:57:29.740512+0000","last_clean":"2024-09-13T17:57:29.740512+0000","last_became_active":"2024-09-13T17:57:29.737664+0000","last_became_peered":"2024-09-13T17:57:29.737664+0000","last_unstale":"2024-09-13T17:57:29.740512+0000","last_undegraded":"2024-09-13T17:57:29.740512+0000","last_fullsized":"2024-09-13T17:57:29.740512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:13.633882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725043+0000","last_change":"2024-09-13T17:57:25.716646+0000","last_active":"2024-09-13T17:57:29.725043+0000","last_peered":"2024-09-13T17:57:29.725043+0000","last_clean":"2024-09-13T17:57:29.725043+0000","last_became_active":"2024-09-13T17:57:25.716005+0000","last_became_peered":"2024-09-13T17:57:25.716005+0000","last_unstale":"2024-09-13T17:57:29.725043+0000","last_undegraded":"2024-09-13T17:57:29.725043+0000","last_fullsized":"2024-09-13T17:57:29.725043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:43.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.730970+0000","last_change":"2024-09-13T17:57:25.374986+0000","last_active":"2024-09-13T17:57:29.730970+0000","last_peered":"2024-09-13T17:57:29.730970+0000","last_clean":"2024-09-13T17:57:29.730970+0000","last_became_active":"2024-09-13T17:57:25.374609+0000","last_became_peered":"2024-09-13T17:57:25.374609+0000","last_unstale":"2024-09-13T17:57:29.730970+0000","last_undegraded":"2024-09-13T17:57:29.730970+0000","last_fullsized":"2024-09-13T17:57:29.730970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:43:20.708183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749009+0000","last_change":"2024-09-13T17:57:29.745707+0000","last_active":"2024-09-13T17:57:29.749009+0000","last_peered":"2024-09-13T17:57:29.749009+0000","last_clean":"2024-09-13T17:57:29.749009+0000","last_became_active":"2024-09-13T17:57:29.745303+0000","last_became_peered":"2024-09-13T17:57:29.745303+0000","last_unstale":"2024-09-13T17:57:29.749009+0000","last_undegraded":"2024-09-13T17:57:29.749009+0000","last_fullsized":"2024-09-13T17:57:29.749009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:06.371464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725823+0000","last_change":"2024-09-13T17:57:25.717385+0000","last_active":"2024-09-13T17:57:29.725823+0000","last_peered":"2024-09-13T17:57:29.725823+0000","last_clean":"2024-09-13T17:57:29.725823+0000","last_became_active":"2024-09-13T17:57:25.717042+0000","last_became_peered":"2024-09-13T17:57:25.717042+0000","last_unstale":"2024-09-13T17:57:29.725823+0000","last_undegraded":"2024-09-13T17:57:29.725823+0000","last_fullsized":"2024-09-13T17:57:29.725823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:49.905668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732190+0000","last_change":"2024-09-13T17:57:25.369802+0000","last_active":"2024-09-13T17:57:29.732190+0000","last_peered":"2024-09-13T17:57:29.732190+0000","last_clean":"2024-09-13T17:57:29.732190+0000","last_became_active":"2024-09-13T17:57:25.369534+0000","last_became_peered":"2024-09-13T17:57:25.369534+0000","last_unstale":"2024-09-13T17:57:29.732190+0000","last_undegraded":"2024-09-13T17:57:29.732190+0000","last_fullsized":"2024-09-13T17:57:29.732190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742623+0000","last_change":"2024-09-13T17:57:29.737826+0000","last_active":"2024-09-13T17:57:29.742623+0000","last_peered":"2024-09-13T17:57:29.742623+0000","last_clean":"2024-09-13T17:57:29.742623+0000","last_became_active":"2024-09-13T17:57:29.737181+0000","last_became_peered":"2024-09-13T17:57:29.737181+0000","last_unstale":"2024-09-13T17:57:29.742623+0000","last_undegraded":"2024-09-13T17:57:29.742623+0000","last_fullsized":"2024-09-13T17:57:29.742623+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:06.225969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728642+0000","last_change":"2024-09-13T17:57:25.734250+0000","last_active":"2024-09-13T17:57:29.728642+0000","last_peered":"2024-09-13T17:57:29.728642+0000","last_clean":"2024-09-13T17:57:29.728642+0000","last_became_active":"2024-09-13T17:57:25.733836+0000","last_became_peered":"2024-09-13T17:57:25.733836+0000","last_unstale":"2024-09-13T17:57:29.728642+0000","last_undegraded":"2024-09-13T17:57:29.728642+0000","last_fullsized":"2024-09-13T17:57:29.728642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:05.710729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728072+0000","last_change":"2024-09-13T17:57:25.374153+0000","last_active":"2024-09-13T17:57:29.728072+0000","last_peered":"2024-09-13T17:57:29.728072+0000","last_clean":"2024-09-13T17:57:29.728072+0000","last_became_active":"2024-09-13T17:57:25.373891+0000","last_became_peered":"2024-09-13T17:57:25.373891+0000","last_unstale":"2024-09-13T17:57:29.728072+0000","last_undegraded":"2024-09-13T17:57:29.728072+0000","last_fullsized":"2024-09-13T17:57:29.728072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744947+0000","last_change":"2024-09-13T17:57:29.741648+0000","last_active":"2024-09-13T17:57:29.744947+0000","last_peered":"2024-09-13T17:57:29.744947+0000","last_clean":"2024-09-13T17:57:29.744947+0000","last_became_active":"2024-09-13T17:57:29.741315+0000","last_became_peered":"2024-09-13T17:57:29.741315+0000","last_unstale":"2024-09-13T17:57:29.744947+0000","last_undegraded":"2024-09-13T17:57:29.744947+0000","last_fullsized":"2024-09-13T17:57:29.744947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:18.432911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725228+0000","last_change":"2024-09-13T17:57:25.716312+0000","last_active":"2024-09-13T17:57:29.725228+0000","last_peered":"2024-09-13T17:57:29.725228+0000","last_clean":"2024-09-13T17:57:29.725228+0000","last_became_active":"2024-09-13T17:57:25.716067+0000","last_became_peered":"2024-09-13T17:57:25.716067+0000","last_unstale":"2024-09-13T17:57:29.725228+0000","last_undegraded":"2024-09-13T17:57:29.725228+0000","last_fullsized":"2024-09-13T17:57:29.725228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:33.971487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724948+0000","last_change":"2024-09-13T17:57:25.380277+0000","last_active":"2024-09-13T17:57:29.724948+0000","last_peered":"2024-09-13T17:57:29.724948+0000","last_clean":"2024-09-13T17:57:29.724948+0000","last_became_active":"2024-09-13T17:57:25.379988+0000","last_became_peered":"2024-09-13T17:57:25.379988+0000","last_unstale":"2024-09-13T17:57:29.724948+0000","last_undegraded":"2024-09-13T17:57:29.724948+0000","last_fullsized":"2024-09-13T17:57:29.724948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744773+0000","last_change":"2024-09-13T17:57:29.733138+0000","last_active":"2024-09-13T17:57:29.744773+0000","last_peered":"2024-09-13T17:57:29.744773+0000","last_clean":"2024-09-13T17:57:29.744773+0000","last_became_active":"2024-09-13T17:57:29.732804+0000","last_became_peered":"2024-09-13T17:57:29.732804+0000","last_unstale":"2024-09-13T17:57:29.744773+0000","last_undegraded":"2024-09-13T17:57:29.744773+0000","last_fullsized":"2024-09-13T17:57:29.744773+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:08.695591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724921+0000","last_change":"2024-09-13T17:57:25.714463+0000","last_active":"2024-09-13T17:57:29.724921+0000","last_peered":"2024-09-13T17:57:29.724921+0000","last_clean":"2024-09-13T17:57:29.724921+0000","last_became_active":"2024-09-13T17:57:25.714209+0000","last_became_peered":"2024-09-13T17:57:25.714209+0000","last_unstale":"2024-09-13T17:57:29.724921+0000","last_undegraded":"2024-09-13T17:57:29.724921+0000","last_fullsized":"2024-09-13T17:57:29.724921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727847+0000","last_change":"2024-09-13T17:57:25.374886+0000","last_active":"2024-09-13T17:57:29.727847+0000","last_peered":"2024-09-13T17:57:29.727847+0000","last_clean":"2024-09-13T17:57:29.727847+0000","last_became_active":"2024-09-13T17:57:25.374025+0000","last_became_peered":"2024-09-13T17:57:25.374025+0000","last_unstale":"2024-09-13T17:57:29.727847+0000","last_undegraded":"2024-09-13T17:57:29.727847+0000","last_fullsized":"2024-09-13T17:57:29.727847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:41.189680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.739379+0000","last_change":"2024-09-13T17:57:29.735654+0000","last_active":"2024-09-13T17:57:29.739379+0000","last_peered":"2024-09-13T17:57:29.739379+0000","last_clean":"2024-09-13T17:57:29.739379+0000","last_became_active":"2024-09-13T17:57:29.735267+0000","last_became_peered":"2024-09-13T17:57:29.735267+0000","last_unstale":"2024-09-13T17:57:29.739379+0000","last_undegraded":"2024-09-13T17:57:29.739379+0000","last_fullsized":"2024-09-13T17:57:29.739379+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:59:14.990893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725041+0000","last_change":"2024-09-13T17:57:25.722101+0000","last_active":"2024-09-13T17:57:29.725041+0000","last_peered":"2024-09-13T17:57:29.725041+0000","last_clean":"2024-09-13T17:57:29.725041+0000","last_became_active":"2024-09-13T17:57:25.721845+0000","last_became_peered":"2024-09-13T17:57:25.721845+0000","last_unstale":"2024-09-13T17:57:29.725041+0000","last_undegraded":"2024-09-13T17:57:29.725041+0000","last_fullsized":"2024-09-13T17:57:29.725041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:11.996031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:57:34.418487+0000","last_change":"2024-09-13T17:57:25.378020+0000","last_active":"2024-09-13T17:57:34.418487+0000","last_peered":"2024-09-13T17:57:34.418487+0000","last_clean":"2024-09-13T17:57:34.418487+0000","last_became_active":"2024-09-13T17:57:25.377746+0000","last_became_peered":"2024-09-13T17:57:25.377746+0000","last_unstale":"2024-09-13T17:57:34.418487+0000","last_undegraded":"2024-09-13T17:57:34.418487+0000","last_fullsized":"2024-09-13T17:57:34.418487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:58.248434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742293+0000","last_change":"2024-09-13T17:57:29.738264+0000","last_active":"2024-09-13T17:57:29.742293+0000","last_peered":"2024-09-13T17:57:29.742293+0000","last_clean":"2024-09-13T17:57:29.742293+0000","last_became_active":"2024-09-13T17:57:29.737897+0000","last_became_peered":"2024-09-13T17:57:29.737897+0000","last_unstale":"2024-09-13T17:57:29.742293+0000","last_undegraded":"2024-09-13T17:57:29.742293+0000","last_fullsized":"2024-09-13T17:57:29.742293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:59:14.990893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.735362+0000","last_change":"2024-09-13T17:57:25.720986+0000","last_active":"2024-09-13T17:57:29.735362+0000","last_peered":"2024-09-13T17:57:29.735362+0000","last_clean":"2024-09-13T17:57:29.735362+0000","last_became_active":"2024-09-13T17:57:25.720696+0000","last_became_peered":"2024-09-13T17:57:25.720696+0000","last_unstale":"2024-09-13T17:57:29.735362+0000","last_undegraded":"2024-09-13T17:57:29.735362+0000","last_fullsized":"2024-09-13T17:57:29.735362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:01.729310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729997+0000","last_change":"2024-09-13T17:57:25.376707+0000","last_active":"2024-09-13T17:57:29.729997+0000","last_peered":"2024-09-13T17:57:29.729997+0000","last_clean":"2024-09-13T17:57:29.729997+0000","last_became_active":"2024-09-13T17:57:25.376080+0000","last_became_peered":"2024-09-13T17:57:25.376080+0000","last_unstale":"2024-09-13T17:57:29.729997+0000","last_undegraded":"2024-09-13T17:57:29.729997+0000","last_fullsized":"2024-09-13T17:57:29.729997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:28.985664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744127+0000","last_change":"2024-09-13T17:57:29.741053+0000","last_active":"2024-09-13T17:57:29.744127+0000","last_peered":"2024-09-13T17:57:29.744127+0000","last_clean":"2024-09-13T17:57:29.744127+0000","last_became_active":"2024-09-13T17:57:29.740751+0000","last_became_peered":"2024-09-13T17:57:29.740751+0000","last_unstale":"2024-09-13T17:57:29.744127+0000","last_undegraded":"2024-09-13T17:57:29.744127+0000","last_fullsized":"2024-09-13T17:57:29.744127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:56:53.973670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728126+0000","last_change":"2024-09-13T17:57:25.725767+0000","last_active":"2024-09-13T17:57:29.728126+0000","last_peered":"2024-09-13T17:57:29.728126+0000","last_clean":"2024-09-13T17:57:29.728126+0000","last_became_active":"2024-09-13T17:57:25.725565+0000","last_became_peered":"2024-09-13T17:57:25.725565+0000","last_unstale":"2024-09-13T17:57:29.728126+0000","last_undegraded":"2024-09-13T17:57:29.728126+0000","last_fullsized":"2024-09-13T17:57:29.728126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:19.536367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725379+0000","last_change":"2024-09-13T17:57:25.380232+0000","last_active":"2024-09-13T17:57:29.725379+0000","last_peered":"2024-09-13T17:57:29.725379+0000","last_clean":"2024-09-13T17:57:29.725379+0000","last_became_active":"2024-09-13T17:57:25.379910+0000","last_became_peered":"2024-09-13T17:57:25.379910+0000","last_unstale":"2024-09-13T17:57:29.725379+0000","last_undegraded":"2024-09-13T17:57:29.725379+0000","last_fullsized":"2024-09-13T17:57:29.725379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:28.958967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749501+0000","last_change":"2024-09-13T17:57:29.745584+0000","last_active":"2024-09-13T17:57:29.749501+0000","last_peered":"2024-09-13T17:57:29.749501+0000","last_clean":"2024-09-13T17:57:29.749501+0000","last_became_active":"2024-09-13T17:57:29.745250+0000","last_became_peered":"2024-09-13T17:57:29.745250+0000","last_unstale":"2024-09-13T17:57:29.749501+0000","last_undegraded":"2024-09-13T17:57:29.749501+0000","last_fullsized":"2024-09-13T17:57:29.749501+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:12.564762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725293+0000","last_change":"2024-09-13T17:57:25.724239+0000","last_active":"2024-09-13T17:57:29.725293+0000","last_peered":"2024-09-13T17:57:29.725293+0000","last_clean":"2024-09-13T17:57:29.725293+0000","last_became_active":"2024-09-13T17:57:25.723999+0000","last_became_peered":"2024-09-13T17:57:25.723999+0000","last_unstale":"2024-09-13T17:57:29.725293+0000","last_undegraded":"2024-09-13T17:57:29.725293+0000","last_fullsized":"2024-09-13T17:57:29.725293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728742+0000","last_change":"2024-09-13T17:57:25.376498+0000","last_active":"2024-09-13T17:57:29.728742+0000","last_peered":"2024-09-13T17:57:29.728742+0000","last_clean":"2024-09-13T17:57:29.728742+0000","last_became_active":"2024-09-13T17:57:25.376171+0000","last_became_peered":"2024-09-13T17:57:25.376171+0000","last_unstale":"2024-09-13T17:57:29.728742+0000","last_undegraded":"2024-09-13T17:57:29.728742+0000","last_fullsized":"2024-09-13T17:57:29.728742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:00.725292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744624+0000","last_change":"2024-09-13T17:57:29.737219+0000","last_active":"2024-09-13T17:57:29.744624+0000","last_peered":"2024-09-13T17:57:29.744624+0000","last_clean":"2024-09-13T17:57:29.744624+0000","last_became_active":"2024-09-13T17:57:29.736853+0000","last_became_peered":"2024-09-13T17:57:29.736853+0000","last_unstale":"2024-09-13T17:57:29.744624+0000","last_undegraded":"2024-09-13T17:57:29.744624+0000","last_fullsized":"2024-09-13T17:57:29.744624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:18.526568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.738047+0000","last_change":"2024-09-13T17:57:29.733906+0000","last_active":"2024-09-13T17:57:29.738047+0000","last_peered":"2024-09-13T17:57:29.738047+0000","last_clean":"2024-09-13T17:57:29.738047+0000","last_became_active":"2024-09-13T17:57:29.733543+0000","last_became_peered":"2024-09-13T17:57:29.733543+0000","last_unstale":"2024-09-13T17:57:29.738047+0000","last_undegraded":"2024-09-13T17:57:29.738047+0000","last_fullsized":"2024-09-13T17:57:29.738047+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:37.101711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729014+0000","last_change":"2024-09-13T17:57:25.721000+0000","last_active":"2024-09-13T17:57:29.729014+0000","last_peered":"2024-09-13T17:57:29.729014+0000","last_clean":"2024-09-13T17:57:29.729014+0000","last_became_active":"2024-09-13T17:57:25.720696+0000","last_became_peered":"2024-09-13T17:57:25.720696+0000","last_unstale":"2024-09-13T17:57:29.729014+0000","last_undegraded":"2024-09-13T17:57:29.729014+0000","last_fullsized":"2024-09-13T17:57:29.729014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:34.797142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724465+0000","last_change":"2024-09-13T17:57:25.374530+0000","last_active":"2024-09-13T17:57:29.724465+0000","last_peered":"2024-09-13T17:57:29.724465+0000","last_clean":"2024-09-13T17:57:29.724465+0000","last_became_active":"2024-09-13T17:57:25.374296+0000","last_became_peered":"2024-09-13T17:57:25.374296+0000","last_unstale":"2024-09-13T17:57:29.724465+0000","last_undegraded":"2024-09-13T17:57:29.724465+0000","last_fullsized":"2024-09-13T17:57:29.724465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:15.663292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744054+0000","last_change":"2024-09-13T17:57:29.740208+0000","last_active":"2024-09-13T17:57:29.744054+0000","last_peered":"2024-09-13T17:57:29.744054+0000","last_clean":"2024-09-13T17:57:29.744054+0000","last_became_active":"2024-09-13T17:57:29.739839+0000","last_became_peered":"2024-09-13T17:57:29.739839+0000","last_unstale":"2024-09-13T17:57:29.744054+0000","last_undegraded":"2024-09-13T17:57:29.744054+0000","last_fullsized":"2024-09-13T17:57:29.744054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:32.682495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721617+0000","last_change":"2024-09-13T17:57:25.720286+0000","last_active":"2024-09-13T17:57:29.721617+0000","last_peered":"2024-09-13T17:57:29.721617+0000","last_clean":"2024-09-13T17:57:29.721617+0000","last_became_active":"2024-09-13T17:57:25.719918+0000","last_became_peered":"2024-09-13T17:57:25.719918+0000","last_unstale":"2024-09-13T17:57:29.721617+0000","last_undegraded":"2024-09-13T17:57:29.721617+0000","last_fullsized":"2024-09-13T17:57:29.721617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:13.197645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721496+0000","last_change":"2024-09-13T17:57:25.380282+0000","last_active":"2024-09-13T17:57:29.721496+0000","last_peered":"2024-09-13T17:57:29.721496+0000","last_clean":"2024-09-13T17:57:29.721496+0000","last_became_active":"2024-09-13T17:57:25.379992+0000","last_became_peered":"2024-09-13T17:57:25.379992+0000","last_unstale":"2024-09-13T17:57:29.721496+0000","last_undegraded":"2024-09-13T17:57:29.721496+0000","last_fullsized":"2024-09-13T17:57:29.721496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:04.441662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728265+0000","last_change":"2024-09-13T17:57:25.373463+0000","last_active":"2024-09-13T17:57:29.728265+0000","last_peered":"2024-09-13T17:57:29.728265+0000","last_clean":"2024-09-13T17:57:29.728265+0000","last_became_active":"2024-09-13T17:57:25.373127+0000","last_became_peered":"2024-09-13T17:57:25.373127+0000","last_unstale":"2024-09-13T17:57:29.728265+0000","last_undegraded":"2024-09-13T17:57:29.728265+0000","last_fullsized":"2024-09-13T17:57:29.728265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:15.663292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727753+0000","last_change":"2024-09-13T17:57:25.719108+0000","last_active":"2024-09-13T17:57:29.727753+0000","last_peered":"2024-09-13T17:57:29.727753+0000","last_clean":"2024-09-13T17:57:29.727753+0000","last_became_active":"2024-09-13T17:57:25.718674+0000","last_became_peered":"2024-09-13T17:57:25.718674+0000","last_unstale":"2024-09-13T17:57:29.727753+0000","last_undegraded":"2024-09-13T17:57:29.727753+0000","last_fullsized":"2024-09-13T17:57:29.727753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:49.905668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.740987+0000","last_change":"2024-09-13T17:57:29.736695+0000","last_active":"2024-09-13T17:57:29.740987+0000","last_peered":"2024-09-13T17:57:29.740987+0000","last_clean":"2024-09-13T17:57:29.740987+0000","last_became_active":"2024-09-13T17:57:29.735361+0000","last_became_peered":"2024-09-13T17:57:29.735361+0000","last_unstale":"2024-09-13T17:57:29.740987+0000","last_undegraded":"2024-09-13T17:57:29.740987+0000","last_fullsized":"2024-09-13T17:57:29.740987+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:50.823708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720788+0000","last_change":"2024-09-13T17:57:25.378442+0000","last_active":"2024-09-13T17:57:29.720788+0000","last_peered":"2024-09-13T17:57:29.720788+0000","last_clean":"2024-09-13T17:57:29.720788+0000","last_became_active":"2024-09-13T17:57:25.378092+0000","last_became_peered":"2024-09-13T17:57:25.378092+0000","last_unstale":"2024-09-13T17:57:29.720788+0000","last_undegraded":"2024-09-13T17:57:29.720788+0000","last_fullsized":"2024-09-13T17:57:29.720788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:41.189680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.733727+0000","last_change":"2024-09-13T17:57:25.714970+0000","last_active":"2024-09-13T17:57:29.733727+0000","last_peered":"2024-09-13T17:57:29.733727+0000","last_clean":"2024-09-13T17:57:29.733727+0000","last_became_active":"2024-09-13T17:57:25.714338+0000","last_became_peered":"2024-09-13T17:57:25.714338+0000","last_unstale":"2024-09-13T17:57:29.733727+0000","last_undegraded":"2024-09-13T17:57:29.733727+0000","last_fullsized":"2024-09-13T17:57:29.733727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:11:58.896243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.745004+0000","last_change":"2024-09-13T17:57:29.741838+0000","last_active":"2024-09-13T17:57:29.745004+0000","last_peered":"2024-09-13T17:57:29.745004+0000","last_clean":"2024-09-13T17:57:29.745004+0000","last_became_active":"2024-09-13T17:57:29.741478+0000","last_became_peered":"2024-09-13T17:57:29.741478+0000","last_unstale":"2024-09-13T17:57:29.745004+0000","last_undegraded":"2024-09-13T17:57:29.745004+0000","last_fullsized":"2024-09-13T17:57:29.745004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:15.815856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728583+0000","last_change":"2024-09-13T17:57:25.734081+0000","last_active":"2024-09-13T17:57:29.728583+0000","last_peered":"2024-09-13T17:57:29.728583+0000","last_clean":"2024-09-13T17:57:29.728583+0000","last_became_active":"2024-09-13T17:57:25.733489+0000","last_became_peered":"2024-09-13T17:57:25.733489+0000","last_unstale":"2024-09-13T17:57:29.728583+0000","last_undegraded":"2024-09-13T17:57:29.728583+0000","last_fullsized":"2024-09-13T17:57:29.728583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:49.905668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.731942+0000","last_change":"2024-09-13T17:57:25.721413+0000","last_active":"2024-09-13T17:57:29.731942+0000","last_peered":"2024-09-13T17:57:29.731942+0000","last_clean":"2024-09-13T17:57:29.731942+0000","last_became_active":"2024-09-13T17:57:25.721167+0000","last_became_peered":"2024-09-13T17:57:25.721167+0000","last_unstale":"2024-09-13T17:57:29.731942+0000","last_undegraded":"2024-09-13T17:57:29.731942+0000","last_fullsized":"2024-09-13T17:57:29.731942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:39.042757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724511+0000","last_change":"2024-09-13T17:57:25.728836+0000","last_active":"2024-09-13T17:57:29.724511+0000","last_peered":"2024-09-13T17:57:29.724511+0000","last_clean":"2024-09-13T17:57:29.724511+0000","last_became_active":"2024-09-13T17:57:25.728566+0000","last_became_peered":"2024-09-13T17:57:25.728566+0000","last_unstale":"2024-09-13T17:57:29.724511+0000","last_undegraded":"2024-09-13T17:57:29.724511+0000","last_fullsized":"2024-09-13T17:57:29.724511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:50.812435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727862+0000","last_change":"2024-09-13T17:57:25.719773+0000","last_active":"2024-09-13T17:57:29.727862+0000","last_peered":"2024-09-13T17:57:29.727862+0000","last_clean":"2024-09-13T17:57:29.727862+0000","last_became_active":"2024-09-13T17:57:25.719057+0000","last_became_peered":"2024-09-13T17:57:25.719057+0000","last_unstale":"2024-09-13T17:57:29.727862+0000","last_undegraded":"2024-09-13T17:57:29.727862+0000","last_fullsized":"2024-09-13T17:57:29.727862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:13.197645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721032+0000","last_change":"2024-09-13T17:57:25.719998+0000","last_active":"2024-09-13T17:57:29.721032+0000","last_peered":"2024-09-13T17:57:29.721032+0000","last_clean":"2024-09-13T17:57:29.721032+0000","last_became_active":"2024-09-13T17:57:25.719739+0000","last_became_peered":"2024-09-13T17:57:25.719739+0000","last_unstale":"2024-09-13T17:57:29.721032+0000","last_undegraded":"2024-09-13T17:57:29.721032+0000","last_fullsized":"2024-09-13T17:57:29.721032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:47.463588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729592+0000","last_change":"2024-09-13T17:57:25.720277+0000","last_active":"2024-09-13T17:57:29.729592+0000","last_peered":"2024-09-13T17:57:29.729592+0000","last_clean":"2024-09-13T17:57:29.729592+0000","last_became_active":"2024-09-13T17:57:25.720034+0000","last_became_peered":"2024-09-13T17:57:25.720034+0000","last_unstale":"2024-09-13T17:57:29.729592+0000","last_undegraded":"2024-09-13T17:57:29.729592+0000","last_fullsized":"2024-09-13T17:57:29.729592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728536+0000","last_change":"2024-09-13T17:57:25.733794+0000","last_active":"2024-09-13T17:57:29.728536+0000","last_peered":"2024-09-13T17:57:29.728536+0000","last_clean":"2024-09-13T17:57:29.728536+0000","last_became_active":"2024-09-13T17:57:25.733512+0000","last_became_peered":"2024-09-13T17:57:25.733512+0000","last_unstale":"2024-09-13T17:57:29.728536+0000","last_undegraded":"2024-09-13T17:57:29.728536+0000","last_fullsized":"2024-09-13T17:57:29.728536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720839+0000","last_change":"2024-09-13T17:57:25.723300+0000","last_active":"2024-09-13T17:57:29.720839+0000","last_peered":"2024-09-13T17:57:29.720839+0000","last_clean":"2024-09-13T17:57:29.720839+0000","last_became_active":"2024-09-13T17:57:25.720475+0000","last_became_peered":"2024-09-13T17:57:25.720475+0000","last_unstale":"2024-09-13T17:57:29.720839+0000","last_undegraded":"2024-09-13T17:57:29.720839+0000","last_fullsized":"2024-09-13T17:57:29.720839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:37:23.859769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724381+0000","last_change":"2024-09-13T17:57:25.721201+0000","last_active":"2024-09-13T17:57:29.724381+0000","last_peered":"2024-09-13T17:57:29.724381+0000","last_clean":"2024-09-13T17:57:29.724381+0000","last_became_active":"2024-09-13T17:57:25.719998+0000","last_became_peered":"2024-09-13T17:57:25.719998+0000","last_unstale":"2024-09-13T17:57:29.724381+0000","last_undegraded":"2024-09-13T17:57:29.724381+0000","last_fullsized":"2024-09-13T17:57:29.724381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:33.971487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721180+0000","last_change":"2024-09-13T17:57:25.715726+0000","last_active":"2024-09-13T17:57:29.721180+0000","last_peered":"2024-09-13T17:57:29.721180+0000","last_clean":"2024-09-13T17:57:29.721180+0000","last_became_active":"2024-09-13T17:57:25.715459+0000","last_became_peered":"2024-09-13T17:57:25.715459+0000","last_unstale":"2024-09-13T17:57:29.721180+0000","last_undegraded":"2024-09-13T17:57:29.721180+0000","last_fullsized":"2024-09-13T17:57:29.721180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:49.905668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724413+0000","last_change":"2024-09-13T17:57:25.730059+0000","last_active":"2024-09-13T17:57:29.724413+0000","last_peered":"2024-09-13T17:57:29.724413+0000","last_clean":"2024-09-13T17:57:29.724413+0000","last_became_active":"2024-09-13T17:57:25.729083+0000","last_became_peered":"2024-09-13T17:57:25.729083+0000","last_unstale":"2024-09-13T17:57:29.724413+0000","last_undegraded":"2024-09-13T17:57:29.724413+0000","last_fullsized":"2024-09-13T17:57:29.724413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:12.960061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727633+0000","last_change":"2024-09-13T17:57:25.713791+0000","last_active":"2024-09-13T17:57:29.727633+0000","last_peered":"2024-09-13T17:57:29.727633+0000","last_clean":"2024-09-13T17:57:29.727633+0000","last_became_active":"2024-09-13T17:57:25.713540+0000","last_became_peered":"2024-09-13T17:57:25.713540+0000","last_unstale":"2024-09-13T17:57:29.727633+0000","last_undegraded":"2024-09-13T17:57:29.727633+0000","last_fullsized":"2024-09-13T17:57:29.727633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728046+0000","last_change":"2024-09-13T17:57:25.734509+0000","last_active":"2024-09-13T17:57:29.728046+0000","last_peered":"2024-09-13T17:57:29.728046+0000","last_clean":"2024-09-13T17:57:29.728046+0000","last_became_active":"2024-09-13T17:57:25.734233+0000","last_became_peered":"2024-09-13T17:57:25.734233+0000","last_unstale":"2024-09-13T17:57:29.728046+0000","last_undegraded":"2024-09-13T17:57:29.728046+0000","last_fullsized":"2024-09-13T17:57:29.728046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725310+0000","last_change":"2024-09-13T17:57:25.718661+0000","last_active":"2024-09-13T17:57:29.725310+0000","last_peered":"2024-09-13T17:57:29.725310+0000","last_clean":"2024-09-13T17:57:29.725310+0000","last_became_active":"2024-09-13T17:57:25.718366+0000","last_became_peered":"2024-09-13T17:57:25.718366+0000","last_unstale":"2024-09-13T17:57:29.725310+0000","last_undegraded":"2024-09-13T17:57:29.725310+0000","last_fullsized":"2024-09-13T17:57:29.725310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:12.960061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732206+0000","last_change":"2024-09-13T17:57:25.734683+0000","last_active":"2024-09-13T17:57:29.732206+0000","last_peered":"2024-09-13T17:57:29.732206+0000","last_clean":"2024-09-13T17:57:29.732206+0000","last_became_active":"2024-09-13T17:57:25.734409+0000","last_became_peered":"2024-09-13T17:57:25.734409+0000","last_unstale":"2024-09-13T17:57:29.732206+0000","last_undegraded":"2024-09-13T17:57:29.732206+0000","last_fullsized":"2024-09-13T17:57:29.732206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:59.252453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725864+0000","last_change":"2024-09-13T17:57:25.729680+0000","last_active":"2024-09-13T17:57:29.725864+0000","last_peered":"2024-09-13T17:57:29.725864+0000","last_clean":"2024-09-13T17:57:29.725864+0000","last_became_active":"2024-09-13T17:57:25.729402+0000","last_became_peered":"2024-09-13T17:57:25.729402+0000","last_unstale":"2024-09-13T17:57:29.725864+0000","last_undegraded":"2024-09-13T17:57:29.725864+0000","last_fullsized":"2024-09-13T17:57:29.725864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:43.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724296+0000","last_change":"2024-09-13T17:57:25.728126+0000","last_active":"2024-09-13T17:57:29.724296+0000","last_peered":"2024-09-13T17:57:29.724296+0000","last_clean":"2024-09-13T17:57:29.724296+0000","last_became_active":"2024-09-13T17:57:25.727793+0000","last_became_peered":"2024-09-13T17:57:25.727793+0000","last_unstale":"2024-09-13T17:57:29.724296+0000","last_undegraded":"2024-09-13T17:57:29.724296+0000","last_fullsized":"2024-09-13T17:57:29.724296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:34.797142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729099+0000","last_change":"2024-09-13T17:57:25.716918+0000","last_active":"2024-09-13T17:57:29.729099+0000","last_peered":"2024-09-13T17:57:29.729099+0000","last_clean":"2024-09-13T17:57:29.729099+0000","last_became_active":"2024-09-13T17:57:25.716550+0000","last_became_peered":"2024-09-13T17:57:25.716550+0000","last_unstale":"2024-09-13T17:57:29.729099+0000","last_undegraded":"2024-09-13T17:57:29.729099+0000","last_fullsized":"2024-09-13T17:57:29.729099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:19.536367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724836+0000","last_change":"2024-09-13T17:57:25.717079+0000","last_active":"2024-09-13T17:57:29.724836+0000","last_peered":"2024-09-13T17:57:29.724836+0000","last_clean":"2024-09-13T17:57:29.724836+0000","last_became_active":"2024-09-13T17:57:25.716782+0000","last_became_peered":"2024-09-13T17:57:25.716782+0000","last_unstale":"2024-09-13T17:57:29.724836+0000","last_undegraded":"2024-09-13T17:57:29.724836+0000","last_fullsized":"2024-09-13T17:57:29.724836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:26:19.049804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720610+0000","last_change":"2024-09-13T17:57:25.731733+0000","last_active":"2024-09-13T17:57:29.720610+0000","last_peered":"2024-09-13T17:57:29.720610+0000","last_clean":"2024-09-13T17:57:29.720610+0000","last_became_active":"2024-09-13T17:57:25.731460+0000","last_became_peered":"2024-09-13T17:57:25.731460+0000","last_unstale":"2024-09-13T17:57:29.720610+0000","last_undegraded":"2024-09-13T17:57:29.720610+0000","last_fullsized":"2024-09-13T17:57:29.720610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:43.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.735275+0000","last_change":"2024-09-13T17:57:25.734897+0000","last_active":"2024-09-13T17:57:29.735275+0000","last_peered":"2024-09-13T17:57:29.735275+0000","last_clean":"2024-09-13T17:57:29.735275+0000","last_became_active":"2024-09-13T17:57:25.734639+0000","last_became_peered":"2024-09-13T17:57:25.734639+0000","last_unstale":"2024-09-13T17:57:29.735275+0000","last_undegraded":"2024-09-13T17:57:29.735275+0000","last_fullsized":"2024-09-13T17:57:29.735275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:29.989683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728450+0000","last_change":"2024-09-13T17:57:25.736997+0000","last_active":"2024-09-13T17:57:29.728450+0000","last_peered":"2024-09-13T17:57:29.728450+0000","last_clean":"2024-09-13T17:57:29.728450+0000","last_became_active":"2024-09-13T17:57:25.736735+0000","last_became_peered":"2024-09-13T17:57:25.736735+0000","last_unstale":"2024-09-13T17:57:29.728450+0000","last_undegraded":"2024-09-13T17:57:29.728450+0000","last_fullsized":"2024-09-13T17:57:29.728450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728451+0000","last_change":"2024-09-13T17:57:25.736821+0000","last_active":"2024-09-13T17:57:29.728451+0000","last_peered":"2024-09-13T17:57:29.728451+0000","last_clean":"2024-09-13T17:57:29.728451+0000","last_became_active":"2024-09-13T17:57:25.736572+0000","last_became_peered":"2024-09-13T17:57:25.736572+0000","last_unstale":"2024-09-13T17:57:29.728451+0000","last_undegraded":"2024-09-13T17:57:29.728451+0000","last_fullsized":"2024-09-13T17:57:29.728451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:13.197645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725470+0000","last_change":"2024-09-13T17:57:25.725989+0000","last_active":"2024-09-13T17:57:29.725470+0000","last_peered":"2024-09-13T17:57:29.725470+0000","last_clean":"2024-09-13T17:57:29.725470+0000","last_became_active":"2024-09-13T17:57:25.725725+0000","last_became_peered":"2024-09-13T17:57:25.725725+0000","last_unstale":"2024-09-13T17:57:29.725470+0000","last_undegraded":"2024-09-13T17:57:29.725470+0000","last_fullsized":"2024-09-13T17:57:29.725470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:50.812435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720423+0000","last_change":"2024-09-13T17:57:25.723786+0000","last_active":"2024-09-13T17:57:29.720423+0000","last_peered":"2024-09-13T17:57:29.720423+0000","last_clean":"2024-09-13T17:57:29.720423+0000","last_became_active":"2024-09-13T17:57:25.723510+0000","last_became_peered":"2024-09-13T17:57:25.723510+0000","last_unstale":"2024-09-13T17:57:29.720423+0000","last_undegraded":"2024-09-13T17:57:29.720423+0000","last_fullsized":"2024-09-13T17:57:29.720423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725741+0000","last_change":"2024-09-13T17:57:25.721799+0000","last_active":"2024-09-13T17:57:29.725741+0000","last_peered":"2024-09-13T17:57:29.725741+0000","last_clean":"2024-09-13T17:57:29.725741+0000","last_became_active":"2024-09-13T17:57:25.721309+0000","last_became_peered":"2024-09-13T17:57:25.721309+0000","last_unstale":"2024-09-13T17:57:29.725741+0000","last_undegraded":"2024-09-13T17:57:29.725741+0000","last_fullsized":"2024-09-13T17:57:29.725741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:27.969805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724305+0000","last_change":"2024-09-13T17:57:25.727329+0000","last_active":"2024-09-13T17:57:29.724305+0000","last_peered":"2024-09-13T17:57:29.724305+0000","last_clean":"2024-09-13T17:57:29.724305+0000","last_became_active":"2024-09-13T17:57:25.727064+0000","last_became_peered":"2024-09-13T17:57:25.727064+0000","last_unstale":"2024-09-13T17:57:29.724305+0000","last_undegraded":"2024-09-13T17:57:29.724305+0000","last_fullsized":"2024-09-13T17:57:29.724305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725134+0000","last_change":"2024-09-13T17:57:25.714493+0000","last_active":"2024-09-13T17:57:29.725134+0000","last_peered":"2024-09-13T17:57:29.725134+0000","last_clean":"2024-09-13T17:57:29.725134+0000","last_became_active":"2024-09-13T17:57:25.714091+0000","last_became_peered":"2024-09-13T17:57:25.714091+0000","last_unstale":"2024-09-13T17:57:29.725134+0000","last_undegraded":"2024-09-13T17:57:29.725134+0000","last_fullsized":"2024-09-13T17:57:29.725134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:51.804478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725105+0000","last_change":"2024-09-13T17:57:25.730275+0000","last_active":"2024-09-13T17:57:29.725105+0000","last_peered":"2024-09-13T17:57:29.725105+0000","last_clean":"2024-09-13T17:57:29.725105+0000","last_became_active":"2024-09-13T17:57:25.729845+0000","last_became_peered":"2024-09-13T17:57:25.729845+0000","last_unstale":"2024-09-13T17:57:29.725105+0000","last_undegraded":"2024-09-13T17:57:29.725105+0000","last_fullsized":"2024-09-13T17:57:29.725105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:11.996031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725148+0000","last_change":"2024-09-13T17:57:25.714547+0000","last_active":"2024-09-13T17:57:29.725148+0000","last_peered":"2024-09-13T17:57:29.725148+0000","last_clean":"2024-09-13T17:57:29.725148+0000","last_became_active":"2024-09-13T17:57:25.714288+0000","last_became_peered":"2024-09-13T17:57:25.714288+0000","last_unstale":"2024-09-13T17:57:29.725148+0000","last_undegraded":"2024-09-13T17:57:29.725148+0000","last_fullsized":"2024-09-13T17:57:29.725148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:37:23.859769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729416+0000","last_change":"2024-09-13T17:57:25.727117+0000","last_active":"2024-09-13T17:57:29.729416+0000","last_peered":"2024-09-13T17:57:29.729416+0000","last_clean":"2024-09-13T17:57:29.729416+0000","last_became_active":"2024-09-13T17:57:25.726864+0000","last_became_peered":"2024-09-13T17:57:25.726864+0000","last_unstale":"2024-09-13T17:57:29.729416+0000","last_undegraded":"2024-09-13T17:57:29.729416+0000","last_fullsized":"2024-09-13T17:57:29.729416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725395+0000","last_change":"2024-09-13T17:57:25.734566+0000","last_active":"2024-09-13T17:57:29.725395+0000","last_peered":"2024-09-13T17:57:29.725395+0000","last_clean":"2024-09-13T17:57:29.725395+0000","last_became_active":"2024-09-13T17:57:25.733963+0000","last_became_peered":"2024-09-13T17:57:25.733963+0000","last_unstale":"2024-09-13T17:57:29.725395+0000","last_undegraded":"2024-09-13T17:57:29.725395+0000","last_fullsized":"2024-09-13T17:57:29.725395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:44.248672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":690730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":99870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":99906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.248}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":99780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":99940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":689767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:57:51.493 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:57:51.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.625+0000 7f3f7430f640 1 Processor -- start 2024-09-13T17:57:51.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.625+0000 7f3f7430f640 1 -- start start 2024-09-13T17:57:51.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.626+0000 7f3f7430f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6c0b4660 0x7f3f6c06b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.626+0000 7f3f7430f640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f6c0d1840 con 0x7f3f6c0d07b0 2024-09-13T17:57:51.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.626+0000 7f3f7430f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f6c0d0c30 con 0x7f3f6c15c6e0 2024-09-13T17:57:51.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.626+0000 7f3f7430f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c0d0f10 con 0x7f3f6c0b4660 2024-09-13T17:57:51.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.626+0000 7f3f718aa640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3f6c0d07b0 0x7f3f6c06ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49842/0 (socket says 172.21.15.111:49842) 2024-09-13T17:57:51.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.626+0000 7f3f718aa640 1 -- 172.21.15.111:0/3847748355 learned_addr learned my addr 172.21.15.111:0/3847748355 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:51.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f728ac640 1 --2- 172.21.15.111:0/3847748355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6c0b4660 0x7f3f6c06b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f710a9640 1 -- 172.21.15.111:0/3847748355 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402475440 0 0) 0x7f3f6c0d1840 con 0x7f3f6c0d07b0 2024-09-13T17:57:51.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f710a9640 1 -- 172.21.15.111:0/3847748355 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f58003520 con 0x7f3f6c0d07b0 2024-09-13T17:57:51.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f710a9640 1 -- 172.21.15.111:0/3847748355 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 275551422 0 0) 0x7f3f58003520 con 0x7f3f6c0d07b0 2024-09-13T17:57:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f710a9640 1 -- 172.21.15.111:0/3847748355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6c0b4660 msgr2=0x7f3f6c06b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f710a9640 1 --2- 172.21.15.111:0/3847748355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6c0b4660 0x7f3f6c06b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f710a9640 1 -- 172.21.15.111:0/3847748355 >> v1:172.21.15.188:6789/0 conn(0x7f3f6c15c6e0 legacy=0x7f3f6c0b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f710a9640 1 -- 172.21.15.111:0/3847748355 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c0d11f0 con 0x7f3f6c0d07b0 2024-09-13T17:57:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f710a9640 1 -- 172.21.15.111:0/3847748355 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f3f5c001670 con 0x7f3f6c0d07b0 2024-09-13T17:57:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f710a9640 1 -- 172.21.15.111:0/3847748355 >> v1:172.21.15.111:6789/0 conn(0x7f3f6c0d07b0 legacy=0x7f3f6c06ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f710a9640 1 --2- 172.21.15.111:0/3847748355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f580039a0 0x7f3f58003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f710a9640 1 --2- 172.21.15.111:0/3847748355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6c0b4660 0x7f3f580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f710a9640 1 --2- 172.21.15.111:0/3847748355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f58004d00 0x7f3f58005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f710a9640 1 -- 172.21.15.111:0/3847748355 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c0d0f10 con 0x7f3f580039a0 2024-09-13T17:57:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f710a9640 1 -- 172.21.15.111:0/3847748355 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5800ba60 con 0x7f3f58004d00 2024-09-13T17:57:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f710a9640 1 -- 172.21.15.111:0/3847748355 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5800bc20 con 0x7f3f6c0b4660 2024-09-13T17:57:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f720ab640 1 --2- 172.21.15.111:0/3847748355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f580039a0 0x7f3f58003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f718aa640 1 --2- 172.21.15.111:0/3847748355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6c0b4660 0x7f3f580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f728ac640 1 --2- 172.21.15.111:0/3847748355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f58004d00 0x7f3f58005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f720ab640 1 -- 172.21.15.111:0/3847748355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6c0b4660 msgr2=0x7f3f580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:51.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f720ab640 1 --2- 172.21.15.111:0/3847748355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6c0b4660 0x7f3f580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f720ab640 1 -- 172.21.15.111:0/3847748355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f58004d00 msgr2=0x7f3f58005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:51.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f720ab640 1 --2- 172.21.15.111:0/3847748355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f58004d00 0x7f3f58005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f720ab640 1 -- 172.21.15.111:0/3847748355 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f58003520 con 0x7f3f580039a0 2024-09-13T17:57:51.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f718aa640 1 --2- 172.21.15.111:0/3847748355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6c0b4660 0x7f3f580047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:51.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.627+0000 7f3f728ac640 1 --2- 172.21.15.111:0/3847748355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f58004d00 0x7f3f58005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:51.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.628+0000 7f3f720ab640 1 --2- 172.21.15.111:0/3847748355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f580039a0 0x7f3f58003d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3f68001910 tx=0x7f3f580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14e9c609ac1ec2b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:51.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.628+0000 7f3f710a9640 1 -- 172.21.15.111:0/3847748355 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6802ac60 con 0x7f3f580039a0 2024-09-13T17:57:51.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.628+0000 7f3f710a9640 1 -- 172.21.15.111:0/3847748355 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6802adc0 con 0x7f3f580039a0 2024-09-13T17:57:51.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.628+0000 7f3f710a9640 1 -- 172.21.15.111:0/3847748355 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68032660 con 0x7f3f580039a0 2024-09-13T17:57:51.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.629+0000 7f3f7430f640 1 -- 172.21.15.111:0/3847748355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f580039a0 msgr2=0x7f3f58003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.629+0000 7f3f7430f640 1 --2- 172.21.15.111:0/3847748355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f580039a0 0x7f3f58003d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3f68001910 tx=0x7f3f580057f0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.629+0000 7f3f728ac640 1 -- 172.21.15.111:0/3847748355 reap_dead start 2024-09-13T17:57:51.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.629+0000 7f3f7430f640 1 -- 172.21.15.111:0/3847748355 shutdown_connections 2024-09-13T17:57:51.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.629+0000 7f3f7430f640 1 -- 172.21.15.111:0/3847748355 >> 172.21.15.111:0/3847748355 conn(0x7f3f6c089f60 msgr2=0x7f3f6c0d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:51.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.629+0000 7f3f7430f640 1 -- 172.21.15.111:0/3847748355 shutdown_connections 2024-09-13T17:57:51.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.629+0000 7f3f7430f640 1 -- 172.21.15.111:0/3847748355 wait complete. 2024-09-13T17:57:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.630+0000 7f3f7430f640 1 Processor -- start 2024-09-13T17:57:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.630+0000 7f3f7430f640 1 -- start start 2024-09-13T17:57:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f7430f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f580039a0 0x7f3f6c0c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f7430f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f6c0cb860 0x7f3f6c09fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f7430f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6c159970 0x7f3f6c0b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f7430f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c0cbde0 con 0x7f3f580039a0 2024-09-13T17:57:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f7430f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c0cc0a0 con 0x7f3f6c0cb860 2024-09-13T17:57:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f7430f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c0cc380 con 0x7f3f6c159970 2024-09-13T17:57:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f720ab640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f580039a0 0x7f3f6c0c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f720ab640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f580039a0 0x7f3f6c0c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54276/0 (socket says 172.21.15.111:54276) 2024-09-13T17:57:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f720ab640 1 -- 172.21.15.111:0/4185726675 learned_addr learned my addr 172.21.15.111:0/4185726675 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f718aa640 1 --2- 172.21.15.111:0/4185726675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f6c0cb860 0x7f3f6c09fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f728ac640 1 --2- 172.21.15.111:0/4185726675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6c159970 0x7f3f6c0b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f720ab640 1 -- 172.21.15.111:0/4185726675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6c159970 msgr2=0x7f3f6c0b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f720ab640 1 --2- 172.21.15.111:0/4185726675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6c159970 0x7f3f6c0b74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f720ab640 1 -- 172.21.15.111:0/4185726675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f6c0cb860 msgr2=0x7f3f6c09fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f720ab640 1 --2- 172.21.15.111:0/4185726675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f6c0cb860 0x7f3f6c09fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f720ab640 1 -- 172.21.15.111:0/4185726675 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c0cc660 con 0x7f3f580039a0 2024-09-13T17:57:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f720ab640 1 --2- 172.21.15.111:0/4185726675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f580039a0 0x7f3f6c0c95a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3f68035040 tx=0x7f3f680012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.631+0000 7f3f62ffd640 1 -- 172.21.15.111:0/4185726675 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68007040 con 0x7f3f580039a0 2024-09-13T17:57:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.632+0000 7f3f62ffd640 1 -- 172.21.15.111:0/4185726675 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f680071a0 con 0x7f3f580039a0 2024-09-13T17:57:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.632+0000 7f3f62ffd640 1 -- 172.21.15.111:0/4185726675 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68007510 con 0x7f3f580039a0 2024-09-13T17:57:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.632+0000 7f3f7430f640 1 -- 172.21.15.111:0/4185726675 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6c09ffb0 con 0x7f3f580039a0 2024-09-13T17:57:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.632+0000 7f3f7430f640 1 -- 172.21.15.111:0/4185726675 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6c159fd0 con 0x7f3f580039a0 2024-09-13T17:57:51.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.632+0000 7f3f7430f640 1 -- 172.21.15.111:0/4185726675 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f6c072b20 con 0x7f3f580039a0 2024-09-13T17:57:51.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.633+0000 7f3f62ffd640 1 -- 172.21.15.111:0/4185726675 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3f68039040 con 0x7f3f580039a0 2024-09-13T17:57:51.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.633+0000 7f3f62ffd640 1 --2- 172.21.15.111:0/4185726675 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f3f5006ac00 0x7f3f5006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.633+0000 7f3f62ffd640 1 -- 172.21.15.111:0/4185726675 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3f680a9660 con 0x7f3f580039a0 2024-09-13T17:57:51.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.633+0000 7f3f718aa640 1 --2- 172.21.15.111:0/4185726675 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f3f5006ac00 0x7f3f5006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.634+0000 7f3f718aa640 1 --2- 172.21.15.111:0/4185726675 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f3f5006ac00 0x7f3f5006d0c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c001c30 tx=0x7f3f5c000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:51.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.637+0000 7f3f62ffd640 1 -- 172.21.15.111:0/4185726675 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f3f68074690 con 0x7f3f580039a0 2024-09-13T17:57:51.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.796+0000 7f3f7430f640 1 -- 172.21.15.111:0/4185726675 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3f6c0ce3b0 con 0x7f3f5006ac00 2024-09-13T17:57:51.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.804+0000 7f3f62ffd640 1 -- 172.21.15.111:0/4185726675 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345188 (secure 0 0 0) 0x7f3f6c0ce3b0 con 0x7f3f5006ac00 2024-09-13T17:57:51.810 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:51.812 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-09-13T17:57:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.807+0000 7f3f7430f640 1 -- 172.21.15.111:0/4185726675 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f3f5006ac00 msgr2=0x7f3f5006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.807+0000 7f3f7430f640 1 --2- 172.21.15.111:0/4185726675 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f3f5006ac00 0x7f3f5006d0c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c001c30 tx=0x7f3f5c000ed0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.807+0000 7f3f7430f640 1 -- 172.21.15.111:0/4185726675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f580039a0 msgr2=0x7f3f6c0c95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.807+0000 7f3f7430f640 1 --2- 172.21.15.111:0/4185726675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f580039a0 0x7f3f6c0c95a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3f68035040 tx=0x7f3f680012b0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.807+0000 7f3f7430f640 1 -- 172.21.15.111:0/4185726675 shutdown_connections 2024-09-13T17:57:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.807+0000 7f3f7430f640 1 --2- 172.21.15.111:0/4185726675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6c159970 0x7f3f6c0b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.807+0000 7f3f7430f640 1 --2- 172.21.15.111:0/4185726675 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f3f5006ac00 0x7f3f5006d0c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.807+0000 7f3f7430f640 1 --2- 172.21.15.111:0/4185726675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f6c0cb860 0x7f3f6c09fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.807+0000 7f3f7430f640 1 --2- 172.21.15.111:0/4185726675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f580039a0 0x7f3f6c0c95a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.807+0000 7f3f7430f640 1 -- 172.21.15.111:0/4185726675 >> 172.21.15.111:0/4185726675 conn(0x7f3f6c089f60 msgr2=0x7f3f6c15ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.807+0000 7f3f7430f640 1 -- 172.21.15.111:0/4185726675 shutdown_connections 2024-09-13T17:57:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.807+0000 7f3f7430f640 1 -- 172.21.15.111:0/4185726675 wait complete. 2024-09-13T17:57:51.821 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-13T17:57:50.271362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3204,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3280896,"data_stored":1987143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002630"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727037+0000","last_change":"2024-09-13T17:57:25.378960+0000","last_active":"2024-09-13T17:57:29.727037+0000","last_peered":"2024-09-13T17:57:29.727037+0000","last_clean":"2024-09-13T17:57:29.727037+0000","last_became_active":"2024-09-13T17:57:25.378681+0000","last_became_peered":"2024-09-13T17:57:25.378681+0000","last_unstale":"2024-09-13T17:57:29.727037+0000","last_undegraded":"2024-09-13T17:57:29.727037+0000","last_fullsized":"2024-09-13T17:57:29.727037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721168+0000","last_change":"2024-09-13T17:57:25.734706+0000","last_active":"2024-09-13T17:57:29.721168+0000","last_peered":"2024-09-13T17:57:29.721168+0000","last_clean":"2024-09-13T17:57:29.721168+0000","last_became_active":"2024-09-13T17:57:25.734432+0000","last_became_peered":"2024-09-13T17:57:25.734432+0000","last_unstale":"2024-09-13T17:57:29.721168+0000","last_undegraded":"2024-09-13T17:57:29.721168+0000","last_fullsized":"2024-09-13T17:57:29.721168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:12.960061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.730686+0000","last_change":"2024-09-13T17:57:29.729380+0000","last_active":"2024-09-13T17:57:29.730686+0000","last_peered":"2024-09-13T17:57:29.730686+0000","last_clean":"2024-09-13T17:57:29.730686+0000","last_became_active":"2024-09-13T17:57:29.729000+0000","last_became_peered":"2024-09-13T17:57:29.729000+0000","last_unstale":"2024-09-13T17:57:29.730686+0000","last_undegraded":"2024-09-13T17:57:29.730686+0000","last_fullsized":"2024-09-13T17:57:29.730686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:31.285328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721336+0000","last_change":"2024-09-13T17:57:25.382416+0000","last_active":"2024-09-13T17:57:29.721336+0000","last_peered":"2024-09-13T17:57:29.721336+0000","last_clean":"2024-09-13T17:57:29.721336+0000","last_became_active":"2024-09-13T17:57:25.382154+0000","last_became_peered":"2024-09-13T17:57:25.382154+0000","last_unstale":"2024-09-13T17:57:29.721336+0000","last_undegraded":"2024-09-13T17:57:29.721336+0000","last_fullsized":"2024-09-13T17:57:29.721336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725491+0000","last_change":"2024-09-13T17:57:25.731759+0000","last_active":"2024-09-13T17:57:29.725491+0000","last_peered":"2024-09-13T17:57:29.725491+0000","last_clean":"2024-09-13T17:57:29.725491+0000","last_became_active":"2024-09-13T17:57:25.731494+0000","last_became_peered":"2024-09-13T17:57:25.731494+0000","last_unstale":"2024-09-13T17:57:29.725491+0000","last_undegraded":"2024-09-13T17:57:29.725491+0000","last_fullsized":"2024-09-13T17:57:29.725491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.751270+0000","last_change":"2024-09-13T17:57:29.750278+0000","last_active":"2024-09-13T17:57:29.751270+0000","last_peered":"2024-09-13T17:57:29.751270+0000","last_clean":"2024-09-13T17:57:29.751270+0000","last_became_active":"2024-09-13T17:57:29.749874+0000","last_became_peered":"2024-09-13T17:57:29.749874+0000","last_unstale":"2024-09-13T17:57:29.751270+0000","last_undegraded":"2024-09-13T17:57:29.751270+0000","last_fullsized":"2024-09-13T17:57:29.751270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:03.749305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725223+0000","last_change":"2024-09-13T17:57:25.377660+0000","last_active":"2024-09-13T17:57:29.725223+0000","last_peered":"2024-09-13T17:57:29.725223+0000","last_clean":"2024-09-13T17:57:29.725223+0000","last_became_active":"2024-09-13T17:57:25.377386+0000","last_became_peered":"2024-09-13T17:57:25.377386+0000","last_unstale":"2024-09-13T17:57:29.725223+0000","last_undegraded":"2024-09-13T17:57:29.725223+0000","last_fullsized":"2024-09-13T17:57:29.725223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:38.038738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725142+0000","last_change":"2024-09-13T17:57:25.728949+0000","last_active":"2024-09-13T17:57:29.725142+0000","last_peered":"2024-09-13T17:57:29.725142+0000","last_clean":"2024-09-13T17:57:29.725142+0000","last_became_active":"2024-09-13T17:57:25.728675+0000","last_became_peered":"2024-09-13T17:57:25.728675+0000","last_unstale":"2024-09-13T17:57:29.725142+0000","last_undegraded":"2024-09-13T17:57:29.725142+0000","last_fullsized":"2024-09-13T17:57:29.725142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749006+0000","last_change":"2024-09-13T17:57:29.745555+0000","last_active":"2024-09-13T17:57:29.749006+0000","last_peered":"2024-09-13T17:57:29.749006+0000","last_clean":"2024-09-13T17:57:29.749006+0000","last_became_active":"2024-09-13T17:57:29.745172+0000","last_became_peered":"2024-09-13T17:57:29.745172+0000","last_unstale":"2024-09-13T17:57:29.749006+0000","last_undegraded":"2024-09-13T17:57:29.749006+0000","last_fullsized":"2024-09-13T17:57:29.749006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:37.101711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725546+0000","last_change":"2024-09-13T17:57:25.375351+0000","last_active":"2024-09-13T17:57:29.725546+0000","last_peered":"2024-09-13T17:57:29.725546+0000","last_clean":"2024-09-13T17:57:29.725546+0000","last_became_active":"2024-09-13T17:57:25.374550+0000","last_became_peered":"2024-09-13T17:57:25.374550+0000","last_unstale":"2024-09-13T17:57:29.725546+0000","last_undegraded":"2024-09-13T17:57:29.725546+0000","last_fullsized":"2024-09-13T17:57:29.725546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:11:57.892225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725579+0000","last_change":"2024-09-13T17:57:25.729527+0000","last_active":"2024-09-13T17:57:29.725579+0000","last_peered":"2024-09-13T17:57:29.725579+0000","last_clean":"2024-09-13T17:57:29.725579+0000","last_became_active":"2024-09-13T17:57:25.729225+0000","last_became_peered":"2024-09-13T17:57:25.729225+0000","last_unstale":"2024-09-13T17:57:29.725579+0000","last_undegraded":"2024-09-13T17:57:29.725579+0000","last_fullsized":"2024-09-13T17:57:29.725579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:27.969805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.748980+0000","last_change":"2024-09-13T17:57:29.745532+0000","last_active":"2024-09-13T17:57:29.748980+0000","last_peered":"2024-09-13T17:57:29.748980+0000","last_clean":"2024-09-13T17:57:29.748980+0000","last_became_active":"2024-09-13T17:57:29.745130+0000","last_became_peered":"2024-09-13T17:57:29.745130+0000","last_unstale":"2024-09-13T17:57:29.748980+0000","last_undegraded":"2024-09-13T17:57:29.748980+0000","last_fullsized":"2024-09-13T17:57:29.748980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:54.172556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728175+0000","last_change":"2024-09-13T17:57:25.735067+0000","last_active":"2024-09-13T17:57:29.728175+0000","last_peered":"2024-09-13T17:57:29.728175+0000","last_clean":"2024-09-13T17:57:29.728175+0000","last_became_active":"2024-09-13T17:57:25.734807+0000","last_became_peered":"2024-09-13T17:57:25.734807+0000","last_unstale":"2024-09-13T17:57:29.728175+0000","last_undegraded":"2024-09-13T17:57:29.728175+0000","last_fullsized":"2024-09-13T17:57:29.728175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:19.536367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725323+0000","last_change":"2024-09-13T17:57:25.374934+0000","last_active":"2024-09-13T17:57:29.725323+0000","last_peered":"2024-09-13T17:57:29.725323+0000","last_clean":"2024-09-13T17:57:29.725323+0000","last_became_active":"2024-09-13T17:57:25.374307+0000","last_became_peered":"2024-09-13T17:57:25.374307+0000","last_unstale":"2024-09-13T17:57:29.725323+0000","last_undegraded":"2024-09-13T17:57:29.725323+0000","last_fullsized":"2024-09-13T17:57:29.725323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:15.663292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.748917+0000","last_change":"2024-09-13T17:57:29.743757+0000","last_active":"2024-09-13T17:57:29.748917+0000","last_peered":"2024-09-13T17:57:29.748917+0000","last_clean":"2024-09-13T17:57:29.748917+0000","last_became_active":"2024-09-13T17:57:29.743362+0000","last_became_peered":"2024-09-13T17:57:29.743362+0000","last_unstale":"2024-09-13T17:57:29.748917+0000","last_undegraded":"2024-09-13T17:57:29.748917+0000","last_fullsized":"2024-09-13T17:57:29.748917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:31.285328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720769+0000","last_change":"2024-09-13T17:57:25.720127+0000","last_active":"2024-09-13T17:57:29.720769+0000","last_peered":"2024-09-13T17:57:29.720769+0000","last_clean":"2024-09-13T17:57:29.720769+0000","last_became_active":"2024-09-13T17:57:25.719766+0000","last_became_peered":"2024-09-13T17:57:25.719766+0000","last_unstale":"2024-09-13T17:57:29.720769+0000","last_undegraded":"2024-09-13T17:57:29.720769+0000","last_fullsized":"2024-09-13T17:57:29.720769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.730752+0000","last_change":"2024-09-13T17:57:25.381085+0000","last_active":"2024-09-13T17:57:29.730752+0000","last_peered":"2024-09-13T17:57:29.730752+0000","last_clean":"2024-09-13T17:57:29.730752+0000","last_became_active":"2024-09-13T17:57:25.380825+0000","last_became_peered":"2024-09-13T17:57:25.380825+0000","last_unstale":"2024-09-13T17:57:29.730752+0000","last_undegraded":"2024-09-13T17:57:29.730752+0000","last_fullsized":"2024-09-13T17:57:29.730752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:15.663292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749031+0000","last_change":"2024-09-13T17:57:29.746426+0000","last_active":"2024-09-13T17:57:29.749031+0000","last_peered":"2024-09-13T17:57:29.749031+0000","last_clean":"2024-09-13T17:57:29.749031+0000","last_became_active":"2024-09-13T17:57:29.745582+0000","last_became_peered":"2024-09-13T17:57:29.745582+0000","last_unstale":"2024-09-13T17:57:29.749031+0000","last_undegraded":"2024-09-13T17:57:29.749031+0000","last_fullsized":"2024-09-13T17:57:29.749031+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:53.991862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721690+0000","last_change":"2024-09-13T17:57:25.724108+0000","last_active":"2024-09-13T17:57:29.721690+0000","last_peered":"2024-09-13T17:57:29.721690+0000","last_clean":"2024-09-13T17:57:29.721690+0000","last_became_active":"2024-09-13T17:57:25.723857+0000","last_became_peered":"2024-09-13T17:57:25.723857+0000","last_unstale":"2024-09-13T17:57:29.721690+0000","last_undegraded":"2024-09-13T17:57:29.721690+0000","last_fullsized":"2024-09-13T17:57:29.721690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:19.536367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729185+0000","last_change":"2024-09-13T17:57:25.374954+0000","last_active":"2024-09-13T17:57:29.729185+0000","last_peered":"2024-09-13T17:57:29.729185+0000","last_clean":"2024-09-13T17:57:29.729185+0000","last_became_active":"2024-09-13T17:57:25.374721+0000","last_became_peered":"2024-09-13T17:57:25.374721+0000","last_unstale":"2024-09-13T17:57:29.729185+0000","last_undegraded":"2024-09-13T17:57:29.729185+0000","last_fullsized":"2024-09-13T17:57:29.729185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:34.109316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749541+0000","last_change":"2024-09-13T17:57:29.744520+0000","last_active":"2024-09-13T17:57:29.749541+0000","last_peered":"2024-09-13T17:57:29.749541+0000","last_clean":"2024-09-13T17:57:29.749541+0000","last_became_active":"2024-09-13T17:57:29.744143+0000","last_became_peered":"2024-09-13T17:57:29.744143+0000","last_unstale":"2024-09-13T17:57:29.749541+0000","last_undegraded":"2024-09-13T17:57:29.749541+0000","last_fullsized":"2024-09-13T17:57:29.749541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:18.526568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.733595+0000","last_change":"2024-09-13T17:57:25.718903+0000","last_active":"2024-09-13T17:57:29.733595+0000","last_peered":"2024-09-13T17:57:29.733595+0000","last_clean":"2024-09-13T17:57:29.733595+0000","last_became_active":"2024-09-13T17:57:25.718483+0000","last_became_peered":"2024-09-13T17:57:25.718483+0000","last_unstale":"2024-09-13T17:57:29.733595+0000","last_undegraded":"2024-09-13T17:57:29.733595+0000","last_fullsized":"2024-09-13T17:57:29.733595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:59.252453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725956+0000","last_change":"2024-09-13T17:57:25.378628+0000","last_active":"2024-09-13T17:57:29.725956+0000","last_peered":"2024-09-13T17:57:29.725956+0000","last_clean":"2024-09-13T17:57:29.725956+0000","last_became_active":"2024-09-13T17:57:25.378359+0000","last_became_peered":"2024-09-13T17:57:25.378359+0000","last_unstale":"2024-09-13T17:57:29.725956+0000","last_undegraded":"2024-09-13T17:57:29.725956+0000","last_fullsized":"2024-09-13T17:57:29.725956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:00.725292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749048+0000","last_change":"2024-09-13T17:57:29.746604+0000","last_active":"2024-09-13T17:57:29.749048+0000","last_peered":"2024-09-13T17:57:29.749048+0000","last_clean":"2024-09-13T17:57:29.749048+0000","last_became_active":"2024-09-13T17:57:29.746216+0000","last_became_peered":"2024-09-13T17:57:29.746216+0000","last_unstale":"2024-09-13T17:57:29.749048+0000","last_undegraded":"2024-09-13T17:57:29.749048+0000","last_fullsized":"2024-09-13T17:57:29.749048+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:18.526568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725404+0000","last_change":"2024-09-13T17:57:25.722540+0000","last_active":"2024-09-13T17:57:29.725404+0000","last_peered":"2024-09-13T17:57:29.725404+0000","last_clean":"2024-09-13T17:57:29.725404+0000","last_became_active":"2024-09-13T17:57:25.722301+0000","last_became_peered":"2024-09-13T17:57:25.722301+0000","last_unstale":"2024-09-13T17:57:29.725404+0000","last_undegraded":"2024-09-13T17:57:29.725404+0000","last_fullsized":"2024-09-13T17:57:29.725404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:34.117674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728850+0000","last_change":"2024-09-13T17:57:25.380087+0000","last_active":"2024-09-13T17:57:29.728850+0000","last_peered":"2024-09-13T17:57:29.728850+0000","last_clean":"2024-09-13T17:57:29.728850+0000","last_became_active":"2024-09-13T17:57:25.379808+0000","last_became_peered":"2024-09-13T17:57:25.379808+0000","last_unstale":"2024-09-13T17:57:29.728850+0000","last_undegraded":"2024-09-13T17:57:29.728850+0000","last_fullsized":"2024-09-13T17:57:29.728850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:38.038738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.743674+0000","last_change":"2024-09-13T17:57:29.740242+0000","last_active":"2024-09-13T17:57:29.743674+0000","last_peered":"2024-09-13T17:57:29.743674+0000","last_clean":"2024-09-13T17:57:29.743674+0000","last_became_active":"2024-09-13T17:57:29.739815+0000","last_became_peered":"2024-09-13T17:57:29.739815+0000","last_unstale":"2024-09-13T17:57:29.743674+0000","last_undegraded":"2024-09-13T17:57:29.743674+0000","last_fullsized":"2024-09-13T17:57:29.743674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:45.306949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720934+0000","last_change":"2024-09-13T17:57:25.718065+0000","last_active":"2024-09-13T17:57:29.720934+0000","last_peered":"2024-09-13T17:57:29.720934+0000","last_clean":"2024-09-13T17:57:29.720934+0000","last_became_active":"2024-09-13T17:57:25.717759+0000","last_became_peered":"2024-09-13T17:57:25.717759+0000","last_unstale":"2024-09-13T17:57:29.720934+0000","last_undegraded":"2024-09-13T17:57:29.720934+0000","last_fullsized":"2024-09-13T17:57:29.720934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.814140+0000","last_change":"2024-09-13T17:57:25.377272+0000","last_active":"2024-09-13T17:57:29.814140+0000","last_peered":"2024-09-13T17:57:29.814140+0000","last_clean":"2024-09-13T17:57:29.814140+0000","last_became_active":"2024-09-13T17:57:25.376938+0000","last_became_peered":"2024-09-13T17:57:25.376938+0000","last_unstale":"2024-09-13T17:57:29.814140+0000","last_undegraded":"2024-09-13T17:57:29.814140+0000","last_fullsized":"2024-09-13T17:57:29.814140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:00.725292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.746741+0000","last_change":"2024-09-13T17:57:29.743363+0000","last_active":"2024-09-13T17:57:29.746741+0000","last_peered":"2024-09-13T17:57:29.746741+0000","last_clean":"2024-09-13T17:57:29.746741+0000","last_became_active":"2024-09-13T17:57:29.738263+0000","last_became_peered":"2024-09-13T17:57:29.738263+0000","last_unstale":"2024-09-13T17:57:29.746741+0000","last_undegraded":"2024-09-13T17:57:29.746741+0000","last_fullsized":"2024-09-13T17:57:29.746741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:42:39.016490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729043+0000","last_change":"2024-09-13T17:57:25.717635+0000","last_active":"2024-09-13T17:57:29.729043+0000","last_peered":"2024-09-13T17:57:29.729043+0000","last_clean":"2024-09-13T17:57:29.729043+0000","last_became_active":"2024-09-13T17:57:25.717360+0000","last_became_peered":"2024-09-13T17:57:25.717360+0000","last_unstale":"2024-09-13T17:57:29.729043+0000","last_undegraded":"2024-09-13T17:57:29.729043+0000","last_fullsized":"2024-09-13T17:57:29.729043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725395+0000","last_change":"2024-09-13T17:57:25.378289+0000","last_active":"2024-09-13T17:57:29.725395+0000","last_peered":"2024-09-13T17:57:29.725395+0000","last_clean":"2024-09-13T17:57:29.725395+0000","last_became_active":"2024-09-13T17:57:25.378038+0000","last_became_peered":"2024-09-13T17:57:25.378038+0000","last_unstale":"2024-09-13T17:57:29.725395+0000","last_undegraded":"2024-09-13T17:57:29.725395+0000","last_fullsized":"2024-09-13T17:57:29.725395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:58.248434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.746870+0000","last_change":"2024-09-13T17:57:29.741347+0000","last_active":"2024-09-13T17:57:29.746870+0000","last_peered":"2024-09-13T17:57:29.746870+0000","last_clean":"2024-09-13T17:57:29.746870+0000","last_became_active":"2024-09-13T17:57:29.740980+0000","last_became_peered":"2024-09-13T17:57:29.740980+0000","last_unstale":"2024-09-13T17:57:29.746870+0000","last_undegraded":"2024-09-13T17:57:29.746870+0000","last_fullsized":"2024-09-13T17:57:29.746870+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:53.991862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725663+0000","last_change":"2024-09-13T17:57:25.728135+0000","last_active":"2024-09-13T17:57:29.725663+0000","last_peered":"2024-09-13T17:57:29.725663+0000","last_clean":"2024-09-13T17:57:29.725663+0000","last_became_active":"2024-09-13T17:57:25.727903+0000","last_became_peered":"2024-09-13T17:57:25.727903+0000","last_unstale":"2024-09-13T17:57:29.725663+0000","last_undegraded":"2024-09-13T17:57:29.725663+0000","last_fullsized":"2024-09-13T17:57:29.725663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:48.163930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729130+0000","last_change":"2024-09-13T17:57:25.375187+0000","last_active":"2024-09-13T17:57:29.729130+0000","last_peered":"2024-09-13T17:57:29.729130+0000","last_clean":"2024-09-13T17:57:29.729130+0000","last_became_active":"2024-09-13T17:57:25.374295+0000","last_became_peered":"2024-09-13T17:57:25.374295+0000","last_unstale":"2024-09-13T17:57:29.729130+0000","last_undegraded":"2024-09-13T17:57:29.729130+0000","last_fullsized":"2024-09-13T17:57:29.729130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:58.248434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742560+0000","last_change":"2024-09-13T17:57:29.740383+0000","last_active":"2024-09-13T17:57:29.742560+0000","last_peered":"2024-09-13T17:57:29.742560+0000","last_clean":"2024-09-13T17:57:29.742560+0000","last_became_active":"2024-09-13T17:57:29.739542+0000","last_became_peered":"2024-09-13T17:57:29.739542+0000","last_unstale":"2024-09-13T17:57:29.742560+0000","last_undegraded":"2024-09-13T17:57:29.742560+0000","last_fullsized":"2024-09-13T17:57:29.742560+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:13.633882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.734117+0000","last_change":"2024-09-13T17:57:25.717973+0000","last_active":"2024-09-13T17:57:29.734117+0000","last_peered":"2024-09-13T17:57:29.734117+0000","last_clean":"2024-09-13T17:57:29.734117+0000","last_became_active":"2024-09-13T17:57:25.717702+0000","last_became_peered":"2024-09-13T17:57:25.717702+0000","last_unstale":"2024-09-13T17:57:29.734117+0000","last_undegraded":"2024-09-13T17:57:29.734117+0000","last_fullsized":"2024-09-13T17:57:29.734117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:35.113334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725035+0000","last_change":"2024-09-13T17:57:25.373814+0000","last_active":"2024-09-13T17:57:29.725035+0000","last_peered":"2024-09-13T17:57:29.725035+0000","last_clean":"2024-09-13T17:57:29.725035+0000","last_became_active":"2024-09-13T17:57:25.373556+0000","last_became_peered":"2024-09-13T17:57:25.373556+0000","last_unstale":"2024-09-13T17:57:29.725035+0000","last_undegraded":"2024-09-13T17:57:29.725035+0000","last_fullsized":"2024-09-13T17:57:29.725035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:04.441662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742509+0000","last_change":"2024-09-13T17:57:29.739908+0000","last_active":"2024-09-13T17:57:29.742509+0000","last_peered":"2024-09-13T17:57:29.742509+0000","last_clean":"2024-09-13T17:57:29.742509+0000","last_became_active":"2024-09-13T17:57:29.739070+0000","last_became_peered":"2024-09-13T17:57:29.739070+0000","last_unstale":"2024-09-13T17:57:29.742509+0000","last_undegraded":"2024-09-13T17:57:29.742509+0000","last_fullsized":"2024-09-13T17:57:29.742509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:37.101711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729849+0000","last_change":"2024-09-13T17:57:25.718810+0000","last_active":"2024-09-13T17:57:29.729849+0000","last_peered":"2024-09-13T17:57:29.729849+0000","last_clean":"2024-09-13T17:57:29.729849+0000","last_became_active":"2024-09-13T17:57:25.718576+0000","last_became_peered":"2024-09-13T17:57:25.718576+0000","last_unstale":"2024-09-13T17:57:29.729849+0000","last_undegraded":"2024-09-13T17:57:29.729849+0000","last_fullsized":"2024-09-13T17:57:29.729849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:13.197645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732373+0000","last_change":"2024-09-13T17:57:25.380336+0000","last_active":"2024-09-13T17:57:29.732373+0000","last_peered":"2024-09-13T17:57:29.732373+0000","last_clean":"2024-09-13T17:57:29.732373+0000","last_became_active":"2024-09-13T17:57:25.380044+0000","last_became_peered":"2024-09-13T17:57:25.380044+0000","last_unstale":"2024-09-13T17:57:29.732373+0000","last_undegraded":"2024-09-13T17:57:29.732373+0000","last_fullsized":"2024-09-13T17:57:29.732373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:41.189680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.746550+0000","last_change":"2024-09-13T17:57:29.741637+0000","last_active":"2024-09-13T17:57:29.746550+0000","last_peered":"2024-09-13T17:57:29.746550+0000","last_clean":"2024-09-13T17:57:29.746550+0000","last_became_active":"2024-09-13T17:57:29.741260+0000","last_became_peered":"2024-09-13T17:57:29.741260+0000","last_unstale":"2024-09-13T17:57:29.746550+0000","last_undegraded":"2024-09-13T17:57:29.746550+0000","last_fullsized":"2024-09-13T17:57:29.746550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:53.991862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724599+0000","last_change":"2024-09-13T17:57:25.718276+0000","last_active":"2024-09-13T17:57:29.724599+0000","last_peered":"2024-09-13T17:57:29.724599+0000","last_clean":"2024-09-13T17:57:29.724599+0000","last_became_active":"2024-09-13T17:57:25.718000+0000","last_became_peered":"2024-09-13T17:57:25.718000+0000","last_unstale":"2024-09-13T17:57:29.724599+0000","last_undegraded":"2024-09-13T17:57:29.724599+0000","last_fullsized":"2024-09-13T17:57:29.724599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:05.710729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724849+0000","last_change":"2024-09-13T17:57:25.375143+0000","last_active":"2024-09-13T17:57:29.724849+0000","last_peered":"2024-09-13T17:57:29.724849+0000","last_clean":"2024-09-13T17:57:29.724849+0000","last_became_active":"2024-09-13T17:57:25.374871+0000","last_became_peered":"2024-09-13T17:57:25.374871+0000","last_unstale":"2024-09-13T17:57:29.724849+0000","last_undegraded":"2024-09-13T17:57:29.724849+0000","last_fullsized":"2024-09-13T17:57:29.724849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:34.109316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.738027+0000","last_change":"2024-09-13T17:57:29.733428+0000","last_active":"2024-09-13T17:57:29.738027+0000","last_peered":"2024-09-13T17:57:29.738027+0000","last_clean":"2024-09-13T17:57:29.738027+0000","last_became_active":"2024-09-13T17:57:29.732952+0000","last_became_peered":"2024-09-13T17:57:29.732952+0000","last_unstale":"2024-09-13T17:57:29.738027+0000","last_undegraded":"2024-09-13T17:57:29.738027+0000","last_fullsized":"2024-09-13T17:57:29.738027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:56:53.973670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720518+0000","last_change":"2024-09-13T17:57:25.732656+0000","last_active":"2024-09-13T17:57:29.720518+0000","last_peered":"2024-09-13T17:57:29.720518+0000","last_clean":"2024-09-13T17:57:29.720518+0000","last_became_active":"2024-09-13T17:57:25.732308+0000","last_became_peered":"2024-09-13T17:57:25.732308+0000","last_unstale":"2024-09-13T17:57:29.720518+0000","last_undegraded":"2024-09-13T17:57:29.720518+0000","last_fullsized":"2024-09-13T17:57:29.720518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:34.797142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724840+0000","last_change":"2024-09-13T17:57:25.374372+0000","last_active":"2024-09-13T17:57:29.724840+0000","last_peered":"2024-09-13T17:57:29.724840+0000","last_clean":"2024-09-13T17:57:29.724840+0000","last_became_active":"2024-09-13T17:57:25.374110+0000","last_became_peered":"2024-09-13T17:57:25.374110+0000","last_unstale":"2024-09-13T17:57:29.724840+0000","last_undegraded":"2024-09-13T17:57:29.724840+0000","last_fullsized":"2024-09-13T17:57:29.724840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:38.038738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724941+0000","last_change":"2024-09-13T17:57:15.560506+0000","last_active":"2024-09-13T17:57:29.724941+0000","last_peered":"2024-09-13T17:57:29.724941+0000","last_clean":"2024-09-13T17:57:29.724941+0000","last_became_active":"2024-09-13T17:57:15.559612+0000","last_became_peered":"2024-09-13T17:57:15.559612+0000","last_unstale":"2024-09-13T17:57:29.724941+0000","last_undegraded":"2024-09-13T17:57:29.724941+0000","last_fullsized":"2024-09-13T17:57:29.724941+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_clean_scrub_stamp":"2024-09-13T17:57:14.317378+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:16:27.663312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.743992+0000","last_change":"2024-09-13T17:57:29.739891+0000","last_active":"2024-09-13T17:57:29.743992+0000","last_peered":"2024-09-13T17:57:29.743992+0000","last_clean":"2024-09-13T17:57:29.743992+0000","last_became_active":"2024-09-13T17:57:29.739331+0000","last_became_peered":"2024-09-13T17:57:29.739331+0000","last_unstale":"2024-09-13T17:57:29.743992+0000","last_undegraded":"2024-09-13T17:57:29.743992+0000","last_fullsized":"2024-09-13T17:57:29.743992+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:53:45.306949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725127+0000","last_change":"2024-09-13T17:57:25.725987+0000","last_active":"2024-09-13T17:57:29.725127+0000","last_peered":"2024-09-13T17:57:29.725127+0000","last_clean":"2024-09-13T17:57:29.725127+0000","last_became_active":"2024-09-13T17:57:25.725695+0000","last_became_peered":"2024-09-13T17:57:25.725695+0000","last_unstale":"2024-09-13T17:57:29.725127+0000","last_undegraded":"2024-09-13T17:57:29.725127+0000","last_fullsized":"2024-09-13T17:57:29.725127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:48.163930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732467+0000","last_change":"2024-09-13T17:57:25.376170+0000","last_active":"2024-09-13T17:57:29.732467+0000","last_peered":"2024-09-13T17:57:29.732467+0000","last_clean":"2024-09-13T17:57:29.732467+0000","last_became_active":"2024-09-13T17:57:25.375798+0000","last_became_peered":"2024-09-13T17:57:25.375798+0000","last_unstale":"2024-09-13T17:57:29.732467+0000","last_undegraded":"2024-09-13T17:57:29.732467+0000","last_fullsized":"2024-09-13T17:57:29.732467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:08.374958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744268+0000","last_change":"2024-09-13T17:57:29.741819+0000","last_active":"2024-09-13T17:57:29.744268+0000","last_peered":"2024-09-13T17:57:29.744268+0000","last_clean":"2024-09-13T17:57:29.744268+0000","last_became_active":"2024-09-13T17:57:29.741410+0000","last_became_peered":"2024-09-13T17:57:29.741410+0000","last_unstale":"2024-09-13T17:57:29.744268+0000","last_undegraded":"2024-09-13T17:57:29.744268+0000","last_fullsized":"2024-09-13T17:57:29.744268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:15.299874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725178+0000","last_change":"2024-09-13T17:57:25.729870+0000","last_active":"2024-09-13T17:57:29.725178+0000","last_peered":"2024-09-13T17:57:29.725178+0000","last_clean":"2024-09-13T17:57:29.725178+0000","last_became_active":"2024-09-13T17:57:25.728383+0000","last_became_peered":"2024-09-13T17:57:25.728383+0000","last_unstale":"2024-09-13T17:57:29.725178+0000","last_undegraded":"2024-09-13T17:57:29.725178+0000","last_fullsized":"2024-09-13T17:57:29.725178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:51.804478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.730112+0000","last_change":"2024-09-13T17:57:25.373843+0000","last_active":"2024-09-13T17:57:29.730112+0000","last_peered":"2024-09-13T17:57:29.730112+0000","last_clean":"2024-09-13T17:57:29.730112+0000","last_became_active":"2024-09-13T17:57:25.373498+0000","last_became_peered":"2024-09-13T17:57:25.373498+0000","last_unstale":"2024-09-13T17:57:29.730112+0000","last_undegraded":"2024-09-13T17:57:29.730112+0000","last_fullsized":"2024-09-13T17:57:29.730112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:11:57.892225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742650+0000","last_change":"2024-09-13T17:57:29.735791+0000","last_active":"2024-09-13T17:57:29.742650+0000","last_peered":"2024-09-13T17:57:29.742650+0000","last_clean":"2024-09-13T17:57:29.742650+0000","last_became_active":"2024-09-13T17:57:29.735444+0000","last_became_peered":"2024-09-13T17:57:29.735444+0000","last_unstale":"2024-09-13T17:57:29.742650+0000","last_undegraded":"2024-09-13T17:57:29.742650+0000","last_fullsized":"2024-09-13T17:57:29.742650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:13.633882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725552+0000","last_change":"2024-09-13T17:57:25.724498+0000","last_active":"2024-09-13T17:57:29.725552+0000","last_peered":"2024-09-13T17:57:29.725552+0000","last_clean":"2024-09-13T17:57:29.725552+0000","last_became_active":"2024-09-13T17:57:25.724159+0000","last_became_peered":"2024-09-13T17:57:25.724159+0000","last_unstale":"2024-09-13T17:57:29.725552+0000","last_undegraded":"2024-09-13T17:57:29.725552+0000","last_fullsized":"2024-09-13T17:57:29.725552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:34.797142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732295+0000","last_change":"2024-09-13T17:57:25.380132+0000","last_active":"2024-09-13T17:57:29.732295+0000","last_peered":"2024-09-13T17:57:29.732295+0000","last_clean":"2024-09-13T17:57:29.732295+0000","last_became_active":"2024-09-13T17:57:25.379862+0000","last_became_peered":"2024-09-13T17:57:25.379862+0000","last_unstale":"2024-09-13T17:57:29.732295+0000","last_undegraded":"2024-09-13T17:57:29.732295+0000","last_fullsized":"2024-09-13T17:57:29.732295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:43:20.708183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.740512+0000","last_change":"2024-09-13T17:57:29.738502+0000","last_active":"2024-09-13T17:57:29.740512+0000","last_peered":"2024-09-13T17:57:29.740512+0000","last_clean":"2024-09-13T17:57:29.740512+0000","last_became_active":"2024-09-13T17:57:29.737664+0000","last_became_peered":"2024-09-13T17:57:29.737664+0000","last_unstale":"2024-09-13T17:57:29.740512+0000","last_undegraded":"2024-09-13T17:57:29.740512+0000","last_fullsized":"2024-09-13T17:57:29.740512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:13.633882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725043+0000","last_change":"2024-09-13T17:57:25.716646+0000","last_active":"2024-09-13T17:57:29.725043+0000","last_peered":"2024-09-13T17:57:29.725043+0000","last_clean":"2024-09-13T17:57:29.725043+0000","last_became_active":"2024-09-13T17:57:25.716005+0000","last_became_peered":"2024-09-13T17:57:25.716005+0000","last_unstale":"2024-09-13T17:57:29.725043+0000","last_undegraded":"2024-09-13T17:57:29.725043+0000","last_fullsized":"2024-09-13T17:57:29.725043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:43.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.730970+0000","last_change":"2024-09-13T17:57:25.374986+0000","last_active":"2024-09-13T17:57:29.730970+0000","last_peered":"2024-09-13T17:57:29.730970+0000","last_clean":"2024-09-13T17:57:29.730970+0000","last_became_active":"2024-09-13T17:57:25.374609+0000","last_became_peered":"2024-09-13T17:57:25.374609+0000","last_unstale":"2024-09-13T17:57:29.730970+0000","last_undegraded":"2024-09-13T17:57:29.730970+0000","last_fullsized":"2024-09-13T17:57:29.730970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:43:20.708183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749009+0000","last_change":"2024-09-13T17:57:29.745707+0000","last_active":"2024-09-13T17:57:29.749009+0000","last_peered":"2024-09-13T17:57:29.749009+0000","last_clean":"2024-09-13T17:57:29.749009+0000","last_became_active":"2024-09-13T17:57:29.745303+0000","last_became_peered":"2024-09-13T17:57:29.745303+0000","last_unstale":"2024-09-13T17:57:29.749009+0000","last_undegraded":"2024-09-13T17:57:29.749009+0000","last_fullsized":"2024-09-13T17:57:29.749009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:06.371464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725823+0000","last_change":"2024-09-13T17:57:25.717385+0000","last_active":"2024-09-13T17:57:29.725823+0000","last_peered":"2024-09-13T17:57:29.725823+0000","last_clean":"2024-09-13T17:57:29.725823+0000","last_became_active":"2024-09-13T17:57:25.717042+0000","last_became_peered":"2024-09-13T17:57:25.717042+0000","last_unstale":"2024-09-13T17:57:29.725823+0000","last_undegraded":"2024-09-13T17:57:29.725823+0000","last_fullsized":"2024-09-13T17:57:29.725823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:49.905668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732190+0000","last_change":"2024-09-13T17:57:25.369802+0000","last_active":"2024-09-13T17:57:29.732190+0000","last_peered":"2024-09-13T17:57:29.732190+0000","last_clean":"2024-09-13T17:57:29.732190+0000","last_became_active":"2024-09-13T17:57:25.369534+0000","last_became_peered":"2024-09-13T17:57:25.369534+0000","last_unstale":"2024-09-13T17:57:29.732190+0000","last_undegraded":"2024-09-13T17:57:29.732190+0000","last_fullsized":"2024-09-13T17:57:29.732190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742623+0000","last_change":"2024-09-13T17:57:29.737826+0000","last_active":"2024-09-13T17:57:29.742623+0000","last_peered":"2024-09-13T17:57:29.742623+0000","last_clean":"2024-09-13T17:57:29.742623+0000","last_became_active":"2024-09-13T17:57:29.737181+0000","last_became_peered":"2024-09-13T17:57:29.737181+0000","last_unstale":"2024-09-13T17:57:29.742623+0000","last_undegraded":"2024-09-13T17:57:29.742623+0000","last_fullsized":"2024-09-13T17:57:29.742623+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:06.225969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728642+0000","last_change":"2024-09-13T17:57:25.734250+0000","last_active":"2024-09-13T17:57:29.728642+0000","last_peered":"2024-09-13T17:57:29.728642+0000","last_clean":"2024-09-13T17:57:29.728642+0000","last_became_active":"2024-09-13T17:57:25.733836+0000","last_became_peered":"2024-09-13T17:57:25.733836+0000","last_unstale":"2024-09-13T17:57:29.728642+0000","last_undegraded":"2024-09-13T17:57:29.728642+0000","last_fullsized":"2024-09-13T17:57:29.728642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:05.710729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728072+0000","last_change":"2024-09-13T17:57:25.374153+0000","last_active":"2024-09-13T17:57:29.728072+0000","last_peered":"2024-09-13T17:57:29.728072+0000","last_clean":"2024-09-13T17:57:29.728072+0000","last_became_active":"2024-09-13T17:57:25.373891+0000","last_became_peered":"2024-09-13T17:57:25.373891+0000","last_unstale":"2024-09-13T17:57:29.728072+0000","last_undegraded":"2024-09-13T17:57:29.728072+0000","last_fullsized":"2024-09-13T17:57:29.728072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744947+0000","last_change":"2024-09-13T17:57:29.741648+0000","last_active":"2024-09-13T17:57:29.744947+0000","last_peered":"2024-09-13T17:57:29.744947+0000","last_clean":"2024-09-13T17:57:29.744947+0000","last_became_active":"2024-09-13T17:57:29.741315+0000","last_became_peered":"2024-09-13T17:57:29.741315+0000","last_unstale":"2024-09-13T17:57:29.744947+0000","last_undegraded":"2024-09-13T17:57:29.744947+0000","last_fullsized":"2024-09-13T17:57:29.744947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:18.432911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725228+0000","last_change":"2024-09-13T17:57:25.716312+0000","last_active":"2024-09-13T17:57:29.725228+0000","last_peered":"2024-09-13T17:57:29.725228+0000","last_clean":"2024-09-13T17:57:29.725228+0000","last_became_active":"2024-09-13T17:57:25.716067+0000","last_became_peered":"2024-09-13T17:57:25.716067+0000","last_unstale":"2024-09-13T17:57:29.725228+0000","last_undegraded":"2024-09-13T17:57:29.725228+0000","last_fullsized":"2024-09-13T17:57:29.725228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:33.971487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724948+0000","last_change":"2024-09-13T17:57:25.380277+0000","last_active":"2024-09-13T17:57:29.724948+0000","last_peered":"2024-09-13T17:57:29.724948+0000","last_clean":"2024-09-13T17:57:29.724948+0000","last_became_active":"2024-09-13T17:57:25.379988+0000","last_became_peered":"2024-09-13T17:57:25.379988+0000","last_unstale":"2024-09-13T17:57:29.724948+0000","last_undegraded":"2024-09-13T17:57:29.724948+0000","last_fullsized":"2024-09-13T17:57:29.724948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:56.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744773+0000","last_change":"2024-09-13T17:57:29.733138+0000","last_active":"2024-09-13T17:57:29.744773+0000","last_peered":"2024-09-13T17:57:29.744773+0000","last_clean":"2024-09-13T17:57:29.744773+0000","last_became_active":"2024-09-13T17:57:29.732804+0000","last_became_peered":"2024-09-13T17:57:29.732804+0000","last_unstale":"2024-09-13T17:57:29.744773+0000","last_undegraded":"2024-09-13T17:57:29.744773+0000","last_fullsized":"2024-09-13T17:57:29.744773+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:08.695591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724921+0000","last_change":"2024-09-13T17:57:25.714463+0000","last_active":"2024-09-13T17:57:29.724921+0000","last_peered":"2024-09-13T17:57:29.724921+0000","last_clean":"2024-09-13T17:57:29.724921+0000","last_became_active":"2024-09-13T17:57:25.714209+0000","last_became_peered":"2024-09-13T17:57:25.714209+0000","last_unstale":"2024-09-13T17:57:29.724921+0000","last_undegraded":"2024-09-13T17:57:29.724921+0000","last_fullsized":"2024-09-13T17:57:29.724921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727847+0000","last_change":"2024-09-13T17:57:25.374886+0000","last_active":"2024-09-13T17:57:29.727847+0000","last_peered":"2024-09-13T17:57:29.727847+0000","last_clean":"2024-09-13T17:57:29.727847+0000","last_became_active":"2024-09-13T17:57:25.374025+0000","last_became_peered":"2024-09-13T17:57:25.374025+0000","last_unstale":"2024-09-13T17:57:29.727847+0000","last_undegraded":"2024-09-13T17:57:29.727847+0000","last_fullsized":"2024-09-13T17:57:29.727847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:41.189680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.739379+0000","last_change":"2024-09-13T17:57:29.735654+0000","last_active":"2024-09-13T17:57:29.739379+0000","last_peered":"2024-09-13T17:57:29.739379+0000","last_clean":"2024-09-13T17:57:29.739379+0000","last_became_active":"2024-09-13T17:57:29.735267+0000","last_became_peered":"2024-09-13T17:57:29.735267+0000","last_unstale":"2024-09-13T17:57:29.739379+0000","last_undegraded":"2024-09-13T17:57:29.739379+0000","last_fullsized":"2024-09-13T17:57:29.739379+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:59:14.990893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725041+0000","last_change":"2024-09-13T17:57:25.722101+0000","last_active":"2024-09-13T17:57:29.725041+0000","last_peered":"2024-09-13T17:57:29.725041+0000","last_clean":"2024-09-13T17:57:29.725041+0000","last_became_active":"2024-09-13T17:57:25.721845+0000","last_became_peered":"2024-09-13T17:57:25.721845+0000","last_unstale":"2024-09-13T17:57:29.725041+0000","last_undegraded":"2024-09-13T17:57:29.725041+0000","last_fullsized":"2024-09-13T17:57:29.725041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:11.996031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:57:34.418487+0000","last_change":"2024-09-13T17:57:25.378020+0000","last_active":"2024-09-13T17:57:34.418487+0000","last_peered":"2024-09-13T17:57:34.418487+0000","last_clean":"2024-09-13T17:57:34.418487+0000","last_became_active":"2024-09-13T17:57:25.377746+0000","last_became_peered":"2024-09-13T17:57:25.377746+0000","last_unstale":"2024-09-13T17:57:34.418487+0000","last_undegraded":"2024-09-13T17:57:34.418487+0000","last_fullsized":"2024-09-13T17:57:34.418487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:58.248434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.742293+0000","last_change":"2024-09-13T17:57:29.738264+0000","last_active":"2024-09-13T17:57:29.742293+0000","last_peered":"2024-09-13T17:57:29.742293+0000","last_clean":"2024-09-13T17:57:29.742293+0000","last_became_active":"2024-09-13T17:57:29.737897+0000","last_became_peered":"2024-09-13T17:57:29.737897+0000","last_unstale":"2024-09-13T17:57:29.742293+0000","last_undegraded":"2024-09-13T17:57:29.742293+0000","last_fullsized":"2024-09-13T17:57:29.742293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:59:14.990893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.735362+0000","last_change":"2024-09-13T17:57:25.720986+0000","last_active":"2024-09-13T17:57:29.735362+0000","last_peered":"2024-09-13T17:57:29.735362+0000","last_clean":"2024-09-13T17:57:29.735362+0000","last_became_active":"2024-09-13T17:57:25.720696+0000","last_became_peered":"2024-09-13T17:57:25.720696+0000","last_unstale":"2024-09-13T17:57:29.735362+0000","last_undegraded":"2024-09-13T17:57:29.735362+0000","last_fullsized":"2024-09-13T17:57:29.735362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:01.729310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729997+0000","last_change":"2024-09-13T17:57:25.376707+0000","last_active":"2024-09-13T17:57:29.729997+0000","last_peered":"2024-09-13T17:57:29.729997+0000","last_clean":"2024-09-13T17:57:29.729997+0000","last_became_active":"2024-09-13T17:57:25.376080+0000","last_became_peered":"2024-09-13T17:57:25.376080+0000","last_unstale":"2024-09-13T17:57:29.729997+0000","last_undegraded":"2024-09-13T17:57:29.729997+0000","last_fullsized":"2024-09-13T17:57:29.729997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:28.985664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744127+0000","last_change":"2024-09-13T17:57:29.741053+0000","last_active":"2024-09-13T17:57:29.744127+0000","last_peered":"2024-09-13T17:57:29.744127+0000","last_clean":"2024-09-13T17:57:29.744127+0000","last_became_active":"2024-09-13T17:57:29.740751+0000","last_became_peered":"2024-09-13T17:57:29.740751+0000","last_unstale":"2024-09-13T17:57:29.744127+0000","last_undegraded":"2024-09-13T17:57:29.744127+0000","last_fullsized":"2024-09-13T17:57:29.744127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:56:53.973670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728126+0000","last_change":"2024-09-13T17:57:25.725767+0000","last_active":"2024-09-13T17:57:29.728126+0000","last_peered":"2024-09-13T17:57:29.728126+0000","last_clean":"2024-09-13T17:57:29.728126+0000","last_became_active":"2024-09-13T17:57:25.725565+0000","last_became_peered":"2024-09-13T17:57:25.725565+0000","last_unstale":"2024-09-13T17:57:29.728126+0000","last_undegraded":"2024-09-13T17:57:29.728126+0000","last_fullsized":"2024-09-13T17:57:29.728126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:19.536367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725379+0000","last_change":"2024-09-13T17:57:25.380232+0000","last_active":"2024-09-13T17:57:29.725379+0000","last_peered":"2024-09-13T17:57:29.725379+0000","last_clean":"2024-09-13T17:57:29.725379+0000","last_became_active":"2024-09-13T17:57:25.379910+0000","last_became_peered":"2024-09-13T17:57:25.379910+0000","last_unstale":"2024-09-13T17:57:29.725379+0000","last_undegraded":"2024-09-13T17:57:29.725379+0000","last_fullsized":"2024-09-13T17:57:29.725379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:28.958967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.749501+0000","last_change":"2024-09-13T17:57:29.745584+0000","last_active":"2024-09-13T17:57:29.749501+0000","last_peered":"2024-09-13T17:57:29.749501+0000","last_clean":"2024-09-13T17:57:29.749501+0000","last_became_active":"2024-09-13T17:57:29.745250+0000","last_became_peered":"2024-09-13T17:57:29.745250+0000","last_unstale":"2024-09-13T17:57:29.749501+0000","last_undegraded":"2024-09-13T17:57:29.749501+0000","last_fullsized":"2024-09-13T17:57:29.749501+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:12.564762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725293+0000","last_change":"2024-09-13T17:57:25.724239+0000","last_active":"2024-09-13T17:57:29.725293+0000","last_peered":"2024-09-13T17:57:29.725293+0000","last_clean":"2024-09-13T17:57:29.725293+0000","last_became_active":"2024-09-13T17:57:25.723999+0000","last_became_peered":"2024-09-13T17:57:25.723999+0000","last_unstale":"2024-09-13T17:57:29.725293+0000","last_undegraded":"2024-09-13T17:57:29.725293+0000","last_fullsized":"2024-09-13T17:57:29.725293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728742+0000","last_change":"2024-09-13T17:57:25.376498+0000","last_active":"2024-09-13T17:57:29.728742+0000","last_peered":"2024-09-13T17:57:29.728742+0000","last_clean":"2024-09-13T17:57:29.728742+0000","last_became_active":"2024-09-13T17:57:25.376171+0000","last_became_peered":"2024-09-13T17:57:25.376171+0000","last_unstale":"2024-09-13T17:57:29.728742+0000","last_undegraded":"2024-09-13T17:57:29.728742+0000","last_fullsized":"2024-09-13T17:57:29.728742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:00.725292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744624+0000","last_change":"2024-09-13T17:57:29.737219+0000","last_active":"2024-09-13T17:57:29.744624+0000","last_peered":"2024-09-13T17:57:29.744624+0000","last_clean":"2024-09-13T17:57:29.744624+0000","last_became_active":"2024-09-13T17:57:29.736853+0000","last_became_peered":"2024-09-13T17:57:29.736853+0000","last_unstale":"2024-09-13T17:57:29.744624+0000","last_undegraded":"2024-09-13T17:57:29.744624+0000","last_fullsized":"2024-09-13T17:57:29.744624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:18.526568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.738047+0000","last_change":"2024-09-13T17:57:29.733906+0000","last_active":"2024-09-13T17:57:29.738047+0000","last_peered":"2024-09-13T17:57:29.738047+0000","last_clean":"2024-09-13T17:57:29.738047+0000","last_became_active":"2024-09-13T17:57:29.733543+0000","last_became_peered":"2024-09-13T17:57:29.733543+0000","last_unstale":"2024-09-13T17:57:29.738047+0000","last_undegraded":"2024-09-13T17:57:29.738047+0000","last_fullsized":"2024-09-13T17:57:29.738047+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:37.101711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729014+0000","last_change":"2024-09-13T17:57:25.721000+0000","last_active":"2024-09-13T17:57:29.729014+0000","last_peered":"2024-09-13T17:57:29.729014+0000","last_clean":"2024-09-13T17:57:29.729014+0000","last_became_active":"2024-09-13T17:57:25.720696+0000","last_became_peered":"2024-09-13T17:57:25.720696+0000","last_unstale":"2024-09-13T17:57:29.729014+0000","last_undegraded":"2024-09-13T17:57:29.729014+0000","last_fullsized":"2024-09-13T17:57:29.729014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:34.797142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724465+0000","last_change":"2024-09-13T17:57:25.374530+0000","last_active":"2024-09-13T17:57:29.724465+0000","last_peered":"2024-09-13T17:57:29.724465+0000","last_clean":"2024-09-13T17:57:29.724465+0000","last_became_active":"2024-09-13T17:57:25.374296+0000","last_became_peered":"2024-09-13T17:57:25.374296+0000","last_unstale":"2024-09-13T17:57:29.724465+0000","last_undegraded":"2024-09-13T17:57:29.724465+0000","last_fullsized":"2024-09-13T17:57:29.724465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:15.663292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.744054+0000","last_change":"2024-09-13T17:57:29.740208+0000","last_active":"2024-09-13T17:57:29.744054+0000","last_peered":"2024-09-13T17:57:29.744054+0000","last_clean":"2024-09-13T17:57:29.744054+0000","last_became_active":"2024-09-13T17:57:29.739839+0000","last_became_peered":"2024-09-13T17:57:29.739839+0000","last_unstale":"2024-09-13T17:57:29.744054+0000","last_undegraded":"2024-09-13T17:57:29.744054+0000","last_fullsized":"2024-09-13T17:57:29.744054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:32.682495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721617+0000","last_change":"2024-09-13T17:57:25.720286+0000","last_active":"2024-09-13T17:57:29.721617+0000","last_peered":"2024-09-13T17:57:29.721617+0000","last_clean":"2024-09-13T17:57:29.721617+0000","last_became_active":"2024-09-13T17:57:25.719918+0000","last_became_peered":"2024-09-13T17:57:25.719918+0000","last_unstale":"2024-09-13T17:57:29.721617+0000","last_undegraded":"2024-09-13T17:57:29.721617+0000","last_fullsized":"2024-09-13T17:57:29.721617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:13.197645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721496+0000","last_change":"2024-09-13T17:57:25.380282+0000","last_active":"2024-09-13T17:57:29.721496+0000","last_peered":"2024-09-13T17:57:29.721496+0000","last_clean":"2024-09-13T17:57:29.721496+0000","last_became_active":"2024-09-13T17:57:25.379992+0000","last_became_peered":"2024-09-13T17:57:25.379992+0000","last_unstale":"2024-09-13T17:57:29.721496+0000","last_undegraded":"2024-09-13T17:57:29.721496+0000","last_fullsized":"2024-09-13T17:57:29.721496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:04.441662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728265+0000","last_change":"2024-09-13T17:57:25.373463+0000","last_active":"2024-09-13T17:57:29.728265+0000","last_peered":"2024-09-13T17:57:29.728265+0000","last_clean":"2024-09-13T17:57:29.728265+0000","last_became_active":"2024-09-13T17:57:25.373127+0000","last_became_peered":"2024-09-13T17:57:25.373127+0000","last_unstale":"2024-09-13T17:57:29.728265+0000","last_undegraded":"2024-09-13T17:57:29.728265+0000","last_fullsized":"2024-09-13T17:57:29.728265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:15.663292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727753+0000","last_change":"2024-09-13T17:57:25.719108+0000","last_active":"2024-09-13T17:57:29.727753+0000","last_peered":"2024-09-13T17:57:29.727753+0000","last_clean":"2024-09-13T17:57:29.727753+0000","last_became_active":"2024-09-13T17:57:25.718674+0000","last_became_peered":"2024-09-13T17:57:25.718674+0000","last_unstale":"2024-09-13T17:57:29.727753+0000","last_undegraded":"2024-09-13T17:57:29.727753+0000","last_fullsized":"2024-09-13T17:57:29.727753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:49.905668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.740987+0000","last_change":"2024-09-13T17:57:29.736695+0000","last_active":"2024-09-13T17:57:29.740987+0000","last_peered":"2024-09-13T17:57:29.740987+0000","last_clean":"2024-09-13T17:57:29.740987+0000","last_became_active":"2024-09-13T17:57:29.735361+0000","last_became_peered":"2024-09-13T17:57:29.735361+0000","last_unstale":"2024-09-13T17:57:29.740987+0000","last_undegraded":"2024-09-13T17:57:29.740987+0000","last_fullsized":"2024-09-13T17:57:29.740987+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:50.823708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720788+0000","last_change":"2024-09-13T17:57:25.378442+0000","last_active":"2024-09-13T17:57:29.720788+0000","last_peered":"2024-09-13T17:57:29.720788+0000","last_clean":"2024-09-13T17:57:29.720788+0000","last_became_active":"2024-09-13T17:57:25.378092+0000","last_became_peered":"2024-09-13T17:57:25.378092+0000","last_unstale":"2024-09-13T17:57:29.720788+0000","last_undegraded":"2024-09-13T17:57:29.720788+0000","last_fullsized":"2024-09-13T17:57:29.720788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:24.348731+0000","last_clean_scrub_stamp":"2024-09-13T17:57:24.348731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:41.189680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.733727+0000","last_change":"2024-09-13T17:57:25.714970+0000","last_active":"2024-09-13T17:57:29.733727+0000","last_peered":"2024-09-13T17:57:29.733727+0000","last_clean":"2024-09-13T17:57:29.733727+0000","last_became_active":"2024-09-13T17:57:25.714338+0000","last_became_peered":"2024-09-13T17:57:25.714338+0000","last_unstale":"2024-09-13T17:57:29.733727+0000","last_undegraded":"2024-09-13T17:57:29.733727+0000","last_fullsized":"2024-09-13T17:57:29.733727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:11:58.896243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.745004+0000","last_change":"2024-09-13T17:57:29.741838+0000","last_active":"2024-09-13T17:57:29.745004+0000","last_peered":"2024-09-13T17:57:29.745004+0000","last_clean":"2024-09-13T17:57:29.745004+0000","last_became_active":"2024-09-13T17:57:29.741478+0000","last_became_peered":"2024-09-13T17:57:29.741478+0000","last_unstale":"2024-09-13T17:57:29.745004+0000","last_undegraded":"2024-09-13T17:57:29.745004+0000","last_fullsized":"2024-09-13T17:57:29.745004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:28.712870+0000","last_clean_scrub_stamp":"2024-09-13T17:57:28.712870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:15.815856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728583+0000","last_change":"2024-09-13T17:57:25.734081+0000","last_active":"2024-09-13T17:57:29.728583+0000","last_peered":"2024-09-13T17:57:29.728583+0000","last_clean":"2024-09-13T17:57:29.728583+0000","last_became_active":"2024-09-13T17:57:25.733489+0000","last_became_peered":"2024-09-13T17:57:25.733489+0000","last_unstale":"2024-09-13T17:57:29.728583+0000","last_undegraded":"2024-09-13T17:57:29.728583+0000","last_fullsized":"2024-09-13T17:57:29.728583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:49.905668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.731942+0000","last_change":"2024-09-13T17:57:25.721413+0000","last_active":"2024-09-13T17:57:29.731942+0000","last_peered":"2024-09-13T17:57:29.731942+0000","last_clean":"2024-09-13T17:57:29.731942+0000","last_became_active":"2024-09-13T17:57:25.721167+0000","last_became_peered":"2024-09-13T17:57:25.721167+0000","last_unstale":"2024-09-13T17:57:29.731942+0000","last_undegraded":"2024-09-13T17:57:29.731942+0000","last_fullsized":"2024-09-13T17:57:29.731942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:39.042757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724511+0000","last_change":"2024-09-13T17:57:25.728836+0000","last_active":"2024-09-13T17:57:29.724511+0000","last_peered":"2024-09-13T17:57:29.724511+0000","last_clean":"2024-09-13T17:57:29.724511+0000","last_became_active":"2024-09-13T17:57:25.728566+0000","last_became_peered":"2024-09-13T17:57:25.728566+0000","last_unstale":"2024-09-13T17:57:29.724511+0000","last_undegraded":"2024-09-13T17:57:29.724511+0000","last_fullsized":"2024-09-13T17:57:29.724511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:50.812435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727862+0000","last_change":"2024-09-13T17:57:25.719773+0000","last_active":"2024-09-13T17:57:29.727862+0000","last_peered":"2024-09-13T17:57:29.727862+0000","last_clean":"2024-09-13T17:57:29.727862+0000","last_became_active":"2024-09-13T17:57:25.719057+0000","last_became_peered":"2024-09-13T17:57:25.719057+0000","last_unstale":"2024-09-13T17:57:29.727862+0000","last_undegraded":"2024-09-13T17:57:29.727862+0000","last_fullsized":"2024-09-13T17:57:29.727862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:13.197645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721032+0000","last_change":"2024-09-13T17:57:25.719998+0000","last_active":"2024-09-13T17:57:29.721032+0000","last_peered":"2024-09-13T17:57:29.721032+0000","last_clean":"2024-09-13T17:57:29.721032+0000","last_became_active":"2024-09-13T17:57:25.719739+0000","last_became_peered":"2024-09-13T17:57:25.719739+0000","last_unstale":"2024-09-13T17:57:29.721032+0000","last_undegraded":"2024-09-13T17:57:29.721032+0000","last_fullsized":"2024-09-13T17:57:29.721032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:47.463588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729592+0000","last_change":"2024-09-13T17:57:25.720277+0000","last_active":"2024-09-13T17:57:29.729592+0000","last_peered":"2024-09-13T17:57:29.729592+0000","last_clean":"2024-09-13T17:57:29.729592+0000","last_became_active":"2024-09-13T17:57:25.720034+0000","last_became_peered":"2024-09-13T17:57:25.720034+0000","last_unstale":"2024-09-13T17:57:29.729592+0000","last_undegraded":"2024-09-13T17:57:29.729592+0000","last_fullsized":"2024-09-13T17:57:29.729592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728536+0000","last_change":"2024-09-13T17:57:25.733794+0000","last_active":"2024-09-13T17:57:29.728536+0000","last_peered":"2024-09-13T17:57:29.728536+0000","last_clean":"2024-09-13T17:57:29.728536+0000","last_became_active":"2024-09-13T17:57:25.733512+0000","last_became_peered":"2024-09-13T17:57:25.733512+0000","last_unstale":"2024-09-13T17:57:29.728536+0000","last_undegraded":"2024-09-13T17:57:29.728536+0000","last_fullsized":"2024-09-13T17:57:29.728536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720839+0000","last_change":"2024-09-13T17:57:25.723300+0000","last_active":"2024-09-13T17:57:29.720839+0000","last_peered":"2024-09-13T17:57:29.720839+0000","last_clean":"2024-09-13T17:57:29.720839+0000","last_became_active":"2024-09-13T17:57:25.720475+0000","last_became_peered":"2024-09-13T17:57:25.720475+0000","last_unstale":"2024-09-13T17:57:29.720839+0000","last_undegraded":"2024-09-13T17:57:29.720839+0000","last_fullsized":"2024-09-13T17:57:29.720839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:37:23.859769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724381+0000","last_change":"2024-09-13T17:57:25.721201+0000","last_active":"2024-09-13T17:57:29.724381+0000","last_peered":"2024-09-13T17:57:29.724381+0000","last_clean":"2024-09-13T17:57:29.724381+0000","last_became_active":"2024-09-13T17:57:25.719998+0000","last_became_peered":"2024-09-13T17:57:25.719998+0000","last_unstale":"2024-09-13T17:57:29.724381+0000","last_undegraded":"2024-09-13T17:57:29.724381+0000","last_fullsized":"2024-09-13T17:57:29.724381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:33.971487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.721180+0000","last_change":"2024-09-13T17:57:25.715726+0000","last_active":"2024-09-13T17:57:29.721180+0000","last_peered":"2024-09-13T17:57:29.721180+0000","last_clean":"2024-09-13T17:57:29.721180+0000","last_became_active":"2024-09-13T17:57:25.715459+0000","last_became_peered":"2024-09-13T17:57:25.715459+0000","last_unstale":"2024-09-13T17:57:29.721180+0000","last_undegraded":"2024-09-13T17:57:29.721180+0000","last_fullsized":"2024-09-13T17:57:29.721180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:49.905668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724413+0000","last_change":"2024-09-13T17:57:25.730059+0000","last_active":"2024-09-13T17:57:29.724413+0000","last_peered":"2024-09-13T17:57:29.724413+0000","last_clean":"2024-09-13T17:57:29.724413+0000","last_became_active":"2024-09-13T17:57:25.729083+0000","last_became_peered":"2024-09-13T17:57:25.729083+0000","last_unstale":"2024-09-13T17:57:29.724413+0000","last_undegraded":"2024-09-13T17:57:29.724413+0000","last_fullsized":"2024-09-13T17:57:29.724413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:12.960061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.727633+0000","last_change":"2024-09-13T17:57:25.713791+0000","last_active":"2024-09-13T17:57:29.727633+0000","last_peered":"2024-09-13T17:57:29.727633+0000","last_clean":"2024-09-13T17:57:29.727633+0000","last_became_active":"2024-09-13T17:57:25.713540+0000","last_became_peered":"2024-09-13T17:57:25.713540+0000","last_unstale":"2024-09-13T17:57:29.727633+0000","last_undegraded":"2024-09-13T17:57:29.727633+0000","last_fullsized":"2024-09-13T17:57:29.727633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728046+0000","last_change":"2024-09-13T17:57:25.734509+0000","last_active":"2024-09-13T17:57:29.728046+0000","last_peered":"2024-09-13T17:57:29.728046+0000","last_clean":"2024-09-13T17:57:29.728046+0000","last_became_active":"2024-09-13T17:57:25.734233+0000","last_became_peered":"2024-09-13T17:57:25.734233+0000","last_unstale":"2024-09-13T17:57:29.728046+0000","last_undegraded":"2024-09-13T17:57:29.728046+0000","last_fullsized":"2024-09-13T17:57:29.728046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:35.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725310+0000","last_change":"2024-09-13T17:57:25.718661+0000","last_active":"2024-09-13T17:57:29.725310+0000","last_peered":"2024-09-13T17:57:29.725310+0000","last_clean":"2024-09-13T17:57:29.725310+0000","last_became_active":"2024-09-13T17:57:25.718366+0000","last_became_peered":"2024-09-13T17:57:25.718366+0000","last_unstale":"2024-09-13T17:57:29.725310+0000","last_undegraded":"2024-09-13T17:57:29.725310+0000","last_fullsized":"2024-09-13T17:57:29.725310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:12.960061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.732206+0000","last_change":"2024-09-13T17:57:25.734683+0000","last_active":"2024-09-13T17:57:29.732206+0000","last_peered":"2024-09-13T17:57:29.732206+0000","last_clean":"2024-09-13T17:57:29.732206+0000","last_became_active":"2024-09-13T17:57:25.734409+0000","last_became_peered":"2024-09-13T17:57:25.734409+0000","last_unstale":"2024-09-13T17:57:29.732206+0000","last_undegraded":"2024-09-13T17:57:29.732206+0000","last_fullsized":"2024-09-13T17:57:29.732206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:59.252453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725864+0000","last_change":"2024-09-13T17:57:25.729680+0000","last_active":"2024-09-13T17:57:29.725864+0000","last_peered":"2024-09-13T17:57:29.725864+0000","last_clean":"2024-09-13T17:57:29.725864+0000","last_became_active":"2024-09-13T17:57:25.729402+0000","last_became_peered":"2024-09-13T17:57:25.729402+0000","last_unstale":"2024-09-13T17:57:29.725864+0000","last_undegraded":"2024-09-13T17:57:29.725864+0000","last_fullsized":"2024-09-13T17:57:29.725864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:43.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724296+0000","last_change":"2024-09-13T17:57:25.728126+0000","last_active":"2024-09-13T17:57:29.724296+0000","last_peered":"2024-09-13T17:57:29.724296+0000","last_clean":"2024-09-13T17:57:29.724296+0000","last_became_active":"2024-09-13T17:57:25.727793+0000","last_became_peered":"2024-09-13T17:57:25.727793+0000","last_unstale":"2024-09-13T17:57:29.724296+0000","last_undegraded":"2024-09-13T17:57:29.724296+0000","last_fullsized":"2024-09-13T17:57:29.724296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:34.797142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729099+0000","last_change":"2024-09-13T17:57:25.716918+0000","last_active":"2024-09-13T17:57:29.729099+0000","last_peered":"2024-09-13T17:57:29.729099+0000","last_clean":"2024-09-13T17:57:29.729099+0000","last_became_active":"2024-09-13T17:57:25.716550+0000","last_became_peered":"2024-09-13T17:57:25.716550+0000","last_unstale":"2024-09-13T17:57:29.729099+0000","last_undegraded":"2024-09-13T17:57:29.729099+0000","last_fullsized":"2024-09-13T17:57:29.729099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:19.536367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724836+0000","last_change":"2024-09-13T17:57:25.717079+0000","last_active":"2024-09-13T17:57:29.724836+0000","last_peered":"2024-09-13T17:57:29.724836+0000","last_clean":"2024-09-13T17:57:29.724836+0000","last_became_active":"2024-09-13T17:57:25.716782+0000","last_became_peered":"2024-09-13T17:57:25.716782+0000","last_unstale":"2024-09-13T17:57:29.724836+0000","last_undegraded":"2024-09-13T17:57:29.724836+0000","last_fullsized":"2024-09-13T17:57:29.724836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:26:19.049804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720610+0000","last_change":"2024-09-13T17:57:25.731733+0000","last_active":"2024-09-13T17:57:29.720610+0000","last_peered":"2024-09-13T17:57:29.720610+0000","last_clean":"2024-09-13T17:57:29.720610+0000","last_became_active":"2024-09-13T17:57:25.731460+0000","last_became_peered":"2024-09-13T17:57:25.731460+0000","last_unstale":"2024-09-13T17:57:29.720610+0000","last_undegraded":"2024-09-13T17:57:29.720610+0000","last_fullsized":"2024-09-13T17:57:29.720610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:43.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.735275+0000","last_change":"2024-09-13T17:57:25.734897+0000","last_active":"2024-09-13T17:57:29.735275+0000","last_peered":"2024-09-13T17:57:29.735275+0000","last_clean":"2024-09-13T17:57:29.735275+0000","last_became_active":"2024-09-13T17:57:25.734639+0000","last_became_peered":"2024-09-13T17:57:25.734639+0000","last_unstale":"2024-09-13T17:57:29.735275+0000","last_undegraded":"2024-09-13T17:57:29.735275+0000","last_fullsized":"2024-09-13T17:57:29.735275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:29.989683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728450+0000","last_change":"2024-09-13T17:57:25.736997+0000","last_active":"2024-09-13T17:57:29.728450+0000","last_peered":"2024-09-13T17:57:29.728450+0000","last_clean":"2024-09-13T17:57:29.728450+0000","last_became_active":"2024-09-13T17:57:25.736735+0000","last_became_peered":"2024-09-13T17:57:25.736735+0000","last_unstale":"2024-09-13T17:57:29.728450+0000","last_undegraded":"2024-09-13T17:57:29.728450+0000","last_fullsized":"2024-09-13T17:57:29.728450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.728451+0000","last_change":"2024-09-13T17:57:25.736821+0000","last_active":"2024-09-13T17:57:29.728451+0000","last_peered":"2024-09-13T17:57:29.728451+0000","last_clean":"2024-09-13T17:57:29.728451+0000","last_became_active":"2024-09-13T17:57:25.736572+0000","last_became_peered":"2024-09-13T17:57:25.736572+0000","last_unstale":"2024-09-13T17:57:29.728451+0000","last_undegraded":"2024-09-13T17:57:29.728451+0000","last_fullsized":"2024-09-13T17:57:29.728451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:13.197645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725470+0000","last_change":"2024-09-13T17:57:25.725989+0000","last_active":"2024-09-13T17:57:29.725470+0000","last_peered":"2024-09-13T17:57:29.725470+0000","last_clean":"2024-09-13T17:57:29.725470+0000","last_became_active":"2024-09-13T17:57:25.725725+0000","last_became_peered":"2024-09-13T17:57:25.725725+0000","last_unstale":"2024-09-13T17:57:29.725470+0000","last_undegraded":"2024-09-13T17:57:29.725470+0000","last_fullsized":"2024-09-13T17:57:29.725470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:50.812435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.720423+0000","last_change":"2024-09-13T17:57:25.723786+0000","last_active":"2024-09-13T17:57:29.720423+0000","last_peered":"2024-09-13T17:57:29.720423+0000","last_clean":"2024-09-13T17:57:29.720423+0000","last_became_active":"2024-09-13T17:57:25.723510+0000","last_became_peered":"2024-09-13T17:57:25.723510+0000","last_unstale":"2024-09-13T17:57:29.720423+0000","last_undegraded":"2024-09-13T17:57:29.720423+0000","last_fullsized":"2024-09-13T17:57:29.720423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725741+0000","last_change":"2024-09-13T17:57:25.721799+0000","last_active":"2024-09-13T17:57:29.725741+0000","last_peered":"2024-09-13T17:57:29.725741+0000","last_clean":"2024-09-13T17:57:29.725741+0000","last_became_active":"2024-09-13T17:57:25.721309+0000","last_became_peered":"2024-09-13T17:57:25.721309+0000","last_unstale":"2024-09-13T17:57:29.725741+0000","last_undegraded":"2024-09-13T17:57:29.725741+0000","last_fullsized":"2024-09-13T17:57:29.725741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:27.969805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.724305+0000","last_change":"2024-09-13T17:57:25.727329+0000","last_active":"2024-09-13T17:57:29.724305+0000","last_peered":"2024-09-13T17:57:29.724305+0000","last_clean":"2024-09-13T17:57:29.724305+0000","last_became_active":"2024-09-13T17:57:25.727064+0000","last_became_peered":"2024-09-13T17:57:25.727064+0000","last_unstale":"2024-09-13T17:57:29.724305+0000","last_undegraded":"2024-09-13T17:57:29.724305+0000","last_fullsized":"2024-09-13T17:57:29.724305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:42.691724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725134+0000","last_change":"2024-09-13T17:57:25.714493+0000","last_active":"2024-09-13T17:57:29.725134+0000","last_peered":"2024-09-13T17:57:29.725134+0000","last_clean":"2024-09-13T17:57:29.725134+0000","last_became_active":"2024-09-13T17:57:25.714091+0000","last_became_peered":"2024-09-13T17:57:25.714091+0000","last_unstale":"2024-09-13T17:57:29.725134+0000","last_undegraded":"2024-09-13T17:57:29.725134+0000","last_fullsized":"2024-09-13T17:57:29.725134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:51.804478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725105+0000","last_change":"2024-09-13T17:57:25.730275+0000","last_active":"2024-09-13T17:57:29.725105+0000","last_peered":"2024-09-13T17:57:29.725105+0000","last_clean":"2024-09-13T17:57:29.725105+0000","last_became_active":"2024-09-13T17:57:25.729845+0000","last_became_peered":"2024-09-13T17:57:25.729845+0000","last_unstale":"2024-09-13T17:57:29.725105+0000","last_undegraded":"2024-09-13T17:57:29.725105+0000","last_fullsized":"2024-09-13T17:57:29.725105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:11.996031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725148+0000","last_change":"2024-09-13T17:57:25.714547+0000","last_active":"2024-09-13T17:57:29.725148+0000","last_peered":"2024-09-13T17:57:29.725148+0000","last_clean":"2024-09-13T17:57:29.725148+0000","last_became_active":"2024-09-13T17:57:25.714288+0000","last_became_peered":"2024-09-13T17:57:25.714288+0000","last_unstale":"2024-09-13T17:57:29.725148+0000","last_undegraded":"2024-09-13T17:57:29.725148+0000","last_fullsized":"2024-09-13T17:57:29.725148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:37:23.859769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.729416+0000","last_change":"2024-09-13T17:57:25.727117+0000","last_active":"2024-09-13T17:57:29.729416+0000","last_peered":"2024-09-13T17:57:29.729416+0000","last_clean":"2024-09-13T17:57:29.729416+0000","last_became_active":"2024-09-13T17:57:25.726864+0000","last_became_peered":"2024-09-13T17:57:25.726864+0000","last_unstale":"2024-09-13T17:57:29.729416+0000","last_undegraded":"2024-09-13T17:57:29.729416+0000","last_fullsized":"2024-09-13T17:57:29.729416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:56:48.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:57:29.725395+0000","last_change":"2024-09-13T17:57:25.734566+0000","last_active":"2024-09-13T17:57:29.725395+0000","last_peered":"2024-09-13T17:57:29.725395+0000","last_clean":"2024-09-13T17:57:29.725395+0000","last_became_active":"2024-09-13T17:57:25.733963+0000","last_became_peered":"2024-09-13T17:57:25.733963+0000","last_unstale":"2024-09-13T17:57:29.725395+0000","last_undegraded":"2024-09-13T17:57:29.725395+0000","last_fullsized":"2024-09-13T17:57:29.725395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:25.352749+0000","last_clean_scrub_stamp":"2024-09-13T17:57:25.352749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:44.248672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":690730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":99870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":99906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.248}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":99780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":99940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":689767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:57:51.826 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T17:57:51.826 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T17:57:51.826 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T17:57:51.826 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T17:57:51.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.954+0000 7f58c0308640 1 Processor -- start 2024-09-13T17:57:51.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.954+0000 7f58c0308640 1 -- start start 2024-09-13T17:57:51.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.955+0000 7f58c0308640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b815c590 0x7f58b806b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.955+0000 7f58c0308640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b815c970 con 0x7f58b80cfff0 2024-09-13T17:57:51.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.955+0000 7f58c0308640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b80d0cc0 con 0x7f58b80d23b0 2024-09-13T17:57:51.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.955+0000 7f58c0308640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80d0fa0 con 0x7f58b815c590 2024-09-13T17:57:51.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.955+0000 7f58be8a5640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f58b80cfff0 0x7f58b80d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:49854/0 (socket says 172.21.15.111:49854) 2024-09-13T17:57:51.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.955+0000 7f58be8a5640 1 -- 172.21.15.111:0/731305245 learned_addr learned my addr 172.21.15.111:0/731305245 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:51.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd0a2640 1 -- 172.21.15.111:0/731305245 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162758387 0 0) 0x7f58b815c970 con 0x7f58b80cfff0 2024-09-13T17:57:51.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58be0a4640 1 --2- 172.21.15.111:0/731305245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b815c590 0x7f58b806b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd0a2640 1 -- 172.21.15.111:0/731305245 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58a0003520 con 0x7f58b80cfff0 2024-09-13T17:57:51.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd0a2640 1 -- 172.21.15.111:0/731305245 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3507391650 0 0) 0x7f58a0003520 con 0x7f58b80cfff0 2024-09-13T17:57:51.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd0a2640 1 -- 172.21.15.111:0/731305245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b815c590 msgr2=0x7f58b806b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:51.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd0a2640 1 --2- 172.21.15.111:0/731305245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b815c590 0x7f58b806b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd0a2640 1 -- 172.21.15.111:0/731305245 >> v1:172.21.15.188:6789/0 conn(0x7f58b80d23b0 legacy=0x7f58b80d27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd0a2640 1 -- 172.21.15.111:0/731305245 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b80d1280 con 0x7f58b80cfff0 2024-09-13T17:57:51.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd0a2640 1 -- 172.21.15.111:0/731305245 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f58a8001670 con 0x7f58b80cfff0 2024-09-13T17:57:51.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd0a2640 1 -- 172.21.15.111:0/731305245 >> v1:172.21.15.111:6789/0 conn(0x7f58b80cfff0 legacy=0x7f58b80d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd0a2640 1 --2- 172.21.15.111:0/731305245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58a00039a0 0x7f58a0003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd0a2640 1 --2- 172.21.15.111:0/731305245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58a0004350 0x7f58a0004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd0a2640 1 --2- 172.21.15.111:0/731305245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b815c590 0x7f58a0005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd0a2640 1 -- 172.21.15.111:0/731305245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80d0fa0 con 0x7f58a0004350 2024-09-13T17:57:51.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd0a2640 1 -- 172.21.15.111:0/731305245 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a000ba30 con 0x7f58a00039a0 2024-09-13T17:57:51.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd0a2640 1 -- 172.21.15.111:0/731305245 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a000bbf0 con 0x7f58b815c590 2024-09-13T17:57:51.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd8a3640 1 --2- 172.21.15.111:0/731305245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58a0004350 0x7f58a0004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58be8a5640 1 --2- 172.21.15.111:0/731305245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b815c590 0x7f58a0005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58be0a4640 1 --2- 172.21.15.111:0/731305245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58a00039a0 0x7f58a0003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd8a3640 1 -- 172.21.15.111:0/731305245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b815c590 msgr2=0x7f58a0005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:51.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd8a3640 1 --2- 172.21.15.111:0/731305245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b815c590 0x7f58a0005120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd8a3640 1 -- 172.21.15.111:0/731305245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58a00039a0 msgr2=0x7f58a0003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:51.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd8a3640 1 --2- 172.21.15.111:0/731305245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58a00039a0 0x7f58a0003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.956+0000 7f58bd8a3640 1 -- 172.21.15.111:0/731305245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a0003520 con 0x7f58a0004350 2024-09-13T17:57:51.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.957+0000 7f58bd8a3640 1 --2- 172.21.15.111:0/731305245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58a0004350 0x7f58a0004820 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f58ac001f80 tx=0x7f58a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fd026c41624712 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:51.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.957+0000 7f58bd0a2640 1 -- 172.21.15.111:0/731305245 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac031600 con 0x7f58a0004350 2024-09-13T17:57:51.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.957+0000 7f58bd0a2640 1 -- 172.21.15.111:0/731305245 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac02d050 con 0x7f58a0004350 2024-09-13T17:57:51.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.957+0000 7f58bd0a2640 1 -- 172.21.15.111:0/731305245 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0319c0 con 0x7f58a0004350 2024-09-13T17:57:51.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.958+0000 7f58c0308640 1 -- 172.21.15.111:0/731305245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58a0004350 msgr2=0x7f58a0004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.958+0000 7f58c0308640 1 --2- 172.21.15.111:0/731305245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58a0004350 0x7f58a0004820 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f58ac001f80 tx=0x7f58a00057f0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.958+0000 7f58be8a5640 1 -- 172.21.15.111:0/731305245 reap_dead start 2024-09-13T17:57:51.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.958+0000 7f58c0308640 1 -- 172.21.15.111:0/731305245 shutdown_connections 2024-09-13T17:57:51.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.958+0000 7f58c0308640 1 -- 172.21.15.111:0/731305245 >> 172.21.15.111:0/731305245 conn(0x7f58b808bf20 msgr2=0x7f58b809b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:51.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.958+0000 7f58c0308640 1 -- 172.21.15.111:0/731305245 shutdown_connections 2024-09-13T17:57:51.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.958+0000 7f58c0308640 1 -- 172.21.15.111:0/731305245 wait complete. 2024-09-13T17:57:51.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.959+0000 7f58c0308640 1 Processor -- start 2024-09-13T17:57:51.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.959+0000 7f58c0308640 1 -- start start 2024-09-13T17:57:51.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.960+0000 7f58c0308640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58a0004350 0x7f58b80c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.960+0000 7f58c0308640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b8158c30 0x7f58b80cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.960+0000 7f58c0308640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b80c99c0 0x7f58b80c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.960+0000 7f58c0308640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80ca390 con 0x7f58a0004350 2024-09-13T17:57:51.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.960+0000 7f58c0308640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b81590b0 con 0x7f58b80c99c0 2024-09-13T17:57:51.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.960+0000 7f58c0308640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80b7a90 con 0x7f58b8158c30 2024-09-13T17:57:51.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.960+0000 7f58be0a4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58a0004350 0x7f58b80c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.960+0000 7f58be0a4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58a0004350 0x7f58b80c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54294/0 (socket says 172.21.15.111:54294) 2024-09-13T17:57:51.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.960+0000 7f58be0a4640 1 -- 172.21.15.111:0/3110143715 learned_addr learned my addr 172.21.15.111:0/3110143715 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:51.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.960+0000 7f58bd8a3640 1 --2- 172.21.15.111:0/3110143715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b8158c30 0x7f58b80cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.960+0000 7f58be8a5640 1 --2- 172.21.15.111:0/3110143715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b80c99c0 0x7f58b80c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.960+0000 7f58be0a4640 1 -- 172.21.15.111:0/3110143715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b8158c30 msgr2=0x7f58b80cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.960+0000 7f58be0a4640 1 --2- 172.21.15.111:0/3110143715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b8158c30 0x7f58b80cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.960+0000 7f58be0a4640 1 -- 172.21.15.111:0/3110143715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b80c99c0 msgr2=0x7f58b80c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.960+0000 7f58be0a4640 1 --2- 172.21.15.111:0/3110143715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b80c99c0 0x7f58b80c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.960+0000 7f58be0a4640 1 -- 172.21.15.111:0/3110143715 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b80b7d70 con 0x7f58a0004350 2024-09-13T17:57:51.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.961+0000 7f58be0a4640 1 --2- 172.21.15.111:0/3110143715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58a0004350 0x7f58b80c6c10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f58b00118e0 tx=0x7f58b0011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:51.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.961+0000 7f58b6ffd640 1 -- 172.21.15.111:0/3110143715 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b000fa90 con 0x7f58a0004350 2024-09-13T17:57:51.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.961+0000 7f58b6ffd640 1 -- 172.21.15.111:0/3110143715 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b000fbf0 con 0x7f58a0004350 2024-09-13T17:57:51.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.961+0000 7f58c0308640 1 -- 172.21.15.111:0/3110143715 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b80b8050 con 0x7f58a0004350 2024-09-13T17:57:51.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.961+0000 7f58b6ffd640 1 -- 172.21.15.111:0/3110143715 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0014440 con 0x7f58a0004350 2024-09-13T17:57:51.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.961+0000 7f58c0308640 1 -- 172.21.15.111:0/3110143715 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b80b55d0 con 0x7f58a0004350 2024-09-13T17:57:51.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.962+0000 7f58c0308640 1 -- 172.21.15.111:0/3110143715 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b80d04a0 con 0x7f58a0004350 2024-09-13T17:57:51.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.962+0000 7f58b6ffd640 1 -- 172.21.15.111:0/3110143715 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f58b00145a0 con 0x7f58a0004350 2024-09-13T17:57:51.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.962+0000 7f58b6ffd640 1 --2- 172.21.15.111:0/3110143715 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f58a406ac30 0x7f58a406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.963+0000 7f58b6ffd640 1 -- 172.21.15.111:0/3110143715 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f58b0089cd0 con 0x7f58a0004350 2024-09-13T17:57:51.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.963+0000 7f58bd8a3640 1 --2- 172.21.15.111:0/3110143715 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f58a406ac30 0x7f58a406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.963+0000 7f58bd8a3640 1 --2- 172.21.15.111:0/3110143715 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f58a406ac30 0x7f58a406d0f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f58ac034000 tx=0x7f58ac02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:51.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:51.967+0000 7f58b6ffd640 1 -- 172.21.15.111:0/3110143715 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f58b0054c70 con 0x7f58a0004350 2024-09-13T17:57:52.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.161+0000 7f58c0308640 1 -- 172.21.15.111:0/3110143715 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f58b80cc120 con 0x7f58a0004350 2024-09-13T17:57:52.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.161+0000 7f58b6ffd640 1 -- 172.21.15.111:0/3110143715 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f58b0058290 con 0x7f58a0004350 2024-09-13T17:57:52.164 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:52.164 INFO:teuthology.orchestra.run.smithi111.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T17:57:52.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.163+0000 7f58c0308640 1 -- 172.21.15.111:0/3110143715 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f58a406ac30 msgr2=0x7f58a406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.163+0000 7f58c0308640 1 --2- 172.21.15.111:0/3110143715 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f58a406ac30 0x7f58a406d0f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f58ac034000 tx=0x7f58ac02f000 comp rx=0 tx=0).stop 2024-09-13T17:57:52.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.163+0000 7f58c0308640 1 -- 172.21.15.111:0/3110143715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58a0004350 msgr2=0x7f58b80c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.163+0000 7f58c0308640 1 --2- 172.21.15.111:0/3110143715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58a0004350 0x7f58b80c6c10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f58b00118e0 tx=0x7f58b0011da0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.163+0000 7f58c0308640 1 -- 172.21.15.111:0/3110143715 shutdown_connections 2024-09-13T17:57:52.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.163+0000 7f58c0308640 1 --2- 172.21.15.111:0/3110143715 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f58a406ac30 0x7f58a406d0f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.163+0000 7f58c0308640 1 --2- 172.21.15.111:0/3110143715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b80c99c0 0x7f58b80c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.163+0000 7f58c0308640 1 --2- 172.21.15.111:0/3110143715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b8158c30 0x7f58b80cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.163+0000 7f58c0308640 1 --2- 172.21.15.111:0/3110143715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58a0004350 0x7f58b80c6c10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.163+0000 7f58c0308640 1 -- 172.21.15.111:0/3110143715 >> 172.21.15.111:0/3110143715 conn(0x7f58b808bf20 msgr2=0x7f58b80b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:52.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.163+0000 7f58c0308640 1 -- 172.21.15.111:0/3110143715 shutdown_connections 2024-09-13T17:57:52.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.163+0000 7f58c0308640 1 -- 172.21.15.111:0/3110143715 wait complete. 2024-09-13T17:57:52.176 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T17:57:52.177 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:52.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.271+0000 7ffacb783640 1 Processor -- start 2024-09-13T17:57:52.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.271+0000 7ffacb783640 1 -- start start 2024-09-13T17:57:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffacb783640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac415c6e0 0x7ffac40b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffacb783640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffac40ccb20 0x7ffac40b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffacb783640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac40d1840 con 0x7ffac40d07b0 2024-09-13T17:57:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffacb783640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac406bd20 con 0x7ffac40ccb20 2024-09-13T17:57:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffacb783640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac40d0c30 con 0x7ffac415c6e0 2024-09-13T17:57:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffac8d1e640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ffac40d07b0 0x7ffac4160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35574/0 (socket says 172.21.15.111:35574) 2024-09-13T17:57:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffac8d1e640 1 -- 172.21.15.111:0/680841007 learned_addr learned my addr 172.21.15.111:0/680841007 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffac9d20640 1 --2- 172.21.15.111:0/680841007 >> v2:172.21.15.188:3300/0 conn(0x7ffac40ccb20 0x7ffac40b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffac951f640 1 --2- 172.21.15.111:0/680841007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac415c6e0 0x7ffac40b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffabbfff640 1 -- 172.21.15.111:0/680841007 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177830923 0 0) 0x7ffac40d1840 con 0x7ffac40d07b0 2024-09-13T17:57:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffabbfff640 1 -- 172.21.15.111:0/680841007 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffab0003520 con 0x7ffac40d07b0 2024-09-13T17:57:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffabbfff640 1 -- 172.21.15.111:0/680841007 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1978864930 0 0) 0x7ffab0003520 con 0x7ffac40d07b0 2024-09-13T17:57:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffabbfff640 1 -- 172.21.15.111:0/680841007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac415c6e0 msgr2=0x7ffac40b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffabbfff640 1 --2- 172.21.15.111:0/680841007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac415c6e0 0x7ffac40b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffabbfff640 1 -- 172.21.15.111:0/680841007 >> v2:172.21.15.188:3300/0 conn(0x7ffac40ccb20 msgr2=0x7ffac40b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffabbfff640 1 --2- 172.21.15.111:0/680841007 >> v2:172.21.15.188:3300/0 conn(0x7ffac40ccb20 0x7ffac40b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffabbfff640 1 -- 172.21.15.111:0/680841007 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac40d0f10 con 0x7ffac40d07b0 2024-09-13T17:57:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffabbfff640 1 -- 172.21.15.111:0/680841007 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7ffab4001670 con 0x7ffac40d07b0 2024-09-13T17:57:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffabbfff640 1 -- 172.21.15.111:0/680841007 >> v1:172.21.15.111:6789/0 conn(0x7ffac40d07b0 legacy=0x7ffac4160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffabbfff640 1 --2- 172.21.15.111:0/680841007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac415c6e0 0x7ffab0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffabbfff640 1 --2- 172.21.15.111:0/680841007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffab0003ef0 0x7ffab00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffabbfff640 1 --2- 172.21.15.111:0/680841007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab0004d00 0x7ffab0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffabbfff640 1 -- 172.21.15.111:0/680841007 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac406bd20 con 0x7ffab0003ef0 2024-09-13T17:57:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffabbfff640 1 -- 172.21.15.111:0/680841007 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac40d0c30 con 0x7ffab0004d00 2024-09-13T17:57:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffabbfff640 1 -- 172.21.15.111:0/680841007 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab000b9d0 con 0x7ffac415c6e0 2024-09-13T17:57:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.272+0000 7ffac8d1e640 1 --2- 172.21.15.111:0/680841007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffab0003ef0 0x7ffab00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.273+0000 7ffac9d20640 1 --2- 172.21.15.111:0/680841007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab0004d00 0x7ffab0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.273+0000 7ffac951f640 1 --2- 172.21.15.111:0/680841007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac415c6e0 0x7ffab0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.273+0000 7ffac8d1e640 1 -- 172.21.15.111:0/680841007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac415c6e0 msgr2=0x7ffab0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.273+0000 7ffac8d1e640 1 --2- 172.21.15.111:0/680841007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac415c6e0 0x7ffab0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.273+0000 7ffac8d1e640 1 -- 172.21.15.111:0/680841007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab0004d00 msgr2=0x7ffab0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.273+0000 7ffac8d1e640 1 --2- 172.21.15.111:0/680841007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab0004d00 0x7ffab0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.273+0000 7ffac8d1e640 1 -- 172.21.15.111:0/680841007 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab0003520 con 0x7ffab0003ef0 2024-09-13T17:57:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.273+0000 7ffac8d1e640 1 --2- 172.21.15.111:0/680841007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffab0003ef0 0x7ffab00047c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffac40b1d80 tx=0x7ffab00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6103c4e8f8db9076 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.274+0000 7ffabbfff640 1 -- 172.21.15.111:0/680841007 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4005ad0 con 0x7ffab0003ef0 2024-09-13T17:57:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.274+0000 7ffabbfff640 1 -- 172.21.15.111:0/680841007 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab402d070 con 0x7ffab0003ef0 2024-09-13T17:57:52.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.274+0000 7ffabbfff640 1 -- 172.21.15.111:0/680841007 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4005e70 con 0x7ffab0003ef0 2024-09-13T17:57:52.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.274+0000 7ffacb783640 1 -- 172.21.15.111:0/680841007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffab0003ef0 msgr2=0x7ffab00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.274+0000 7ffacb783640 1 --2- 172.21.15.111:0/680841007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffab0003ef0 0x7ffab00047c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffac40b1d80 tx=0x7ffab00057f0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.274+0000 7ffac9d20640 1 -- 172.21.15.111:0/680841007 reap_dead start 2024-09-13T17:57:52.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.274+0000 7ffacb783640 1 -- 172.21.15.111:0/680841007 shutdown_connections 2024-09-13T17:57:52.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.274+0000 7ffacb783640 1 -- 172.21.15.111:0/680841007 >> 172.21.15.111:0/680841007 conn(0x7ffac4089f60 msgr2=0x7ffac40d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:52.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.274+0000 7ffacb783640 1 -- 172.21.15.111:0/680841007 shutdown_connections 2024-09-13T17:57:52.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.274+0000 7ffacb783640 1 -- 172.21.15.111:0/680841007 wait complete. 2024-09-13T17:57:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.275+0000 7ffacb783640 1 Processor -- start 2024-09-13T17:57:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.276+0000 7ffacb783640 1 -- start start 2024-09-13T17:57:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.276+0000 7ffacb783640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab0003ef0 0x7ffac4155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.276+0000 7ffacb783640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40b5440 0x7ffac40b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.276+0000 7ffacb783640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac415e240 0x7ffac415e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.276+0000 7ffacb783640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac415de20 con 0x7ffac40b5440 2024-09-13T17:57:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.276+0000 7ffacb783640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac40cbfe0 con 0x7ffac415e240 2024-09-13T17:57:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.276+0000 7ffacb783640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac40b5e00 con 0x7ffab0003ef0 2024-09-13T17:57:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.276+0000 7ffac8d1e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40b5440 0x7ffac40b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.276+0000 7ffac8d1e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40b5440 0x7ffac40b58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53126/0 (socket says 172.21.15.111:53126) 2024-09-13T17:57:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.276+0000 7ffac8d1e640 1 -- 172.21.15.111:0/2676204593 learned_addr learned my addr 172.21.15.111:0/2676204593 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.276+0000 7ffac951f640 1 --2- 172.21.15.111:0/2676204593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab0003ef0 0x7ffac4155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.276+0000 7ffac9d20640 1 --2- 172.21.15.111:0/2676204593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac415e240 0x7ffac415e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.277+0000 7ffac8d1e640 1 -- 172.21.15.111:0/2676204593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab0003ef0 msgr2=0x7ffac4155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.277+0000 7ffac8d1e640 1 --2- 172.21.15.111:0/2676204593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab0003ef0 0x7ffac4155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.277+0000 7ffac8d1e640 1 -- 172.21.15.111:0/2676204593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac415e240 msgr2=0x7ffac415e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.277+0000 7ffac8d1e640 1 --2- 172.21.15.111:0/2676204593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac415e240 0x7ffac415e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.277+0000 7ffac8d1e640 1 -- 172.21.15.111:0/2676204593 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac4156000 con 0x7ffac40b5440 2024-09-13T17:57:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.277+0000 7ffac8d1e640 1 --2- 172.21.15.111:0/2676204593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40b5440 0x7ffac40b58c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffab400c350 tx=0x7ffab4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.277+0000 7ffab9ffb640 1 -- 172.21.15.111:0/2676204593 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4042030 con 0x7ffac40b5440 2024-09-13T17:57:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.277+0000 7ffab9ffb640 1 -- 172.21.15.111:0/2676204593 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab4005e50 con 0x7ffac40b5440 2024-09-13T17:57:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.277+0000 7ffab9ffb640 1 -- 172.21.15.111:0/2676204593 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4004bf0 con 0x7ffac40b5440 2024-09-13T17:57:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.277+0000 7ffacb783640 1 -- 172.21.15.111:0/2676204593 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac415d480 con 0x7ffac40b5440 2024-09-13T17:57:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.277+0000 7ffacb783640 1 -- 172.21.15.111:0/2676204593 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac415da40 con 0x7ffac40b5440 2024-09-13T17:57:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.278+0000 7ffacb783640 1 -- 172.21.15.111:0/2676204593 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac415dc00 con 0x7ffac40b5440 2024-09-13T17:57:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.278+0000 7ffab9ffb640 1 -- 172.21.15.111:0/2676204593 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ffab4045070 con 0x7ffac40b5440 2024-09-13T17:57:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.279+0000 7ffab9ffb640 1 --2- 172.21.15.111:0/2676204593 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffaa806ac30 0x7ffaa806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.279+0000 7ffab9ffb640 1 -- 172.21.15.111:0/2676204593 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ffab4043070 con 0x7ffac40b5440 2024-09-13T17:57:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.279+0000 7ffac951f640 1 --2- 172.21.15.111:0/2676204593 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffaa806ac30 0x7ffaa806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.279+0000 7ffac951f640 1 --2- 172.21.15.111:0/2676204593 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffaa806ac30 0x7ffaa806d0f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffac0014430 tx=0x7ffac000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.283+0000 7ffab9ffb640 1 -- 172.21.15.111:0/2676204593 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ffab4074260 con 0x7ffac40b5440 2024-09-13T17:57:52.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.446+0000 7ffacb783640 1 -- 172.21.15.111:0/2676204593 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffac415dc00 con 0x7ffac40b5440 2024-09-13T17:57:52.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.447+0000 7ffab9ffb640 1 -- 172.21.15.111:0/2676204593 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7ffab4077880 con 0x7ffac40b5440 2024-09-13T17:57:52.450 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:52.450 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:30.065235+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:52.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.449+0000 7ffacb783640 1 -- 172.21.15.111:0/2676204593 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffaa806ac30 msgr2=0x7ffaa806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.449+0000 7ffacb783640 1 --2- 172.21.15.111:0/2676204593 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffaa806ac30 0x7ffaa806d0f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffac0014430 tx=0x7ffac000b040 comp rx=0 tx=0).stop 2024-09-13T17:57:52.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.449+0000 7ffacb783640 1 -- 172.21.15.111:0/2676204593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40b5440 msgr2=0x7ffac40b58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.449+0000 7ffacb783640 1 --2- 172.21.15.111:0/2676204593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40b5440 0x7ffac40b58c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffab400c350 tx=0x7ffab4001a00 comp rx=0 tx=0).stop 2024-09-13T17:57:52.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.449+0000 7ffacb783640 1 -- 172.21.15.111:0/2676204593 shutdown_connections 2024-09-13T17:57:52.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.449+0000 7ffacb783640 1 --2- 172.21.15.111:0/2676204593 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffaa806ac30 0x7ffaa806d0f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.449+0000 7ffacb783640 1 --2- 172.21.15.111:0/2676204593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac415e240 0x7ffac415e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.449+0000 7ffacb783640 1 --2- 172.21.15.111:0/2676204593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40b5440 0x7ffac40b58c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.449+0000 7ffacb783640 1 --2- 172.21.15.111:0/2676204593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab0003ef0 0x7ffac4155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.449+0000 7ffacb783640 1 -- 172.21.15.111:0/2676204593 >> 172.21.15.111:0/2676204593 conn(0x7ffac4089f60 msgr2=0x7ffac40b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:52.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.449+0000 7ffacb783640 1 -- 172.21.15.111:0/2676204593 shutdown_connections 2024-09-13T17:57:52.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.449+0000 7ffacb783640 1 -- 172.21.15.111:0/2676204593 wait complete. 2024-09-13T17:57:52.463 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:57:52.463 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:57:52.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.561+0000 7f20c34b1640 1 Processor -- start 2024-09-13T17:57:52.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.561+0000 7f20c34b1640 1 -- start start 2024-09-13T17:57:52.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.562+0000 7f20c34b1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f20bc15fbc0 0x7f20bc0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.562+0000 7f20c34b1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f20bc0d29e0 0x7f20bc06bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.562+0000 7f20c34b1640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20bc06b6c0 con 0x7f20bc0cd6c0 2024-09-13T17:57:52.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.562+0000 7f20c34b1640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20bc0ce500 con 0x7f20bc15fbc0 2024-09-13T17:57:52.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.562+0000 7f20c34b1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20bc0cdc60 con 0x7f20bc0d29e0 2024-09-13T17:57:52.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.562+0000 7f20c124d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f20bc15fbc0 0x7f20bc0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.562+0000 7f20c0a4c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f20bc0cd6c0 0x7f20bc0d22d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35578/0 (socket says 172.21.15.111:35578) 2024-09-13T17:57:52.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.562+0000 7f20c0a4c640 1 -- 172.21.15.111:0/2509698799 learned_addr learned my addr 172.21.15.111:0/2509698799 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:52.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.562+0000 7f20c124d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f20bc15fbc0 0x7f20bc0cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53128/0 (socket says 172.21.15.111:53128) 2024-09-13T17:57:52.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.562+0000 7f20c1a4e640 1 --2- 172.21.15.111:0/2509698799 >> v2:172.21.15.188:3300/0 conn(0x7f20bc0d29e0 0x7f20bc06bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.562+0000 7f20c124d640 1 -- 172.21.15.111:0/2509698799 >> v2:172.21.15.188:3300/0 conn(0x7f20bc0d29e0 msgr2=0x7f20bc06bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:52.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.562+0000 7f20c124d640 1 --2- 172.21.15.111:0/2509698799 >> v2:172.21.15.188:3300/0 conn(0x7f20bc0d29e0 0x7f20bc06bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.562+0000 7f20c124d640 1 -- 172.21.15.111:0/2509698799 >> v1:172.21.15.111:6789/0 conn(0x7f20bc0cd6c0 legacy=0x7f20bc0d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.562+0000 7f20c124d640 1 -- 172.21.15.111:0/2509698799 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20bc0cdf40 con 0x7f20bc15fbc0 2024-09-13T17:57:52.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.562+0000 7f20b3fff640 1 -- 172.21.15.111:0/2509698799 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604061482 0 0) 0x7f20bc06b6c0 con 0x7f20bc0cd6c0 2024-09-13T17:57:52.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.563+0000 7f20c124d640 1 --2- 172.21.15.111:0/2509698799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20bc15fbc0 0x7f20bc0cd180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0ceb10 tx=0x7f20b802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547f727ca46e2ac2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:52.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.563+0000 7f20b3fff640 1 -- 172.21.15.111:0/2509698799 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8032440 con 0x7f20bc15fbc0 2024-09-13T17:57:52.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.563+0000 7f20b3fff640 1 -- 172.21.15.111:0/2509698799 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b80325a0 con 0x7f20bc15fbc0 2024-09-13T17:57:52.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.563+0000 7f20b3fff640 1 -- 172.21.15.111:0/2509698799 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8032930 con 0x7f20bc15fbc0 2024-09-13T17:57:52.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.563+0000 7f20c34b1640 1 -- 172.21.15.111:0/2509698799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20bc15fbc0 msgr2=0x7f20bc0cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.563+0000 7f20c34b1640 1 --2- 172.21.15.111:0/2509698799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20bc15fbc0 0x7f20bc0cd180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0ceb10 tx=0x7f20b802a1a0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.563+0000 7f20c34b1640 1 -- 172.21.15.111:0/2509698799 shutdown_connections 2024-09-13T17:57:52.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.563+0000 7f20c34b1640 1 --2- 172.21.15.111:0/2509698799 >> v2:172.21.15.188:3300/0 conn(0x7f20bc0d29e0 0x7f20bc06bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.563+0000 7f20c34b1640 1 --2- 172.21.15.111:0/2509698799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20bc15fbc0 0x7f20bc0cd180 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.563+0000 7f20c34b1640 1 -- 172.21.15.111:0/2509698799 >> 172.21.15.111:0/2509698799 conn(0x7f20bc09b830 msgr2=0x7f20bc089f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:52.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.564+0000 7f20c34b1640 1 -- 172.21.15.111:0/2509698799 shutdown_connections 2024-09-13T17:57:52.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.564+0000 7f20c34b1640 1 -- 172.21.15.111:0/2509698799 wait complete. 2024-09-13T17:57:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.565+0000 7f20c34b1640 1 Processor -- start 2024-09-13T17:57:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.565+0000 7f20c34b1640 1 -- start start 2024-09-13T17:57:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.565+0000 7f20c34b1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20bc0cd6c0 0x7f20bc1f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.565+0000 7f20c34b1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20bc0d29e0 0x7f20bc1f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.565+0000 7f20c34b1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20bc15fbc0 0x7f20bc1f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.565+0000 7f20c34b1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc0b37b0 con 0x7f20bc15fbc0 2024-09-13T17:57:52.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.565+0000 7f20c34b1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc15c8c0 con 0x7f20bc0d29e0 2024-09-13T17:57:52.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.565+0000 7f20c34b1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc15c5e0 con 0x7f20bc0cd6c0 2024-09-13T17:57:52.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.566+0000 7f20c1a4e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20bc15fbc0 0x7f20bc1f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.566+0000 7f20c1a4e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20bc15fbc0 0x7f20bc1f2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53142/0 (socket says 172.21.15.111:53142) 2024-09-13T17:57:52.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.566+0000 7f20c1a4e640 1 -- 172.21.15.111:0/2260029881 learned_addr learned my addr 172.21.15.111:0/2260029881 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:52.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.566+0000 7f20c0a4c640 1 --2- 172.21.15.111:0/2260029881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20bc0d29e0 0x7f20bc1f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.566+0000 7f20c124d640 1 --2- 172.21.15.111:0/2260029881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20bc0cd6c0 0x7f20bc1f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.566+0000 7f20c1a4e640 1 -- 172.21.15.111:0/2260029881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20bc0cd6c0 msgr2=0x7f20bc1f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.566+0000 7f20c1a4e640 1 --2- 172.21.15.111:0/2260029881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20bc0cd6c0 0x7f20bc1f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.566+0000 7f20c1a4e640 1 -- 172.21.15.111:0/2260029881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20bc0d29e0 msgr2=0x7f20bc1f31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.566+0000 7f20c1a4e640 1 --2- 172.21.15.111:0/2260029881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20bc0d29e0 0x7f20bc1f31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.566+0000 7f20c1a4e640 1 -- 172.21.15.111:0/2260029881 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b8009400 con 0x7f20bc15fbc0 2024-09-13T17:57:52.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.566+0000 7f20c124d640 1 --2- 172.21.15.111:0/2260029881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20bc0cd6c0 0x7f20bc1f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:52.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.566+0000 7f20c1a4e640 1 --2- 172.21.15.111:0/2260029881 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20bc15fbc0 0x7f20bc1f2b40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f20b4007e40 tx=0x7f20b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:52.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.566+0000 7f20b1ffb640 1 -- 172.21.15.111:0/2260029881 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4014620 con 0x7f20bc15fbc0 2024-09-13T17:57:52.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.566+0000 7f20b1ffb640 1 -- 172.21.15.111:0/2260029881 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b400ee90 con 0x7f20bc15fbc0 2024-09-13T17:57:52.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.566+0000 7f20c34b1640 1 -- 172.21.15.111:0/2260029881 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20bc0b30f0 con 0x7f20bc15fbc0 2024-09-13T17:57:52.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.566+0000 7f20b1ffb640 1 -- 172.21.15.111:0/2260029881 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b40149e0 con 0x7f20bc15fbc0 2024-09-13T17:57:52.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.566+0000 7f20c34b1640 1 -- 172.21.15.111:0/2260029881 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20bc15be90 con 0x7f20bc15fbc0 2024-09-13T17:57:52.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.567+0000 7f20c34b1640 1 -- 172.21.15.111:0/2260029881 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20bc0b2f50 con 0x7f20bc15fbc0 2024-09-13T17:57:52.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.568+0000 7f20b1ffb640 1 -- 172.21.15.111:0/2260029881 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f20b4014b40 con 0x7f20bc15fbc0 2024-09-13T17:57:52.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.568+0000 7f20b1ffb640 1 --2- 172.21.15.111:0/2260029881 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f20a006ac00 0x7f20a006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.568+0000 7f20b1ffb640 1 -- 172.21.15.111:0/2260029881 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f20b4089f20 con 0x7f20bc15fbc0 2024-09-13T17:57:52.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.568+0000 7f20c124d640 1 --2- 172.21.15.111:0/2260029881 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f20a006ac00 0x7f20a006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.569+0000 7f20c124d640 1 --2- 172.21.15.111:0/2260029881 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f20a006ac00 0x7f20a006d0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f20b8034000 tx=0x7f20b802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:52.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.572+0000 7f20b1ffb640 1 -- 172.21.15.111:0/2260029881 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f20bc0b2f50 con 0x7f20bc15fbc0 2024-09-13T17:57:52.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.732+0000 7f20c34b1640 1 -- 172.21.15.111:0/2260029881 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f20bc0b2f50 con 0x7f20bc15fbc0 2024-09-13T17:57:52.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.732+0000 7f20b1ffb640 1 -- 172.21.15.111:0/2260029881 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f20bc0b2f50 con 0x7f20bc15fbc0 2024-09-13T17:57:52.735 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:57:52.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.734+0000 7f20c34b1640 1 -- 172.21.15.111:0/2260029881 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f20a006ac00 msgr2=0x7f20a006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.734+0000 7f20c34b1640 1 --2- 172.21.15.111:0/2260029881 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f20a006ac00 0x7f20a006d0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f20b8034000 tx=0x7f20b802e000 comp rx=0 tx=0).stop 2024-09-13T17:57:52.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.734+0000 7f20c34b1640 1 -- 172.21.15.111:0/2260029881 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20bc15fbc0 msgr2=0x7f20bc1f2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.734+0000 7f20c34b1640 1 --2- 172.21.15.111:0/2260029881 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20bc15fbc0 0x7f20bc1f2b40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f20b4007e40 tx=0x7f20b400e4b0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.734+0000 7f20c34b1640 1 -- 172.21.15.111:0/2260029881 shutdown_connections 2024-09-13T17:57:52.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.734+0000 7f20c34b1640 1 --2- 172.21.15.111:0/2260029881 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20bc15fbc0 0x7f20bc1f2b40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.734+0000 7f20c34b1640 1 --2- 172.21.15.111:0/2260029881 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f20a006ac00 0x7f20a006d0c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.734+0000 7f20c34b1640 1 --2- 172.21.15.111:0/2260029881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20bc0d29e0 0x7f20bc1f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.734+0000 7f20c34b1640 1 --2- 172.21.15.111:0/2260029881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20bc0cd6c0 0x7f20bc1f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.734+0000 7f20c34b1640 1 -- 172.21.15.111:0/2260029881 >> 172.21.15.111:0/2260029881 conn(0x7f20bc09b830 msgr2=0x7f20bc0cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:52.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.734+0000 7f20c34b1640 1 -- 172.21.15.111:0/2260029881 shutdown_connections 2024-09-13T17:57:52.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.734+0000 7f20c34b1640 1 -- 172.21.15.111:0/2260029881 wait complete. 2024-09-13T17:57:52.748 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:57:52.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.844+0000 7ffb5bd91640 1 Processor -- start 2024-09-13T17:57:52.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.844+0000 7ffb5bd91640 1 -- start start 2024-09-13T17:57:52.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.845+0000 7ffb5bd91640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb540b2b40 0x7ffb540ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.845+0000 7ffb5bd91640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb540ce4f0 0x7ffb540ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.845+0000 7ffb5bd91640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb540ccbe0 con 0x7ffb54159310 2024-09-13T17:57:52.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.845+0000 7ffb5bd91640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb540ad940 con 0x7ffb540ce4f0 2024-09-13T17:57:52.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.845+0000 7ffb5bd91640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb540ceea0 con 0x7ffb540b2b40 2024-09-13T17:57:52.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.845+0000 7ffb59b2d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ffb54159310 0x7ffb540ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35584/0 (socket says 172.21.15.111:35584) 2024-09-13T17:57:52.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.845+0000 7ffb59b2d640 1 -- 172.21.15.111:0/3648095007 learned_addr learned my addr 172.21.15.111:0/3648095007 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:52.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.845+0000 7ffb5932c640 1 --2- 172.21.15.111:0/3648095007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb540b2b40 0x7ffb540ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.845+0000 7ffb5a32e640 1 --2- 172.21.15.111:0/3648095007 >> v2:172.21.15.188:3300/0 conn(0x7ffb540ce4f0 0x7ffb540ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb58b2b640 1 -- 172.21.15.111:0/3648095007 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 264399128 0 0) 0x7ffb540ccbe0 con 0x7ffb54159310 2024-09-13T17:57:52.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb58b2b640 1 -- 172.21.15.111:0/3648095007 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb40003520 con 0x7ffb54159310 2024-09-13T17:57:52.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb58b2b640 1 -- 172.21.15.111:0/3648095007 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1950465173 0 0) 0x7ffb40003520 con 0x7ffb54159310 2024-09-13T17:57:52.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb58b2b640 1 -- 172.21.15.111:0/3648095007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb540b2b40 msgr2=0x7ffb540ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:52.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb58b2b640 1 --2- 172.21.15.111:0/3648095007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb540b2b40 0x7ffb540ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb58b2b640 1 -- 172.21.15.111:0/3648095007 >> v2:172.21.15.188:3300/0 conn(0x7ffb540ce4f0 msgr2=0x7ffb540ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:52.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb58b2b640 1 --2- 172.21.15.111:0/3648095007 >> v2:172.21.15.188:3300/0 conn(0x7ffb540ce4f0 0x7ffb540ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb58b2b640 1 -- 172.21.15.111:0/3648095007 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb540b1810 con 0x7ffb54159310 2024-09-13T17:57:52.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb58b2b640 1 -- 172.21.15.111:0/3648095007 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7ffb500019f0 con 0x7ffb54159310 2024-09-13T17:57:52.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb5a32e640 1 --2- 172.21.15.111:0/3648095007 >> v2:172.21.15.188:3300/0 conn(0x7ffb540ce4f0 0x7ffb540ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:52.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb58b2b640 1 -- 172.21.15.111:0/3648095007 >> v1:172.21.15.111:6789/0 conn(0x7ffb54159310 legacy=0x7ffb540ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb58b2b640 1 --2- 172.21.15.111:0/3648095007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400039a0 0x7ffb40003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb58b2b640 1 --2- 172.21.15.111:0/3648095007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb400047c0 0x7ffb40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb58b2b640 1 --2- 172.21.15.111:0/3648095007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb40005180 0x7ffb4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb58b2b640 1 -- 172.21.15.111:0/3648095007 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb540ceea0 con 0x7ffb40005180 2024-09-13T17:57:52.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb58b2b640 1 -- 172.21.15.111:0/3648095007 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4000bf70 con 0x7ffb400047c0 2024-09-13T17:57:52.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb58b2b640 1 -- 172.21.15.111:0/3648095007 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4000c100 con 0x7ffb400039a0 2024-09-13T17:57:52.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb5932c640 1 --2- 172.21.15.111:0/3648095007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb540b2b40 0x7ffb540ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:52.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb5a32e640 1 --2- 172.21.15.111:0/3648095007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb40005180 0x7ffb4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb59b2d640 1 --2- 172.21.15.111:0/3648095007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400039a0 0x7ffb40003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb5932c640 1 --2- 172.21.15.111:0/3648095007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb400047c0 0x7ffb40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb5a32e640 1 -- 172.21.15.111:0/3648095007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400039a0 msgr2=0x7ffb40003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:52.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb5a32e640 1 --2- 172.21.15.111:0/3648095007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400039a0 0x7ffb40003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb5a32e640 1 -- 172.21.15.111:0/3648095007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb400047c0 msgr2=0x7ffb40004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:52.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb5a32e640 1 --2- 172.21.15.111:0/3648095007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb400047c0 0x7ffb40004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb5a32e640 1 -- 172.21.15.111:0/3648095007 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb40003520 con 0x7ffb40005180 2024-09-13T17:57:52.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.846+0000 7ffb5932c640 1 --2- 172.21.15.111:0/3648095007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb400047c0 0x7ffb40004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:52.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.847+0000 7ffb5a32e640 1 --2- 172.21.15.111:0/3648095007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb40005180 0x7ffb4000b980 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c009c90 tx=0x7ffb400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9e3ec09ddd5610 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:52.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.847+0000 7ffb58b2b640 1 -- 172.21.15.111:0/3648095007 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c031400 con 0x7ffb40005180 2024-09-13T17:57:52.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.847+0000 7ffb58b2b640 1 -- 172.21.15.111:0/3648095007 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4c02d050 con 0x7ffb40005180 2024-09-13T17:57:52.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.847+0000 7ffb58b2b640 1 -- 172.21.15.111:0/3648095007 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c031760 con 0x7ffb40005180 2024-09-13T17:57:52.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.848+0000 7ffb5bd91640 1 -- 172.21.15.111:0/3648095007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb40005180 msgr2=0x7ffb4000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.848+0000 7ffb5bd91640 1 --2- 172.21.15.111:0/3648095007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb40005180 0x7ffb4000b980 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c009c90 tx=0x7ffb400057f0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.848+0000 7ffb5a32e640 1 -- 172.21.15.111:0/3648095007 reap_dead start 2024-09-13T17:57:52.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.848+0000 7ffb5bd91640 1 -- 172.21.15.111:0/3648095007 shutdown_connections 2024-09-13T17:57:52.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.848+0000 7ffb5bd91640 1 -- 172.21.15.111:0/3648095007 >> 172.21.15.111:0/3648095007 conn(0x7ffb5406a0b0 msgr2=0x7ffb5406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:52.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.848+0000 7ffb5bd91640 1 -- 172.21.15.111:0/3648095007 shutdown_connections 2024-09-13T17:57:52.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.848+0000 7ffb5bd91640 1 -- 172.21.15.111:0/3648095007 wait complete. 2024-09-13T17:57:52.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.849+0000 7ffb5bd91640 1 Processor -- start 2024-09-13T17:57:52.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.849+0000 7ffb5bd91640 1 -- start start 2024-09-13T17:57:52.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5bd91640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb40005180 0x7ffb5408b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5bd91640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb54098c20 0x7ffb5409ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5bd91640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb541574a0 0x7ffb541f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5bd91640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb54099cc0 con 0x7ffb54098c20 2024-09-13T17:57:52.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5bd91640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb540990b0 con 0x7ffb40005180 2024-09-13T17:57:52.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5bd91640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb54099390 con 0x7ffb541574a0 2024-09-13T17:57:52.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5932c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb54098c20 0x7ffb5409ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5932c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb54098c20 0x7ffb5409ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53158/0 (socket says 172.21.15.111:53158) 2024-09-13T17:57:52.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5932c640 1 -- 172.21.15.111:0/4022429604 learned_addr learned my addr 172.21.15.111:0/4022429604 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:52.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb59b2d640 1 --2- 172.21.15.111:0/4022429604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb40005180 0x7ffb5408b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5a32e640 1 --2- 172.21.15.111:0/4022429604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb541574a0 0x7ffb541f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5932c640 1 -- 172.21.15.111:0/4022429604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb541574a0 msgr2=0x7ffb541f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5932c640 1 --2- 172.21.15.111:0/4022429604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb541574a0 0x7ffb541f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5932c640 1 -- 172.21.15.111:0/4022429604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb40005180 msgr2=0x7ffb5408b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:52.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5932c640 1 --2- 172.21.15.111:0/4022429604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb40005180 0x7ffb5408b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:52.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5932c640 1 -- 172.21.15.111:0/4022429604 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb54099670 con 0x7ffb54098c20 2024-09-13T17:57:52.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5a32e640 1 --2- 172.21.15.111:0/4022429604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb541574a0 0x7ffb541f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:52.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5932c640 1 --2- 172.21.15.111:0/4022429604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb54098c20 0x7ffb5409ba90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffb44011880 tx=0x7ffb44011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:52.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb4a7fc640 1 -- 172.21.15.111:0/4022429604 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4400fb50 con 0x7ffb54098c20 2024-09-13T17:57:52.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb4a7fc640 1 -- 172.21.15.111:0/4022429604 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4400fcb0 con 0x7ffb54098c20 2024-09-13T17:57:52.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5bd91640 1 -- 172.21.15.111:0/4022429604 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb54099950 con 0x7ffb54098c20 2024-09-13T17:57:52.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb4a7fc640 1 -- 172.21.15.111:0/4022429604 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb44011e70 con 0x7ffb54098c20 2024-09-13T17:57:52.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.850+0000 7ffb5bd91640 1 -- 172.21.15.111:0/4022429604 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5409bfd0 con 0x7ffb54098c20 2024-09-13T17:57:52.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.851+0000 7ffb5bd91640 1 -- 172.21.15.111:0/4022429604 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb54099ae0 con 0x7ffb54098c20 2024-09-13T17:57:52.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.852+0000 7ffb4a7fc640 1 -- 172.21.15.111:0/4022429604 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ffb44015460 con 0x7ffb54098c20 2024-09-13T17:57:52.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.852+0000 7ffb4a7fc640 1 --2- 172.21.15.111:0/4022429604 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb3806ac00 0x7ffb3806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:52.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.852+0000 7ffb59b2d640 1 --2- 172.21.15.111:0/4022429604 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb3806ac00 0x7ffb3806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:52.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.852+0000 7ffb4a7fc640 1 -- 172.21.15.111:0/4022429604 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ffb44089f70 con 0x7ffb54098c20 2024-09-13T17:57:52.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.853+0000 7ffb59b2d640 1 --2- 172.21.15.111:0/4022429604 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb3806ac00 0x7ffb3806d0c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffb540ae550 tx=0x7ffb50006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:52.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:52.857+0000 7ffb4a7fc640 1 -- 172.21.15.111:0/4022429604 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ffb440bd9c0 con 0x7ffb54098c20 2024-09-13T17:57:53.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.016+0000 7ffb5bd91640 1 -- 172.21.15.111:0/4022429604 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffb540cd2e0 con 0x7ffb54098c20 2024-09-13T17:57:53.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.017+0000 7ffb4a7fc640 1 -- 172.21.15.111:0/4022429604 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffb440588f0 con 0x7ffb54098c20 2024-09-13T17:57:53.020 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:57:53.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.018+0000 7ffb5bd91640 1 -- 172.21.15.111:0/4022429604 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb3806ac00 msgr2=0x7ffb3806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.018+0000 7ffb5bd91640 1 --2- 172.21.15.111:0/4022429604 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb3806ac00 0x7ffb3806d0c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffb540ae550 tx=0x7ffb50006040 comp rx=0 tx=0).stop 2024-09-13T17:57:53.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.019+0000 7ffb5bd91640 1 -- 172.21.15.111:0/4022429604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb54098c20 msgr2=0x7ffb5409ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.019+0000 7ffb5bd91640 1 --2- 172.21.15.111:0/4022429604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb54098c20 0x7ffb5409ba90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffb44011880 tx=0x7ffb44011d40 comp rx=0 tx=0).stop 2024-09-13T17:57:53.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.019+0000 7ffb5bd91640 1 -- 172.21.15.111:0/4022429604 shutdown_connections 2024-09-13T17:57:53.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.019+0000 7ffb5bd91640 1 --2- 172.21.15.111:0/4022429604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb541574a0 0x7ffb541f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.019+0000 7ffb5bd91640 1 --2- 172.21.15.111:0/4022429604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb54098c20 0x7ffb5409ba90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.019+0000 7ffb5bd91640 1 --2- 172.21.15.111:0/4022429604 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb3806ac00 0x7ffb3806d0c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.019+0000 7ffb5bd91640 1 --2- 172.21.15.111:0/4022429604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb40005180 0x7ffb5408b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.019+0000 7ffb5bd91640 1 -- 172.21.15.111:0/4022429604 >> 172.21.15.111:0/4022429604 conn(0x7ffb5406a0b0 msgr2=0x7ffb540b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:53.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.019+0000 7ffb5bd91640 1 -- 172.21.15.111:0/4022429604 shutdown_connections 2024-09-13T17:57:53.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.019+0000 7ffb5bd91640 1 -- 172.21.15.111:0/4022429604 wait complete. 2024-09-13T17:57:53.032 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:57:53.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.130+0000 7f41489b5640 1 Processor -- start 2024-09-13T17:57:53.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.130+0000 7f41489b5640 1 -- start start 2024-09-13T17:57:53.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.131+0000 7f41489b5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f41440b20a0 0x7f41440ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.131+0000 7f41489b5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41440cee60 con 0x7f414415a3b0 2024-09-13T17:57:53.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.131+0000 7f41489b5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f414415a790 con 0x7f41440cea30 2024-09-13T17:57:53.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.131+0000 7f41489b5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41440b2680 con 0x7f41440b20a0 2024-09-13T17:57:53.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.131+0000 7f4139d9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f41440b20a0 0x7f41440ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.131+0000 7f414259c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f414415a3b0 0x7f41440b1720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35590/0 (socket says 172.21.15.111:35590) 2024-09-13T17:57:53.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.131+0000 7f414259c640 1 -- 172.21.15.111:0/3583097424 learned_addr learned my addr 172.21.15.111:0/3583097424 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:53.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.131+0000 7f4139d9b640 1 -- 172.21.15.111:0/3583097424 >> v1:172.21.15.188:6789/0 conn(0x7f41440cea30 legacy=0x7f41440ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.131+0000 7f4139d9b640 1 -- 172.21.15.111:0/3583097424 >> v1:172.21.15.111:6789/0 conn(0x7f414415a3b0 legacy=0x7f41440b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.131+0000 7f4139d9b640 1 -- 172.21.15.111:0/3583097424 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41440b2960 con 0x7f41440b20a0 2024-09-13T17:57:53.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.131+0000 7f4141d9b640 1 -- 172.21.15.111:0/3583097424 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849217398 0 0) 0x7f41440cee60 con 0x7f414415a3b0 2024-09-13T17:57:53.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.132+0000 7f4139d9b640 1 --2- 172.21.15.111:0/3583097424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41440b20a0 0x7f41440ce4f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f412c0097a0 tx=0x7f412c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232aacc100bd5a29 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:53.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.132+0000 7f4141d9b640 1 -- 172.21.15.111:0/3583097424 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c032440 con 0x7f41440b20a0 2024-09-13T17:57:53.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.132+0000 7f4141d9b640 1 -- 172.21.15.111:0/3583097424 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412c0325a0 con 0x7f41440b20a0 2024-09-13T17:57:53.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.132+0000 7f4141d9b640 1 -- 172.21.15.111:0/3583097424 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c032930 con 0x7f41440b20a0 2024-09-13T17:57:53.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.132+0000 7f41489b5640 1 -- 172.21.15.111:0/3583097424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41440b20a0 msgr2=0x7f41440ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.132+0000 7f41489b5640 1 --2- 172.21.15.111:0/3583097424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41440b20a0 0x7f41440ce4f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f412c0097a0 tx=0x7f412c02a230 comp rx=0 tx=0).stop 2024-09-13T17:57:53.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.133+0000 7f41489b5640 1 -- 172.21.15.111:0/3583097424 shutdown_connections 2024-09-13T17:57:53.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.133+0000 7f41489b5640 1 --2- 172.21.15.111:0/3583097424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41440b20a0 0x7f41440ce4f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.133+0000 7f41489b5640 1 -- 172.21.15.111:0/3583097424 >> 172.21.15.111:0/3583097424 conn(0x7f414406a0b0 msgr2=0x7f414406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:53.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.133+0000 7f41489b5640 1 -- 172.21.15.111:0/3583097424 shutdown_connections 2024-09-13T17:57:53.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.133+0000 7f41489b5640 1 -- 172.21.15.111:0/3583097424 wait complete. 2024-09-13T17:57:53.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.134+0000 7f41489b5640 1 Processor -- start 2024-09-13T17:57:53.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.134+0000 7f41489b5640 1 -- start start 2024-09-13T17:57:53.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.135+0000 7f41489b5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41440b20a0 0x7f414415ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.135+0000 7f41489b5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41440cea30 0x7f414415b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.135+0000 7f41489b5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f414415a3b0 0x7f41441eec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.135+0000 7f41489b5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414415bc30 con 0x7f414415a3b0 2024-09-13T17:57:53.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.135+0000 7f41489b5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414415bf10 con 0x7f41440b20a0 2024-09-13T17:57:53.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.135+0000 7f41489b5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414415c1f0 con 0x7f41440cea30 2024-09-13T17:57:53.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.135+0000 7f4142d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f414415a3b0 0x7f41441eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.135+0000 7f4142d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f414415a3b0 0x7f41441eec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53170/0 (socket says 172.21.15.111:53170) 2024-09-13T17:57:53.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.135+0000 7f4139d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41440cea30 0x7f414415b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.135+0000 7f414259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41440b20a0 0x7f414415ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.135+0000 7f4142d9d640 1 -- 172.21.15.111:0/2686282188 learned_addr learned my addr 172.21.15.111:0/2686282188 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:53.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.135+0000 7f4142d9d640 1 -- 172.21.15.111:0/2686282188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41440cea30 msgr2=0x7f414415b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.135+0000 7f4142d9d640 1 --2- 172.21.15.111:0/2686282188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41440cea30 0x7f414415b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.135+0000 7f4142d9d640 1 -- 172.21.15.111:0/2686282188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41440b20a0 msgr2=0x7f414415ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.135+0000 7f4142d9d640 1 --2- 172.21.15.111:0/2686282188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41440b20a0 0x7f414415ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.135+0000 7f4142d9d640 1 -- 172.21.15.111:0/2686282188 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c009400 con 0x7f414415a3b0 2024-09-13T17:57:53.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.135+0000 7f4142d9d640 1 --2- 172.21.15.111:0/2686282188 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f414415a3b0 0x7f41441eec60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f413c00c7a0 tx=0x7f413c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:53.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.136+0000 7f413b7fe640 1 -- 172.21.15.111:0/2686282188 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c003b70 con 0x7f414415a3b0 2024-09-13T17:57:53.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.136+0000 7f413b7fe640 1 -- 172.21.15.111:0/2686282188 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413c003cf0 con 0x7f414415a3b0 2024-09-13T17:57:53.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.136+0000 7f41489b5640 1 -- 172.21.15.111:0/2686282188 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41440aee80 con 0x7f414415a3b0 2024-09-13T17:57:53.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.136+0000 7f413b7fe640 1 -- 172.21.15.111:0/2686282188 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c003ea0 con 0x7f414415a3b0 2024-09-13T17:57:53.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.136+0000 7f41489b5640 1 -- 172.21.15.111:0/2686282188 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414415d530 con 0x7f414415a3b0 2024-09-13T17:57:53.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.137+0000 7f41489b5640 1 -- 172.21.15.111:0/2686282188 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41440aece0 con 0x7f414415a3b0 2024-09-13T17:57:53.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.137+0000 7f413b7fe640 1 -- 172.21.15.111:0/2686282188 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f413c00aa90 con 0x7f414415a3b0 2024-09-13T17:57:53.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.137+0000 7f413b7fe640 1 --2- 172.21.15.111:0/2686282188 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f412406ac00 0x7f412406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.138+0000 7f414259c640 1 --2- 172.21.15.111:0/2686282188 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f412406ac00 0x7f412406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.138+0000 7f413b7fe640 1 -- 172.21.15.111:0/2686282188 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f413c0891d0 con 0x7f414415a3b0 2024-09-13T17:57:53.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.138+0000 7f414259c640 1 --2- 172.21.15.111:0/2686282188 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f412406ac00 0x7f412406d0c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4134007c30 tx=0x7f4134007b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:53.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.142+0000 7f413b7fe640 1 -- 172.21.15.111:0/2686282188 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f41440aece0 con 0x7f414415a3b0 2024-09-13T17:57:53.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.300+0000 7f41489b5640 1 -- 172.21.15.111:0/2686282188 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f414415d530 con 0x7f414415a3b0 2024-09-13T17:57:53.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.301+0000 7f413b7fe640 1 -- 172.21.15.111:0/2686282188 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f414415d530 con 0x7f414415a3b0 2024-09-13T17:57:53.304 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:57:53.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.302+0000 7f41489b5640 1 -- 172.21.15.111:0/2686282188 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f412406ac00 msgr2=0x7f412406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.302+0000 7f41489b5640 1 --2- 172.21.15.111:0/2686282188 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f412406ac00 0x7f412406d0c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4134007c30 tx=0x7f4134007b80 comp rx=0 tx=0).stop 2024-09-13T17:57:53.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.302+0000 7f41489b5640 1 -- 172.21.15.111:0/2686282188 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f414415a3b0 msgr2=0x7f41441eec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.302+0000 7f41489b5640 1 --2- 172.21.15.111:0/2686282188 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f414415a3b0 0x7f41441eec60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f413c00c7a0 tx=0x7f413c00cc60 comp rx=0 tx=0).stop 2024-09-13T17:57:53.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.302+0000 7f41489b5640 1 -- 172.21.15.111:0/2686282188 shutdown_connections 2024-09-13T17:57:53.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.302+0000 7f41489b5640 1 --2- 172.21.15.111:0/2686282188 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f414415a3b0 0x7f41441eec60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.302+0000 7f41489b5640 1 --2- 172.21.15.111:0/2686282188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41440cea30 0x7f414415b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.302+0000 7f41489b5640 1 --2- 172.21.15.111:0/2686282188 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f412406ac00 0x7f412406d0c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.302+0000 7f41489b5640 1 --2- 172.21.15.111:0/2686282188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41440b20a0 0x7f414415ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.302+0000 7f41489b5640 1 -- 172.21.15.111:0/2686282188 >> 172.21.15.111:0/2686282188 conn(0x7f414406a0b0 msgr2=0x7f414406a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:53.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.303+0000 7f41489b5640 1 -- 172.21.15.111:0/2686282188 shutdown_connections 2024-09-13T17:57:53.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.303+0000 7f41489b5640 1 -- 172.21.15.111:0/2686282188 wait complete. 2024-09-13T17:57:53.316 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:57:53.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.414+0000 7f34a17b5640 1 Processor -- start 2024-09-13T17:57:53.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.414+0000 7f34a17b5640 1 -- start start 2024-09-13T17:57:53.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.414+0000 7f34a17b5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f349c0cd500 0x7f349c0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.414+0000 7f34a17b5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f349c0b1370 0x7f349c0d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.414+0000 7f34a17b5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c0d1650 0x7f349c0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.414+0000 7f34a17b5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f349c0d1f40 con 0x7f349c0b1370 2024-09-13T17:57:53.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.415+0000 7f34a17b5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f349c0d2190 con 0x7f349c0cd500 2024-09-13T17:57:53.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.415+0000 7f34a17b5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349c0ccb20 con 0x7f349c0d1650 2024-09-13T17:57:53.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.415+0000 7f349a7fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f349c0b1370 0x7f349c0d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.415+0000 7f349a7fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f349c0b1370 0x7f349c0d2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53182/0 (socket says 172.21.15.111:53182) 2024-09-13T17:57:53.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.415+0000 7f349a7fc640 1 -- 172.21.15.111:0/3455972279 learned_addr learned my addr 172.21.15.111:0/3455972279 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:53.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.415+0000 7f349b7fe640 1 --2- 172.21.15.111:0/3455972279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c0d1650 0x7f349c0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.415+0000 7f349affd640 1 --2- 172.21.15.111:0/3455972279 >> v2:172.21.15.188:3300/0 conn(0x7f349c0cd500 0x7f349c0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.415+0000 7f349a7fc640 1 -- 172.21.15.111:0/3455972279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c0d1650 msgr2=0x7f349c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:53.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.415+0000 7f349a7fc640 1 --2- 172.21.15.111:0/3455972279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c0d1650 0x7f349c0b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.415+0000 7f349a7fc640 1 -- 172.21.15.111:0/3455972279 >> v2:172.21.15.188:3300/0 conn(0x7f349c0cd500 msgr2=0x7f349c0b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:53.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.415+0000 7f349a7fc640 1 --2- 172.21.15.111:0/3455972279 >> v2:172.21.15.188:3300/0 conn(0x7f349c0cd500 0x7f349c0b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.415+0000 7f349a7fc640 1 -- 172.21.15.111:0/3455972279 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349c0cce00 con 0x7f349c0b1370 2024-09-13T17:57:53.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.416+0000 7f349a7fc640 1 --2- 172.21.15.111:0/3455972279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f349c0b1370 0x7f349c0d2f50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f349c0ccad0 tx=0x7f349002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276761e74afc2d7d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:53.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.416+0000 7f3499ffb640 1 -- 172.21.15.111:0/3455972279 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490032440 con 0x7f349c0b1370 2024-09-13T17:57:53.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.416+0000 7f3499ffb640 1 -- 172.21.15.111:0/3455972279 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34900325a0 con 0x7f349c0b1370 2024-09-13T17:57:53.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.416+0000 7f3499ffb640 1 -- 172.21.15.111:0/3455972279 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490032930 con 0x7f349c0b1370 2024-09-13T17:57:53.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.416+0000 7f34a17b5640 1 -- 172.21.15.111:0/3455972279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f349c0b1370 msgr2=0x7f349c0d2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.416+0000 7f34a17b5640 1 --2- 172.21.15.111:0/3455972279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f349c0b1370 0x7f349c0d2f50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f349c0ccad0 tx=0x7f349002a1e0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.416+0000 7f34a17b5640 1 -- 172.21.15.111:0/3455972279 shutdown_connections 2024-09-13T17:57:53.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.416+0000 7f34a17b5640 1 --2- 172.21.15.111:0/3455972279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c0d1650 0x7f349c0b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.416+0000 7f34a17b5640 1 --2- 172.21.15.111:0/3455972279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f349c0b1370 0x7f349c0d2f50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.416+0000 7f34a17b5640 1 --2- 172.21.15.111:0/3455972279 >> v2:172.21.15.188:3300/0 conn(0x7f349c0cd500 0x7f349c0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.416+0000 7f34a17b5640 1 -- 172.21.15.111:0/3455972279 >> 172.21.15.111:0/3455972279 conn(0x7f349c089f60 msgr2=0x7f349c0d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:53.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.416+0000 7f34a17b5640 1 -- 172.21.15.111:0/3455972279 shutdown_connections 2024-09-13T17:57:53.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.416+0000 7f34a17b5640 1 -- 172.21.15.111:0/3455972279 wait complete. 2024-09-13T17:57:53.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.418+0000 7f34a17b5640 1 Processor -- start 2024-09-13T17:57:53.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.418+0000 7f34a17b5640 1 -- start start 2024-09-13T17:57:53.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.418+0000 7f34a17b5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f349c0cd500 0x7f349c156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.418+0000 7f34a17b5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f349c0d1650 0x7f349c0cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.418+0000 7f34a17b5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c0b7d80 0x7f349c1f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.418+0000 7f34a17b5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c0cc680 con 0x7f349c0d1650 2024-09-13T17:57:53.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.418+0000 7f34a17b5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c0c8d00 con 0x7f349c0cd500 2024-09-13T17:57:53.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.418+0000 7f34a17b5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349c0c8740 con 0x7f349c0b7d80 2024-09-13T17:57:53.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.419+0000 7f349a7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f349c0d1650 0x7f349c0cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.419+0000 7f349a7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f349c0d1650 0x7f349c0cbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53190/0 (socket says 172.21.15.111:53190) 2024-09-13T17:57:53.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.419+0000 7f349a7fc640 1 -- 172.21.15.111:0/1708170592 learned_addr learned my addr 172.21.15.111:0/1708170592 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:53.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.419+0000 7f349b7fe640 1 --2- 172.21.15.111:0/1708170592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c0b7d80 0x7f349c1f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.419+0000 7f349affd640 1 --2- 172.21.15.111:0/1708170592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f349c0cd500 0x7f349c156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.419+0000 7f349a7fc640 1 -- 172.21.15.111:0/1708170592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c0b7d80 msgr2=0x7f349c1f3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.419+0000 7f349a7fc640 1 --2- 172.21.15.111:0/1708170592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c0b7d80 0x7f349c1f3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.419+0000 7f349a7fc640 1 -- 172.21.15.111:0/1708170592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f349c0cd500 msgr2=0x7f349c156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.419+0000 7f349a7fc640 1 --2- 172.21.15.111:0/1708170592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f349c0cd500 0x7f349c156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.419+0000 7f349a7fc640 1 -- 172.21.15.111:0/1708170592 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34900094d0 con 0x7f349c0d1650 2024-09-13T17:57:53.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.419+0000 7f349a7fc640 1 --2- 172.21.15.111:0/1708170592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f349c0d1650 0x7f349c0cbf80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f349002a220 tx=0x7f34900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:53.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.419+0000 7f34837fe640 1 -- 172.21.15.111:0/1708170592 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490032440 con 0x7f349c0d1650 2024-09-13T17:57:53.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.419+0000 7f34837fe640 1 -- 172.21.15.111:0/1708170592 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3490030ba0 con 0x7f349c0d1650 2024-09-13T17:57:53.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.419+0000 7f34a17b5640 1 -- 172.21.15.111:0/1708170592 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349c0c8a20 con 0x7f349c0d1650 2024-09-13T17:57:53.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.419+0000 7f34837fe640 1 -- 172.21.15.111:0/1708170592 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349003d440 con 0x7f349c0d1650 2024-09-13T17:57:53.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.419+0000 7f34a17b5640 1 -- 172.21.15.111:0/1708170592 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349c0b8210 con 0x7f349c0d1650 2024-09-13T17:57:53.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.420+0000 7f34a17b5640 1 -- 172.21.15.111:0/1708170592 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349c1f22b0 con 0x7f349c0d1650 2024-09-13T17:57:53.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.421+0000 7f34837fe640 1 -- 172.21.15.111:0/1708170592 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3490039040 con 0x7f349c0d1650 2024-09-13T17:57:53.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.421+0000 7f34837fe640 1 --2- 172.21.15.111:0/1708170592 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f348806ac00 0x7f348806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.421+0000 7f34837fe640 1 -- 172.21.15.111:0/1708170592 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f34900aa0d0 con 0x7f349c0d1650 2024-09-13T17:57:53.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.421+0000 7f349affd640 1 --2- 172.21.15.111:0/1708170592 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f348806ac00 0x7f348806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.422+0000 7f349affd640 1 --2- 172.21.15.111:0/1708170592 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f348806ac00 0x7f348806d0c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f348c006fd0 tx=0x7f348c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:53.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.425+0000 7f34837fe640 1 -- 172.21.15.111:0/1708170592 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f349c1f22b0 con 0x7f349c0d1650 2024-09-13T17:57:53.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.585+0000 7f34a17b5640 1 -- 172.21.15.111:0/1708170592 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f349c1f22b0 con 0x7f349c0d1650 2024-09-13T17:57:53.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.585+0000 7f34837fe640 1 -- 172.21.15.111:0/1708170592 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f349c1f22b0 con 0x7f349c0d1650 2024-09-13T17:57:53.588 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:57:53.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.587+0000 7f34a17b5640 1 -- 172.21.15.111:0/1708170592 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f348806ac00 msgr2=0x7f348806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.587+0000 7f34a17b5640 1 --2- 172.21.15.111:0/1708170592 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f348806ac00 0x7f348806d0c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f348c006fd0 tx=0x7f348c008040 comp rx=0 tx=0).stop 2024-09-13T17:57:53.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.587+0000 7f34a17b5640 1 -- 172.21.15.111:0/1708170592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f349c0d1650 msgr2=0x7f349c0cbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.587+0000 7f34a17b5640 1 --2- 172.21.15.111:0/1708170592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f349c0d1650 0x7f349c0cbf80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f349002a220 tx=0x7f34900046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.587+0000 7f34a17b5640 1 -- 172.21.15.111:0/1708170592 shutdown_connections 2024-09-13T17:57:53.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.587+0000 7f34a17b5640 1 --2- 172.21.15.111:0/1708170592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c0b7d80 0x7f349c1f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.587+0000 7f34a17b5640 1 --2- 172.21.15.111:0/1708170592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f349c0d1650 0x7f349c0cbf80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.587+0000 7f34a17b5640 1 --2- 172.21.15.111:0/1708170592 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f348806ac00 0x7f348806d0c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.587+0000 7f34a17b5640 1 --2- 172.21.15.111:0/1708170592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f349c0cd500 0x7f349c156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.587+0000 7f34a17b5640 1 -- 172.21.15.111:0/1708170592 >> 172.21.15.111:0/1708170592 conn(0x7f349c089f60 msgr2=0x7f349c0d0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:53.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.587+0000 7f34a17b5640 1 -- 172.21.15.111:0/1708170592 shutdown_connections 2024-09-13T17:57:53.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.587+0000 7f34a17b5640 1 -- 172.21.15.111:0/1708170592 wait complete. 2024-09-13T17:57:53.601 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:57:53.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.702+0000 7f07deea3640 1 Processor -- start 2024-09-13T17:57:53.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.702+0000 7f07deea3640 1 -- start start 2024-09-13T17:57:53.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.703+0000 7f07deea3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07d8072110 0x7f07d8070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.703+0000 7f07deea3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f07d80c8920 0x7f07d80c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.703+0000 7f07deea3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d80abdf0 con 0x7f07d8071d30 2024-09-13T17:57:53.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.703+0000 7f07deea3640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d80c9310 con 0x7f07d80c8920 2024-09-13T17:57:53.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.703+0000 7f07deea3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d80b2280 con 0x7f07d8072110 2024-09-13T17:57:53.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.703+0000 7f07dd440640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f07d80c8920 0x7f07d80c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.703+0000 7f07dd440640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f07d80c8920 0x7f07d80c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53196/0 (socket says 172.21.15.111:53196) 2024-09-13T17:57:53.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.703+0000 7f07dd440640 1 -- 172.21.15.111:0/2655519419 learned_addr learned my addr 172.21.15.111:0/2655519419 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:53.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.703+0000 7f07cffff640 1 --2- 172.21.15.111:0/2655519419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07d8072110 0x7f07d8070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.703+0000 7f07dd440640 1 -- 172.21.15.111:0/2655519419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07d8072110 msgr2=0x7f07d8070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:53.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.703+0000 7f07dd440640 1 --2- 172.21.15.111:0/2655519419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07d8072110 0x7f07d8070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.703+0000 7f07dd440640 1 -- 172.21.15.111:0/2655519419 >> v1:172.21.15.188:6789/0 conn(0x7f07d8071d30 legacy=0x7f07d80b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:53.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.703+0000 7f07dd440640 1 -- 172.21.15.111:0/2655519419 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d80ab1e0 con 0x7f07d80c8920 2024-09-13T17:57:53.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.703+0000 7f07cffff640 1 --2- 172.21.15.111:0/2655519419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07d8072110 0x7f07d8070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:53.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.704+0000 7f07dd440640 1 --2- 172.21.15.111:0/2655519419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07d80c8920 0x7f07d80c8dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f07d000bb40 tx=0x7f07d002c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf193ce6a778a0fc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:53.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.704+0000 7f07cf7fe640 1 -- 172.21.15.111:0/2655519419 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d00327c0 con 0x7f07d80c8920 2024-09-13T17:57:53.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.704+0000 7f07cf7fe640 1 -- 172.21.15.111:0/2655519419 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d0032920 con 0x7f07d80c8920 2024-09-13T17:57:53.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.704+0000 7f07cf7fe640 1 -- 172.21.15.111:0/2655519419 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0032cb0 con 0x7f07d80c8920 2024-09-13T17:57:53.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.704+0000 7f07deea3640 1 -- 172.21.15.111:0/2655519419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07d80c8920 msgr2=0x7f07d80c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.704+0000 7f07deea3640 1 --2- 172.21.15.111:0/2655519419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07d80c8920 0x7f07d80c8dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f07d000bb40 tx=0x7f07d002c900 comp rx=0 tx=0).stop 2024-09-13T17:57:53.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.704+0000 7f07deea3640 1 -- 172.21.15.111:0/2655519419 shutdown_connections 2024-09-13T17:57:53.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.704+0000 7f07deea3640 1 --2- 172.21.15.111:0/2655519419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07d80c8920 0x7f07d80c8dd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.704+0000 7f07deea3640 1 --2- 172.21.15.111:0/2655519419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07d8072110 0x7f07d8070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.704+0000 7f07deea3640 1 -- 172.21.15.111:0/2655519419 >> 172.21.15.111:0/2655519419 conn(0x7f07d8069ed0 msgr2=0x7f07d806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:53.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.705+0000 7f07deea3640 1 -- 172.21.15.111:0/2655519419 shutdown_connections 2024-09-13T17:57:53.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.705+0000 7f07deea3640 1 -- 172.21.15.111:0/2655519419 wait complete. 2024-09-13T17:57:53.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.706+0000 7f07deea3640 1 Processor -- start 2024-09-13T17:57:53.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.706+0000 7f07deea3640 1 -- start start 2024-09-13T17:57:53.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.706+0000 7f07deea3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07d8071d30 0x7f07d8151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07deea3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07d8072110 0x7f07d80c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07deea3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07d80c8920 0x7f07d81f07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07deea3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d80c6930 con 0x7f07d80c8920 2024-09-13T17:57:53.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07deea3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d80c6b60 con 0x7f07d8071d30 2024-09-13T17:57:53.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07deea3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d80c6e40 con 0x7f07d8072110 2024-09-13T17:57:53.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07dd440640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07d80c8920 0x7f07d81f07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07dd440640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07d80c8920 0x7f07d81f07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53200/0 (socket says 172.21.15.111:53200) 2024-09-13T17:57:53.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07dd440640 1 -- 172.21.15.111:0/2783691461 learned_addr learned my addr 172.21.15.111:0/2783691461 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:53.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07cffff640 1 --2- 172.21.15.111:0/2783691461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07d8072110 0x7f07d80c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07dcc3f640 1 --2- 172.21.15.111:0/2783691461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07d8071d30 0x7f07d8151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07dd440640 1 -- 172.21.15.111:0/2783691461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07d8072110 msgr2=0x7f07d80c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07dd440640 1 --2- 172.21.15.111:0/2783691461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07d8072110 0x7f07d80c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07dd440640 1 -- 172.21.15.111:0/2783691461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07d8071d30 msgr2=0x7f07d8151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07dd440640 1 --2- 172.21.15.111:0/2783691461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07d8071d30 0x7f07d8151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07dd440640 1 -- 172.21.15.111:0/2783691461 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d000b770 con 0x7f07d80c8920 2024-09-13T17:57:53.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07dd440640 1 --2- 172.21.15.111:0/2783691461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07d80c8920 0x7f07d81f07b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f07d000b2c0 tx=0x7f07d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:53.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07cd7fa640 1 -- 172.21.15.111:0/2783691461 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0004230 con 0x7f07d80c8920 2024-09-13T17:57:53.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07cd7fa640 1 -- 172.21.15.111:0/2783691461 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d0004390 con 0x7f07d80c8920 2024-09-13T17:57:53.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07deea3640 1 -- 172.21.15.111:0/2783691461 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d80c7120 con 0x7f07d80c8920 2024-09-13T17:57:53.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07cd7fa640 1 -- 172.21.15.111:0/2783691461 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0033960 con 0x7f07d80c8920 2024-09-13T17:57:53.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.707+0000 7f07deea3640 1 -- 172.21.15.111:0/2783691461 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d80c76e0 con 0x7f07d80c8920 2024-09-13T17:57:53.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.708+0000 7f07deea3640 1 -- 172.21.15.111:0/2783691461 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d80c78d0 con 0x7f07d80c8920 2024-09-13T17:57:53.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.709+0000 7f07cd7fa640 1 -- 172.21.15.111:0/2783691461 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f07d0033ac0 con 0x7f07d80c8920 2024-09-13T17:57:53.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.709+0000 7f07cd7fa640 1 --2- 172.21.15.111:0/2783691461 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f07c406ac00 0x7f07c406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:53.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.709+0000 7f07cd7fa640 1 -- 172.21.15.111:0/2783691461 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f07d00accf0 con 0x7f07d80c8920 2024-09-13T17:57:53.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.709+0000 7f07dcc3f640 1 --2- 172.21.15.111:0/2783691461 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f07c406ac00 0x7f07c406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:53.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.710+0000 7f07dcc3f640 1 --2- 172.21.15.111:0/2783691461 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f07c406ac00 0x7f07c406d0c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f07d4005660 tx=0x7f07d4003d90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:53.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.713+0000 7f07cd7fa640 1 -- 172.21.15.111:0/2783691461 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f07d00b1070 con 0x7f07d80c8920 2024-09-13T17:57:53.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.912+0000 7f07deea3640 1 -- 172.21.15.111:0/2783691461 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07d80c78d0 con 0x7f07d80c8920 2024-09-13T17:57:53.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.913+0000 7f07cd7fa640 1 -- 172.21.15.111:0/2783691461 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4742 (secure 0 0 0) 0x7f07d007b8d0 con 0x7f07d80c8920 2024-09-13T17:57:53.916 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:53.916 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4001406804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":4001406804},{"type":"v1","addr":"172.21.15.188:6835","nonce":4001406804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2565466103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2565466103},{"type":"v1","addr":"172.21.15.111:6837","nonce":2565466103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/1252690799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1252690799},{"type":"v1","addr":"172.21.15.111:6836","nonce":1252690799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:57:25.702997+0000","modified":"2024-09-13T17:57:46.351493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/1266310843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1266310843},{"type":"v1","addr":"172.21.15.188:6834","nonce":1266310843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:57:53.916 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-09-13T17:57:53.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.915+0000 7f07deea3640 1 -- 172.21.15.111:0/2783691461 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f07c406ac00 msgr2=0x7f07c406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.915+0000 7f07deea3640 1 --2- 172.21.15.111:0/2783691461 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f07c406ac00 0x7f07c406d0c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f07d4005660 tx=0x7f07d4003d90 comp rx=0 tx=0).stop 2024-09-13T17:57:53.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.915+0000 7f07deea3640 1 -- 172.21.15.111:0/2783691461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07d80c8920 msgr2=0x7f07d81f07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:53.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.915+0000 7f07deea3640 1 --2- 172.21.15.111:0/2783691461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07d80c8920 0x7f07d81f07b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f07d000b2c0 tx=0x7f07d00046c0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.915+0000 7f07deea3640 1 -- 172.21.15.111:0/2783691461 shutdown_connections 2024-09-13T17:57:53.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.915+0000 7f07deea3640 1 --2- 172.21.15.111:0/2783691461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07d80c8920 0x7f07d81f07b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.915+0000 7f07deea3640 1 --2- 172.21.15.111:0/2783691461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07d8072110 0x7f07d80c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.915+0000 7f07deea3640 1 --2- 172.21.15.111:0/2783691461 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f07c406ac00 0x7f07c406d0c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.915+0000 7f07deea3640 1 --2- 172.21.15.111:0/2783691461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07d8071d30 0x7f07d8151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:53.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.915+0000 7f07deea3640 1 -- 172.21.15.111:0/2783691461 >> 172.21.15.111:0/2783691461 conn(0x7f07d8069ed0 msgr2=0x7f07d806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:53.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.915+0000 7f07deea3640 1 -- 172.21.15.111:0/2783691461 shutdown_connections 2024-09-13T17:57:53.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:53.915+0000 7f07deea3640 1 -- 172.21.15.111:0/2783691461 wait complete. 2024-09-13T17:57:53.929 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:54.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.022+0000 7fdf24c8f640 1 Processor -- start 2024-09-13T17:57:54.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.022+0000 7fdf24c8f640 1 -- start start 2024-09-13T17:57:54.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.023+0000 7fdf24c8f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf2006c4c0 0x7fdf2006c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.023+0000 7fdf24c8f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdf200b2590 0x7fdf2006d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.023+0000 7fdf24c8f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdf200cbea0 0x7fdf200cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.023+0000 7fdf24c8f640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2006deb0 con 0x7fdf200b2590 2024-09-13T17:57:54.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.023+0000 7fdf24c8f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf200cc890 con 0x7fdf200cbea0 2024-09-13T17:57:54.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.023+0000 7fdf24c8f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2006e0c0 con 0x7fdf2006c4c0 2024-09-13T17:57:54.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.023+0000 7fdf1dd9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdf200b2590 0x7fdf2006d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.023+0000 7fdf1dd9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdf200b2590 0x7fdf2006d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53214/0 (socket says 172.21.15.111:53214) 2024-09-13T17:57:54.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.023+0000 7fdf1dd9b640 1 -- 172.21.15.111:0/216302843 learned_addr learned my addr 172.21.15.111:0/216302843 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:54.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.023+0000 7fdf1ed9d640 1 --2- 172.21.15.111:0/216302843 >> v2:172.21.15.188:3300/0 conn(0x7fdf200cbea0 0x7fdf200cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.023+0000 7fdf1e59c640 1 --2- 172.21.15.111:0/216302843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf2006c4c0 0x7fdf2006c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.023+0000 7fdf1dd9b640 1 -- 172.21.15.111:0/216302843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf2006c4c0 msgr2=0x7fdf2006c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:54.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.023+0000 7fdf1dd9b640 1 --2- 172.21.15.111:0/216302843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf2006c4c0 0x7fdf2006c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.023+0000 7fdf1dd9b640 1 -- 172.21.15.111:0/216302843 >> v2:172.21.15.188:3300/0 conn(0x7fdf200cbea0 msgr2=0x7fdf200cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:54.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.023+0000 7fdf1dd9b640 1 --2- 172.21.15.111:0/216302843 >> v2:172.21.15.188:3300/0 conn(0x7fdf200cbea0 0x7fdf200cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.024+0000 7fdf1dd9b640 1 -- 172.21.15.111:0/216302843 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2006e3a0 con 0x7fdf200b2590 2024-09-13T17:57:54.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.024+0000 7fdf1e59c640 1 --2- 172.21.15.111:0/216302843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf2006c4c0 0x7fdf2006c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:54.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.024+0000 7fdf1dd9b640 1 --2- 172.21.15.111:0/216302843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf200b2590 0x7fdf2006d3a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009870 tx=0x7fdf1402b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a523ca770cffe0a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:54.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.024+0000 7fdf1d59a640 1 -- 172.21.15.111:0/216302843 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14030710 con 0x7fdf200b2590 2024-09-13T17:57:54.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.024+0000 7fdf1d59a640 1 -- 172.21.15.111:0/216302843 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1402e050 con 0x7fdf200b2590 2024-09-13T17:57:54.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.024+0000 7fdf1d59a640 1 -- 172.21.15.111:0/216302843 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14030a90 con 0x7fdf200b2590 2024-09-13T17:57:54.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.025+0000 7fdf24c8f640 1 -- 172.21.15.111:0/216302843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf200b2590 msgr2=0x7fdf2006d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.025+0000 7fdf24c8f640 1 --2- 172.21.15.111:0/216302843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf200b2590 0x7fdf2006d3a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009870 tx=0x7fdf1402b430 comp rx=0 tx=0).stop 2024-09-13T17:57:54.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.025+0000 7fdf24c8f640 1 -- 172.21.15.111:0/216302843 shutdown_connections 2024-09-13T17:57:54.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.025+0000 7fdf24c8f640 1 --2- 172.21.15.111:0/216302843 >> v2:172.21.15.188:3300/0 conn(0x7fdf200cbea0 0x7fdf200cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.025+0000 7fdf24c8f640 1 --2- 172.21.15.111:0/216302843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf200b2590 0x7fdf2006d3a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.025+0000 7fdf24c8f640 1 --2- 172.21.15.111:0/216302843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf2006c4c0 0x7fdf2006c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.025+0000 7fdf24c8f640 1 -- 172.21.15.111:0/216302843 >> 172.21.15.111:0/216302843 conn(0x7fdf20069ed0 msgr2=0x7fdf2006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:54.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.025+0000 7fdf24c8f640 1 -- 172.21.15.111:0/216302843 shutdown_connections 2024-09-13T17:57:54.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.025+0000 7fdf24c8f640 1 -- 172.21.15.111:0/216302843 wait complete. 2024-09-13T17:57:54.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.026+0000 7fdf24c8f640 1 Processor -- start 2024-09-13T17:57:54.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.026+0000 7fdf24c8f640 1 -- start start 2024-09-13T17:57:54.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.027+0000 7fdf24c8f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf2006c4c0 0x7fdf20156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.027+0000 7fdf24c8f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf200b2590 0x7fdf200c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.027+0000 7fdf24c8f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf200cbea0 0x7fdf20158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.027+0000 7fdf24c8f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf201594d0 con 0x7fdf200cbea0 2024-09-13T17:57:54.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.027+0000 7fdf24c8f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf201457e0 con 0x7fdf2006c4c0 2024-09-13T17:57:54.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.027+0000 7fdf24c8f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2015b850 con 0x7fdf200b2590 2024-09-13T17:57:54.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.027+0000 7fdf1ed9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf200cbea0 0x7fdf20158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.027+0000 7fdf1ed9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf200cbea0 0x7fdf20158f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53228/0 (socket says 172.21.15.111:53228) 2024-09-13T17:57:54.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.027+0000 7fdf1ed9d640 1 -- 172.21.15.111:0/4006143736 learned_addr learned my addr 172.21.15.111:0/4006143736 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:54.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.027+0000 7fdf1e59c640 1 --2- 172.21.15.111:0/4006143736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf2006c4c0 0x7fdf20156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.027+0000 7fdf1dd9b640 1 --2- 172.21.15.111:0/4006143736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf200b2590 0x7fdf200c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.027+0000 7fdf1ed9d640 1 -- 172.21.15.111:0/4006143736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf200b2590 msgr2=0x7fdf200c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.027+0000 7fdf1ed9d640 1 --2- 172.21.15.111:0/4006143736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf200b2590 0x7fdf200c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.027+0000 7fdf1ed9d640 1 -- 172.21.15.111:0/4006143736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf2006c4c0 msgr2=0x7fdf20156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.027+0000 7fdf1ed9d640 1 --2- 172.21.15.111:0/4006143736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf2006c4c0 0x7fdf20156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.027+0000 7fdf1ed9d640 1 -- 172.21.15.111:0/4006143736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf140094d0 con 0x7fdf200cbea0 2024-09-13T17:57:54.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.028+0000 7fdf1ed9d640 1 --2- 172.21.15.111:0/4006143736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf200cbea0 0x7fdf20158f90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800efc0 tx=0x7fdf1800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:54.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.028+0000 7fdf0effd640 1 -- 172.21.15.111:0/4006143736 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18014480 con 0x7fdf200cbea0 2024-09-13T17:57:54.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.028+0000 7fdf0effd640 1 -- 172.21.15.111:0/4006143736 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1800f040 con 0x7fdf200cbea0 2024-09-13T17:57:54.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.028+0000 7fdf24c8f640 1 -- 172.21.15.111:0/4006143736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf20145220 con 0x7fdf200cbea0 2024-09-13T17:57:54.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.028+0000 7fdf0effd640 1 -- 172.21.15.111:0/4006143736 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18014820 con 0x7fdf200cbea0 2024-09-13T17:57:54.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.028+0000 7fdf24c8f640 1 -- 172.21.15.111:0/4006143736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2015bb10 con 0x7fdf200cbea0 2024-09-13T17:57:54.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.029+0000 7fdf24c8f640 1 -- 172.21.15.111:0/4006143736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2015bd00 con 0x7fdf200cbea0 2024-09-13T17:57:54.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.029+0000 7fdf0effd640 1 -- 172.21.15.111:0/4006143736 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdf18014980 con 0x7fdf200cbea0 2024-09-13T17:57:54.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.030+0000 7fdf0effd640 1 --2- 172.21.15.111:0/4006143736 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdf0006ac00 0x7fdf0006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.030+0000 7fdf0effd640 1 -- 172.21.15.111:0/4006143736 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdf18089fa0 con 0x7fdf200cbea0 2024-09-13T17:57:54.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.030+0000 7fdf1e59c640 1 --2- 172.21.15.111:0/4006143736 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdf0006ac00 0x7fdf0006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.030+0000 7fdf1e59c640 1 --2- 172.21.15.111:0/4006143736 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdf0006ac00 0x7fdf0006d0c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdf201f2c20 tx=0x7fdf10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:54.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.034+0000 7fdf0effd640 1 -- 172.21.15.111:0/4006143736 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fdf180bd9c0 con 0x7fdf200cbea0 2024-09-13T17:57:54.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.194+0000 7fdf24c8f640 1 -- 172.21.15.111:0/4006143736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf2006d3a0 con 0x7fdf200cbea0 2024-09-13T17:57:54.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.195+0000 7fdf0effd640 1 -- 172.21.15.111:0/4006143736 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7fdf18087800 con 0x7fdf200cbea0 2024-09-13T17:57:54.198 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:54.198 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:30.065235+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:54.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.197+0000 7fdf24c8f640 1 -- 172.21.15.111:0/4006143736 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdf0006ac00 msgr2=0x7fdf0006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.197+0000 7fdf24c8f640 1 --2- 172.21.15.111:0/4006143736 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdf0006ac00 0x7fdf0006d0c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdf201f2c20 tx=0x7fdf10008040 comp rx=0 tx=0).stop 2024-09-13T17:57:54.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.197+0000 7fdf24c8f640 1 -- 172.21.15.111:0/4006143736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf200cbea0 msgr2=0x7fdf20158f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.197+0000 7fdf24c8f640 1 --2- 172.21.15.111:0/4006143736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf200cbea0 0x7fdf20158f90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800efc0 tx=0x7fdf1800c490 comp rx=0 tx=0).stop 2024-09-13T17:57:54.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.197+0000 7fdf24c8f640 1 -- 172.21.15.111:0/4006143736 shutdown_connections 2024-09-13T17:57:54.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.197+0000 7fdf24c8f640 1 --2- 172.21.15.111:0/4006143736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf200cbea0 0x7fdf20158f90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.197+0000 7fdf24c8f640 1 --2- 172.21.15.111:0/4006143736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf200b2590 0x7fdf200c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.197+0000 7fdf24c8f640 1 --2- 172.21.15.111:0/4006143736 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdf0006ac00 0x7fdf0006d0c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.197+0000 7fdf24c8f640 1 --2- 172.21.15.111:0/4006143736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf2006c4c0 0x7fdf20156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.197+0000 7fdf24c8f640 1 -- 172.21.15.111:0/4006143736 >> 172.21.15.111:0/4006143736 conn(0x7fdf20069ed0 msgr2=0x7fdf200cd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:54.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.197+0000 7fdf24c8f640 1 -- 172.21.15.111:0/4006143736 shutdown_connections 2024-09-13T17:57:54.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.197+0000 7fdf24c8f640 1 -- 172.21.15.111:0/4006143736 wait complete. 2024-09-13T17:57:54.211 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:57:54.211 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:57:54.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.309+0000 7ff62054c640 1 Processor -- start 2024-09-13T17:57:54.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.310+0000 7ff62054c640 1 -- start start 2024-09-13T17:57:54.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.311+0000 7ff62054c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6180b1370 0x7ff6180b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.311+0000 7ff62054c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff6180d1650 0x7ff6180d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.311+0000 7ff62054c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6180d1f40 con 0x7ff6180cd500 2024-09-13T17:57:54.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.311+0000 7ff62054c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6180d2190 con 0x7ff6180d1650 2024-09-13T17:57:54.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.311+0000 7ff62054c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180ccb20 con 0x7ff6180b1370 2024-09-13T17:57:54.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.311+0000 7ff61eae9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff6180d1650 0x7ff6180d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.311+0000 7ff61dae7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6180b1370 0x7ff6180b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.311+0000 7ff61eae9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff6180d1650 0x7ff6180d2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53244/0 (socket says 172.21.15.111:53244) 2024-09-13T17:57:54.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.311+0000 7ff61e2e8640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ff6180cd500 0x7ff6180b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:33342/0 (socket says 172.21.15.111:33342) 2024-09-13T17:57:54.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.311+0000 7ff61eae9640 1 -- 172.21.15.111:0/3323140838 learned_addr learned my addr 172.21.15.111:0/3323140838 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:54.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.311+0000 7ff61eae9640 1 -- 172.21.15.111:0/3323140838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6180b1370 msgr2=0x7ff6180b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:54.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.311+0000 7ff61eae9640 1 --2- 172.21.15.111:0/3323140838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6180b1370 0x7ff6180b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.311+0000 7ff61eae9640 1 -- 172.21.15.111:0/3323140838 >> v1:172.21.15.188:6789/0 conn(0x7ff6180cd500 legacy=0x7ff6180b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.311+0000 7ff61eae9640 1 -- 172.21.15.111:0/3323140838 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6180cce00 con 0x7ff6180d1650 2024-09-13T17:57:54.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.311+0000 7ff61dae7640 1 --2- 172.21.15.111:0/3323140838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6180b1370 0x7ff6180b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:54.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.312+0000 7ff61eae9640 1 --2- 172.21.15.111:0/3323140838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6180d1650 0x7ff6180d2f50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff610009e80 tx=0x7ff61002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9145083853ca83 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:54.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.312+0000 7ff61d2e6640 1 -- 172.21.15.111:0/3323140838 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610031600 con 0x7ff6180d1650 2024-09-13T17:57:54.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.312+0000 7ff61d2e6640 1 -- 172.21.15.111:0/3323140838 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61002d050 con 0x7ff6180d1650 2024-09-13T17:57:54.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.312+0000 7ff61d2e6640 1 -- 172.21.15.111:0/3323140838 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610031980 con 0x7ff6180d1650 2024-09-13T17:57:54.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.312+0000 7ff62054c640 1 -- 172.21.15.111:0/3323140838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6180d1650 msgr2=0x7ff6180d2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.313+0000 7ff62054c640 1 --2- 172.21.15.111:0/3323140838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6180d1650 0x7ff6180d2f50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff610009e80 tx=0x7ff61002a750 comp rx=0 tx=0).stop 2024-09-13T17:57:54.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.313+0000 7ff62054c640 1 -- 172.21.15.111:0/3323140838 shutdown_connections 2024-09-13T17:57:54.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.313+0000 7ff62054c640 1 --2- 172.21.15.111:0/3323140838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6180d1650 0x7ff6180d2f50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.313+0000 7ff62054c640 1 --2- 172.21.15.111:0/3323140838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6180b1370 0x7ff6180b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.313+0000 7ff62054c640 1 -- 172.21.15.111:0/3323140838 >> 172.21.15.111:0/3323140838 conn(0x7ff618089f60 msgr2=0x7ff6180d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:54.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.313+0000 7ff62054c640 1 -- 172.21.15.111:0/3323140838 shutdown_connections 2024-09-13T17:57:54.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.313+0000 7ff62054c640 1 -- 172.21.15.111:0/3323140838 wait complete. 2024-09-13T17:57:54.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.314+0000 7ff62054c640 1 Processor -- start 2024-09-13T17:57:54.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.314+0000 7ff62054c640 1 -- start start 2024-09-13T17:57:54.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff62054c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6180cd500 0x7ff61815b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff62054c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6180d1650 0x7ff6180c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff62054c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff61815bd50 0x7ff61806c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff62054c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180ca790 con 0x7ff61815bd50 2024-09-13T17:57:54.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff62054c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180caa70 con 0x7ff6180cd500 2024-09-13T17:57:54.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff62054c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180cad50 con 0x7ff6180d1650 2024-09-13T17:57:54.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff61eae9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff61815bd50 0x7ff61806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff61e2e8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6180cd500 0x7ff61815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff61eae9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff61815bd50 0x7ff61806c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53252/0 (socket says 172.21.15.111:53252) 2024-09-13T17:57:54.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff61eae9640 1 -- 172.21.15.111:0/1232744596 learned_addr learned my addr 172.21.15.111:0/1232744596 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:54.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff61dae7640 1 --2- 172.21.15.111:0/1232744596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6180d1650 0x7ff6180c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff61eae9640 1 -- 172.21.15.111:0/1232744596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6180d1650 msgr2=0x7ff6180c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff61eae9640 1 --2- 172.21.15.111:0/1232744596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6180d1650 0x7ff6180c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff61eae9640 1 -- 172.21.15.111:0/1232744596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6180cd500 msgr2=0x7ff61815b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff61eae9640 1 --2- 172.21.15.111:0/1232744596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6180cd500 0x7ff61815b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff61eae9640 1 -- 172.21.15.111:0/1232744596 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff608009400 con 0x7ff61815bd50 2024-09-13T17:57:54.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff61eae9640 1 --2- 172.21.15.111:0/1232744596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff61815bd50 0x7ff61806c090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff610034040 tx=0x7ff610004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:54.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff60effd640 1 -- 172.21.15.111:0/1232744596 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61003d860 con 0x7ff61815bd50 2024-09-13T17:57:54.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff60effd640 1 -- 172.21.15.111:0/1232744596 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61003d9e0 con 0x7ff61815bd50 2024-09-13T17:57:54.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff62054c640 1 -- 172.21.15.111:0/1232744596 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff610009b30 con 0x7ff61815bd50 2024-09-13T17:57:54.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff60effd640 1 -- 172.21.15.111:0/1232744596 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61003dd90 con 0x7ff61815bd50 2024-09-13T17:57:54.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.315+0000 7ff62054c640 1 -- 172.21.15.111:0/1232744596 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6180cb310 con 0x7ff61815bd50 2024-09-13T17:57:54.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.316+0000 7ff62054c640 1 -- 172.21.15.111:0/1232744596 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6180cb530 con 0x7ff61815bd50 2024-09-13T17:57:54.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.317+0000 7ff60effd640 1 -- 172.21.15.111:0/1232744596 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff61003b400 con 0x7ff61815bd50 2024-09-13T17:57:54.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.317+0000 7ff60effd640 1 --2- 172.21.15.111:0/1232744596 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff5fc06ac00 0x7ff5fc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.317+0000 7ff60effd640 1 -- 172.21.15.111:0/1232744596 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff6100aa320 con 0x7ff61815bd50 2024-09-13T17:57:54.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.317+0000 7ff61e2e8640 1 --2- 172.21.15.111:0/1232744596 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff5fc06ac00 0x7ff5fc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.318+0000 7ff61e2e8640 1 --2- 172.21.15.111:0/1232744596 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff5fc06ac00 0x7ff5fc06d0c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff614000f60 tx=0x7ff614000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:54.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.321+0000 7ff60effd640 1 -- 172.21.15.111:0/1232744596 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff6100752c0 con 0x7ff61815bd50 2024-09-13T17:57:54.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.485+0000 7ff62054c640 1 -- 172.21.15.111:0/1232744596 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff6180b3850 con 0x7ff61815bd50 2024-09-13T17:57:54.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.485+0000 7ff60effd640 1 -- 172.21.15.111:0/1232744596 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff610009510 con 0x7ff61815bd50 2024-09-13T17:57:54.488 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:57:54.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.487+0000 7ff62054c640 1 -- 172.21.15.111:0/1232744596 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff5fc06ac00 msgr2=0x7ff5fc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.487+0000 7ff62054c640 1 --2- 172.21.15.111:0/1232744596 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff5fc06ac00 0x7ff5fc06d0c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff614000f60 tx=0x7ff614000ed0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.487+0000 7ff62054c640 1 -- 172.21.15.111:0/1232744596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff61815bd50 msgr2=0x7ff61806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.487+0000 7ff62054c640 1 --2- 172.21.15.111:0/1232744596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff61815bd50 0x7ff61806c090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff610034040 tx=0x7ff610004a40 comp rx=0 tx=0).stop 2024-09-13T17:57:54.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.487+0000 7ff62054c640 1 -- 172.21.15.111:0/1232744596 shutdown_connections 2024-09-13T17:57:54.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.487+0000 7ff62054c640 1 --2- 172.21.15.111:0/1232744596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff61815bd50 0x7ff61806c090 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.487+0000 7ff62054c640 1 --2- 172.21.15.111:0/1232744596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6180d1650 0x7ff6180c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.487+0000 7ff62054c640 1 --2- 172.21.15.111:0/1232744596 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff5fc06ac00 0x7ff5fc06d0c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.487+0000 7ff62054c640 1 --2- 172.21.15.111:0/1232744596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6180cd500 0x7ff61815b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.487+0000 7ff62054c640 1 -- 172.21.15.111:0/1232744596 >> 172.21.15.111:0/1232744596 conn(0x7ff618089f60 msgr2=0x7ff61815dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:54.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.487+0000 7ff62054c640 1 -- 172.21.15.111:0/1232744596 shutdown_connections 2024-09-13T17:57:54.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.487+0000 7ff62054c640 1 -- 172.21.15.111:0/1232744596 wait complete. 2024-09-13T17:57:54.500 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:57:54.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.598+0000 7f58fc4c5640 1 Processor -- start 2024-09-13T17:57:54.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.598+0000 7f58fc4c5640 1 -- start start 2024-09-13T17:57:54.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.599+0000 7f58fc4c5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f4158b00 0x7f58f406c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.599+0000 7f58fc4c5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f58f406c6d0 0x7f58f4070b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.599+0000 7f58fc4c5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f58f4071050 0x7f58f4071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.599+0000 7f58fc4c5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f406cb30 con 0x7f58f406c6d0 2024-09-13T17:57:54.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.599+0000 7f58fc4c5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f40b2d10 con 0x7f58f4071050 2024-09-13T17:57:54.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.599+0000 7f58fc4c5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40b2ff0 con 0x7f58f4158b00 2024-09-13T17:57:54.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.599+0000 7f58f9a60640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f58f406c6d0 0x7f58f4070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.599+0000 7f58f9a60640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f58f406c6d0 0x7f58f4070b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53256/0 (socket says 172.21.15.111:53256) 2024-09-13T17:57:54.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.599+0000 7f58f9a60640 1 -- 172.21.15.111:0/18831526 learned_addr learned my addr 172.21.15.111:0/18831526 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:54.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.599+0000 7f58fa261640 1 --2- 172.21.15.111:0/18831526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f4158b00 0x7f58f406c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.599+0000 7f58faa62640 1 --2- 172.21.15.111:0/18831526 >> v2:172.21.15.188:3300/0 conn(0x7f58f4071050 0x7f58f4071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.599+0000 7f58f9a60640 1 -- 172.21.15.111:0/18831526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f4158b00 msgr2=0x7f58f406c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:54.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.599+0000 7f58f9a60640 1 --2- 172.21.15.111:0/18831526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f4158b00 0x7f58f406c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.599+0000 7f58f9a60640 1 -- 172.21.15.111:0/18831526 >> v2:172.21.15.188:3300/0 conn(0x7f58f4071050 msgr2=0x7f58f4071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:54.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.599+0000 7f58f9a60640 1 --2- 172.21.15.111:0/18831526 >> v2:172.21.15.188:3300/0 conn(0x7f58f4071050 0x7f58f4071500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.599+0000 7f58f9a60640 1 -- 172.21.15.111:0/18831526 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f4070310 con 0x7f58f406c6d0 2024-09-13T17:57:54.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.600+0000 7f58f9a60640 1 --2- 172.21.15.111:0/18831526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f406c6d0 0x7f58f4070b10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f58f406da20 tx=0x7f58e402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57118bd225a08242 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:54.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.600+0000 7f58f925f640 1 -- 172.21.15.111:0/18831526 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4032790 con 0x7f58f406c6d0 2024-09-13T17:57:54.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.600+0000 7f58f925f640 1 -- 172.21.15.111:0/18831526 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e40328f0 con 0x7f58f406c6d0 2024-09-13T17:57:54.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.600+0000 7f58f925f640 1 -- 172.21.15.111:0/18831526 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4032c80 con 0x7f58f406c6d0 2024-09-13T17:57:54.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.601+0000 7f58fc4c5640 1 -- 172.21.15.111:0/18831526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f406c6d0 msgr2=0x7f58f4070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.601+0000 7f58fc4c5640 1 --2- 172.21.15.111:0/18831526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f406c6d0 0x7f58f4070b10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f58f406da20 tx=0x7f58e402c890 comp rx=0 tx=0).stop 2024-09-13T17:57:54.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.601+0000 7f58fc4c5640 1 -- 172.21.15.111:0/18831526 shutdown_connections 2024-09-13T17:57:54.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.601+0000 7f58fc4c5640 1 --2- 172.21.15.111:0/18831526 >> v2:172.21.15.188:3300/0 conn(0x7f58f4071050 0x7f58f4071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.601+0000 7f58fc4c5640 1 --2- 172.21.15.111:0/18831526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f406c6d0 0x7f58f4070b10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.601+0000 7f58fc4c5640 1 --2- 172.21.15.111:0/18831526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f4158b00 0x7f58f406c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.601+0000 7f58fc4c5640 1 -- 172.21.15.111:0/18831526 >> 172.21.15.111:0/18831526 conn(0x7f58f406a0b0 msgr2=0x7f58f406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:54.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.601+0000 7f58fc4c5640 1 -- 172.21.15.111:0/18831526 shutdown_connections 2024-09-13T17:57:54.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.601+0000 7f58fc4c5640 1 -- 172.21.15.111:0/18831526 wait complete. 2024-09-13T17:57:54.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.602+0000 7f58fc4c5640 1 Processor -- start 2024-09-13T17:57:54.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.602+0000 7f58fc4c5640 1 -- start start 2024-09-13T17:57:54.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.602+0000 7f58fc4c5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f406c6d0 0x7f58f40c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.602+0000 7f58fc4c5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58f4071050 0x7f58f41e3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58fc4c5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f4158b00 0x7f58f41e12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58fc4c5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f415b6a0 con 0x7f58f4158b00 2024-09-13T17:57:54.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58fc4c5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f4159100 con 0x7f58f4071050 2024-09-13T17:57:54.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58fc4c5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f41e3c40 con 0x7f58f406c6d0 2024-09-13T17:57:54.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58faa62640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f4158b00 0x7f58f41e12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58faa62640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f4158b00 0x7f58f41e12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53258/0 (socket says 172.21.15.111:53258) 2024-09-13T17:57:54.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58faa62640 1 -- 172.21.15.111:0/1057909587 learned_addr learned my addr 172.21.15.111:0/1057909587 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:54.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58f9a60640 1 --2- 172.21.15.111:0/1057909587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58f4071050 0x7f58f41e3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58fa261640 1 --2- 172.21.15.111:0/1057909587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f406c6d0 0x7f58f40c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58faa62640 1 -- 172.21.15.111:0/1057909587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f406c6d0 msgr2=0x7f58f40c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58faa62640 1 --2- 172.21.15.111:0/1057909587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f406c6d0 0x7f58f40c43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58faa62640 1 -- 172.21.15.111:0/1057909587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58f4071050 msgr2=0x7f58f41e3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58faa62640 1 --2- 172.21.15.111:0/1057909587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58f4071050 0x7f58f41e3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58faa62640 1 -- 172.21.15.111:0/1057909587 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e400b770 con 0x7f58f4158b00 2024-09-13T17:57:54.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58faa62640 1 --2- 172.21.15.111:0/1057909587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f4158b00 0x7f58f41e12f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f58ec007e40 tx=0x7f58ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:54.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58eaffd640 1 -- 172.21.15.111:0/1057909587 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec014620 con 0x7f58f4158b00 2024-09-13T17:57:54.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58eaffd640 1 -- 172.21.15.111:0/1057909587 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ec00ee90 con 0x7f58f4158b00 2024-09-13T17:57:54.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58fc4c5640 1 -- 172.21.15.111:0/1057909587 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f415b8b0 con 0x7f58f4158b00 2024-09-13T17:57:54.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58eaffd640 1 -- 172.21.15.111:0/1057909587 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec0149e0 con 0x7f58f4158b00 2024-09-13T17:57:54.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.603+0000 7f58fc4c5640 1 -- 172.21.15.111:0/1057909587 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f4159680 con 0x7f58f4158b00 2024-09-13T17:57:54.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.604+0000 7f58fc4c5640 1 -- 172.21.15.111:0/1057909587 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f4159870 con 0x7f58f4158b00 2024-09-13T17:57:54.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.605+0000 7f58eaffd640 1 -- 172.21.15.111:0/1057909587 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f58ec014b40 con 0x7f58f4158b00 2024-09-13T17:57:54.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.605+0000 7f58eaffd640 1 --2- 172.21.15.111:0/1057909587 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f58d806ac00 0x7f58d806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.605+0000 7f58eaffd640 1 -- 172.21.15.111:0/1057909587 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f58ec089e60 con 0x7f58f4158b00 2024-09-13T17:57:54.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.605+0000 7f58fa261640 1 --2- 172.21.15.111:0/1057909587 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f58d806ac00 0x7f58d806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.605+0000 7f58fa261640 1 --2- 172.21.15.111:0/1057909587 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f58d806ac00 0x7f58d806d0c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f58f0007920 tx=0x7f58f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:54.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.609+0000 7f58eaffd640 1 -- 172.21.15.111:0/1057909587 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f58ec054e90 con 0x7f58f4158b00 2024-09-13T17:57:54.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.772+0000 7f58fc4c5640 1 -- 172.21.15.111:0/1057909587 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58f4158ee0 con 0x7f58f4158b00 2024-09-13T17:57:54.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.772+0000 7f58eaffd640 1 -- 172.21.15.111:0/1057909587 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f58ec0584b0 con 0x7f58f4158b00 2024-09-13T17:57:54.775 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:57:54.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.774+0000 7f58fc4c5640 1 -- 172.21.15.111:0/1057909587 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f58d806ac00 msgr2=0x7f58d806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.774+0000 7f58fc4c5640 1 --2- 172.21.15.111:0/1057909587 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f58d806ac00 0x7f58d806d0c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f58f0007920 tx=0x7f58f0008040 comp rx=0 tx=0).stop 2024-09-13T17:57:54.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.774+0000 7f58fc4c5640 1 -- 172.21.15.111:0/1057909587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f4158b00 msgr2=0x7f58f41e12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.774+0000 7f58fc4c5640 1 --2- 172.21.15.111:0/1057909587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f4158b00 0x7f58f41e12f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f58ec007e40 tx=0x7f58ec00e4b0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.774+0000 7f58fc4c5640 1 -- 172.21.15.111:0/1057909587 shutdown_connections 2024-09-13T17:57:54.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.774+0000 7f58fc4c5640 1 --2- 172.21.15.111:0/1057909587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f4158b00 0x7f58f41e12f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.774+0000 7f58fc4c5640 1 --2- 172.21.15.111:0/1057909587 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f58d806ac00 0x7f58d806d0c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.774+0000 7f58fc4c5640 1 --2- 172.21.15.111:0/1057909587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58f4071050 0x7f58f41e3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.774+0000 7f58fc4c5640 1 --2- 172.21.15.111:0/1057909587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f406c6d0 0x7f58f40c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.774+0000 7f58fc4c5640 1 -- 172.21.15.111:0/1057909587 >> 172.21.15.111:0/1057909587 conn(0x7f58f406a0b0 msgr2=0x7f58f4071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:54.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.775+0000 7f58fc4c5640 1 -- 172.21.15.111:0/1057909587 shutdown_connections 2024-09-13T17:57:54.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.775+0000 7f58fc4c5640 1 -- 172.21.15.111:0/1057909587 wait complete. 2024-09-13T17:57:54.788 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:57:54.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.886+0000 7f8ccb1d4640 1 Processor -- start 2024-09-13T17:57:54.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.886+0000 7f8ccb1d4640 1 -- start start 2024-09-13T17:57:54.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.886+0000 7f8ccb1d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc40d2760 0x7f8cc40cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.886+0000 7f8ccb1d4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8cc40cde20 0x7f8cc40d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.886+0000 7f8ccb1d4640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc40cd450 con 0x7f8cc415c500 2024-09-13T17:57:54.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.886+0000 7f8ccb1d4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc40d0cf0 con 0x7f8cc40cde20 2024-09-13T17:57:54.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.886+0000 7f8ccb1d4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc40d0fd0 con 0x7f8cc40d2760 2024-09-13T17:57:54.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.887+0000 7f8cc8f70640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f8cc415c500 0x7f8cc40d2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35602/0 (socket says 172.21.15.111:35602) 2024-09-13T17:57:54.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.887+0000 7f8cc8f70640 1 -- 172.21.15.111:0/781396276 learned_addr learned my addr 172.21.15.111:0/781396276 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:54.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.887+0000 7f8cc3fff640 1 --2- 172.21.15.111:0/781396276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc40d2760 0x7f8cc40cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.887+0000 7f8cc9771640 1 --2- 172.21.15.111:0/781396276 >> v2:172.21.15.188:3300/0 conn(0x7f8cc40cde20 0x7f8cc40d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.887+0000 7f8cc37fe640 1 -- 172.21.15.111:0/781396276 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722542827 0 0) 0x7f8cc40cd450 con 0x7f8cc415c500 2024-09-13T17:57:54.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.887+0000 7f8cc37fe640 1 -- 172.21.15.111:0/781396276 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cb0003520 con 0x7f8cc415c500 2024-09-13T17:57:54.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.887+0000 7f8cc3fff640 1 -- 172.21.15.111:0/781396276 >> v2:172.21.15.188:3300/0 conn(0x7f8cc40cde20 msgr2=0x7f8cc40d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:54.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.887+0000 7f8cc3fff640 1 --2- 172.21.15.111:0/781396276 >> v2:172.21.15.188:3300/0 conn(0x7f8cc40cde20 0x7f8cc40d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.887+0000 7f8cc37fe640 1 -- 172.21.15.111:0/781396276 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 485316445 0 0) 0x7f8cb0003520 con 0x7f8cc415c500 2024-09-13T17:57:54.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.887+0000 7f8cc3fff640 1 -- 172.21.15.111:0/781396276 >> v1:172.21.15.111:6789/0 conn(0x7f8cc415c500 legacy=0x7f8cc40d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.887+0000 7f8cc3fff640 1 -- 172.21.15.111:0/781396276 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc40cd0e0 con 0x7f8cc40d2760 2024-09-13T17:57:54.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.887+0000 7f8cc9771640 1 --2- 172.21.15.111:0/781396276 >> v2:172.21.15.188:3300/0 conn(0x7f8cc40cde20 0x7f8cc40d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:54.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.888+0000 7f8cc3fff640 1 --2- 172.21.15.111:0/781396276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc40d2760 0x7f8cc40cd8e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009d90 tx=0x7f8cb402a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e42cbf93dafb9752 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:54.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.888+0000 7f8cc37fe640 1 -- 172.21.15.111:0/781396276 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc001670 con 0x7f8cc40d2760 2024-09-13T17:57:54.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.888+0000 7f8cc37fe640 1 -- 172.21.15.111:0/781396276 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb402d050 con 0x7f8cc40d2760 2024-09-13T17:57:54.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.888+0000 7f8cc37fe640 1 -- 172.21.15.111:0/781396276 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4007740 con 0x7f8cc40d2760 2024-09-13T17:57:54.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.889+0000 7f8ccb1d4640 1 -- 172.21.15.111:0/781396276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc40d2760 msgr2=0x7f8cc40cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.889+0000 7f8ccb1d4640 1 --2- 172.21.15.111:0/781396276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc40d2760 0x7f8cc40cd8e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009d90 tx=0x7f8cb402a5f0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.889+0000 7f8ccb1d4640 1 -- 172.21.15.111:0/781396276 shutdown_connections 2024-09-13T17:57:54.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.889+0000 7f8ccb1d4640 1 --2- 172.21.15.111:0/781396276 >> v2:172.21.15.188:3300/0 conn(0x7f8cc40cde20 0x7f8cc40d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.889+0000 7f8ccb1d4640 1 --2- 172.21.15.111:0/781396276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc40d2760 0x7f8cc40cd8e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.889+0000 7f8ccb1d4640 1 -- 172.21.15.111:0/781396276 >> 172.21.15.111:0/781396276 conn(0x7f8cc4089f60 msgr2=0x7f8cc409b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:54.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.889+0000 7f8ccb1d4640 1 -- 172.21.15.111:0/781396276 shutdown_connections 2024-09-13T17:57:54.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.889+0000 7f8ccb1d4640 1 -- 172.21.15.111:0/781396276 wait complete. 2024-09-13T17:57:54.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.890+0000 7f8ccb1d4640 1 Processor -- start 2024-09-13T17:57:54.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.890+0000 7f8ccb1d4640 1 -- start start 2024-09-13T17:57:54.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8ccb1d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cc40cde20 0x7f8cc4157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8ccb1d4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cc40d2760 0x7f8cc415ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8ccb1d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc415c500 0x7f8cc415ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8ccb1d4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc4158cb0 con 0x7f8cc40d2760 2024-09-13T17:57:54.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8ccb1d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc41590d0 con 0x7f8cc40cde20 2024-09-13T17:57:54.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8ccb1d4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc4158030 con 0x7f8cc415c500 2024-09-13T17:57:54.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8cc3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cc40d2760 0x7f8cc415ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8cc3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cc40d2760 0x7f8cc415ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53266/0 (socket says 172.21.15.111:53266) 2024-09-13T17:57:54.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8cc3fff640 1 -- 172.21.15.111:0/2958735254 learned_addr learned my addr 172.21.15.111:0/2958735254 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:54.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8cc8f70640 1 --2- 172.21.15.111:0/2958735254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cc40cde20 0x7f8cc4157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8cc3fff640 1 -- 172.21.15.111:0/2958735254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc415c500 msgr2=0x7f8cc415ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8cc3fff640 1 --2- 172.21.15.111:0/2958735254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc415c500 0x7f8cc415ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8cc3fff640 1 -- 172.21.15.111:0/2958735254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cc40cde20 msgr2=0x7f8cc4157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8cc3fff640 1 --2- 172.21.15.111:0/2958735254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cc40cde20 0x7f8cc4157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8cc3fff640 1 -- 172.21.15.111:0/2958735254 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac009400 con 0x7f8cc40d2760 2024-09-13T17:57:54.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8cc3fff640 1 --2- 172.21.15.111:0/2958735254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cc40d2760 0x7f8cc415ccd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009d90 tx=0x7f8cb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:54.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8cc17fa640 1 -- 172.21.15.111:0/2958735254 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb403c830 con 0x7f8cc40d2760 2024-09-13T17:57:54.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8cc17fa640 1 -- 172.21.15.111:0/2958735254 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb403c990 con 0x7f8cc40d2760 2024-09-13T17:57:54.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8ccb1d4640 1 -- 172.21.15.111:0/2958735254 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb4009a70 con 0x7f8cc40d2760 2024-09-13T17:57:54.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.891+0000 7f8cc17fa640 1 -- 172.21.15.111:0/2958735254 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb403cd20 con 0x7f8cc40d2760 2024-09-13T17:57:54.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.892+0000 7f8ccb1d4640 1 -- 172.21.15.111:0/2958735254 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc41585f0 con 0x7f8cc40d2760 2024-09-13T17:57:54.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.893+0000 7f8ccb1d4640 1 -- 172.21.15.111:0/2958735254 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc4158df0 con 0x7f8cc40d2760 2024-09-13T17:57:54.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.893+0000 7f8cc17fa640 1 -- 172.21.15.111:0/2958735254 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8cb402ad50 con 0x7f8cc40d2760 2024-09-13T17:57:54.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.893+0000 7f8cc17fa640 1 --2- 172.21.15.111:0/2958735254 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8ca406ac00 0x7f8ca406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.893+0000 7f8cc17fa640 1 -- 172.21.15.111:0/2958735254 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8cb40aabb0 con 0x7f8cc40d2760 2024-09-13T17:57:54.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.893+0000 7f8cc8f70640 1 --2- 172.21.15.111:0/2958735254 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8ca406ac00 0x7f8ca406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.894+0000 7f8cc8f70640 1 --2- 172.21.15.111:0/2958735254 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8ca406ac00 0x7f8ca406d0c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc005880 tx=0x7f8cbc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:54.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:54.897+0000 7f8cc17fa640 1 -- 172.21.15.111:0/2958735254 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f8cc4158df0 con 0x7f8cc40d2760 2024-09-13T17:57:55.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.058+0000 7f8ccb1d4640 1 -- 172.21.15.111:0/2958735254 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8cc4158df0 con 0x7f8cc40d2760 2024-09-13T17:57:55.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.058+0000 7f8cc17fa640 1 -- 172.21.15.111:0/2958735254 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8cc4158df0 con 0x7f8cc40d2760 2024-09-13T17:57:55.061 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:57:55.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.060+0000 7f8ccb1d4640 1 -- 172.21.15.111:0/2958735254 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8ca406ac00 msgr2=0x7f8ca406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.060+0000 7f8ccb1d4640 1 --2- 172.21.15.111:0/2958735254 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8ca406ac00 0x7f8ca406d0c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc005880 tx=0x7f8cbc009040 comp rx=0 tx=0).stop 2024-09-13T17:57:55.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.060+0000 7f8ccb1d4640 1 -- 172.21.15.111:0/2958735254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cc40d2760 msgr2=0x7f8cc415ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.060+0000 7f8ccb1d4640 1 --2- 172.21.15.111:0/2958735254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cc40d2760 0x7f8cc415ccd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009d90 tx=0x7f8cb4004a40 comp rx=0 tx=0).stop 2024-09-13T17:57:55.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.060+0000 7f8ccb1d4640 1 -- 172.21.15.111:0/2958735254 shutdown_connections 2024-09-13T17:57:55.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.060+0000 7f8ccb1d4640 1 --2- 172.21.15.111:0/2958735254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc415c500 0x7f8cc415ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.060+0000 7f8ccb1d4640 1 --2- 172.21.15.111:0/2958735254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cc40d2760 0x7f8cc415ccd0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.060+0000 7f8ccb1d4640 1 --2- 172.21.15.111:0/2958735254 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8ca406ac00 0x7f8ca406d0c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.060+0000 7f8ccb1d4640 1 --2- 172.21.15.111:0/2958735254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cc40cde20 0x7f8cc4157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.060+0000 7f8ccb1d4640 1 -- 172.21.15.111:0/2958735254 >> 172.21.15.111:0/2958735254 conn(0x7f8cc4089f60 msgr2=0x7f8cc40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:55.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.061+0000 7f8ccb1d4640 1 -- 172.21.15.111:0/2958735254 shutdown_connections 2024-09-13T17:57:55.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.061+0000 7f8ccb1d4640 1 -- 172.21.15.111:0/2958735254 wait complete. 2024-09-13T17:57:55.074 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:57:55.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.171+0000 7fd3a415e640 1 Processor -- start 2024-09-13T17:57:55.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.171+0000 7fd3a415e640 1 -- start start 2024-09-13T17:57:55.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.172+0000 7fd3a415e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd39c159310 0x7fd39c0ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.172+0000 7fd3a415e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd39c0b2b40 0x7fd39c0ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.172+0000 7fd3a415e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd39c0ab680 0x7fd39c0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.172+0000 7fd3a415e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd39c0ad940 con 0x7fd39c0b2b40 2024-09-13T17:57:55.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.172+0000 7fd3a415e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd39c0ccbe0 con 0x7fd39c159310 2024-09-13T17:57:55.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.172+0000 7fd3a415e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c0b1810 con 0x7fd39c0ab680 2024-09-13T17:57:55.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.172+0000 7fd3a16f9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd39c0b2b40 0x7fd39c0ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.172+0000 7fd3a16f9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd39c0b2b40 0x7fd39c0ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53278/0 (socket says 172.21.15.111:53278) 2024-09-13T17:57:55.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.172+0000 7fd3a16f9640 1 -- 172.21.15.111:0/1988769218 learned_addr learned my addr 172.21.15.111:0/1988769218 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:55.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.172+0000 7fd3a1efa640 1 --2- 172.21.15.111:0/1988769218 >> v2:172.21.15.188:3300/0 conn(0x7fd39c159310 0x7fd39c0ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.172+0000 7fd3a26fb640 1 --2- 172.21.15.111:0/1988769218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd39c0ab680 0x7fd39c0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.172+0000 7fd3a16f9640 1 -- 172.21.15.111:0/1988769218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd39c0ab680 msgr2=0x7fd39c0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:55.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.172+0000 7fd3a16f9640 1 --2- 172.21.15.111:0/1988769218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd39c0ab680 0x7fd39c0ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.172+0000 7fd3a16f9640 1 -- 172.21.15.111:0/1988769218 >> v2:172.21.15.188:3300/0 conn(0x7fd39c159310 msgr2=0x7fd39c0ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:55.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.172+0000 7fd3a16f9640 1 --2- 172.21.15.111:0/1988769218 >> v2:172.21.15.188:3300/0 conn(0x7fd39c159310 0x7fd39c0ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.172+0000 7fd3a16f9640 1 -- 172.21.15.111:0/1988769218 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39c06d9e0 con 0x7fd39c0b2b40 2024-09-13T17:57:55.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.172+0000 7fd3a1efa640 1 --2- 172.21.15.111:0/1988769218 >> v2:172.21.15.188:3300/0 conn(0x7fd39c159310 0x7fd39c0ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:55.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.173+0000 7fd3a16f9640 1 --2- 172.21.15.111:0/1988769218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd39c0b2b40 0x7fd39c0ab140 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0cac90 tx=0x7fd38c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9f6b3057ef81ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:55.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.173+0000 7fd3a0ef8640 1 -- 172.21.15.111:0/1988769218 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c02ae90 con 0x7fd39c0b2b40 2024-09-13T17:57:55.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.173+0000 7fd3a0ef8640 1 -- 172.21.15.111:0/1988769218 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38c032440 con 0x7fd39c0b2b40 2024-09-13T17:57:55.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.173+0000 7fd3a0ef8640 1 -- 172.21.15.111:0/1988769218 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c0327d0 con 0x7fd39c0b2b40 2024-09-13T17:57:55.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.174+0000 7fd3a415e640 1 -- 172.21.15.111:0/1988769218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd39c0b2b40 msgr2=0x7fd39c0ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.174+0000 7fd3a415e640 1 --2- 172.21.15.111:0/1988769218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd39c0b2b40 0x7fd39c0ab140 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0cac90 tx=0x7fd38c02a140 comp rx=0 tx=0).stop 2024-09-13T17:57:55.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.174+0000 7fd3a415e640 1 -- 172.21.15.111:0/1988769218 shutdown_connections 2024-09-13T17:57:55.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.174+0000 7fd3a415e640 1 --2- 172.21.15.111:0/1988769218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd39c0ab680 0x7fd39c0ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.174+0000 7fd3a415e640 1 --2- 172.21.15.111:0/1988769218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd39c0b2b40 0x7fd39c0ab140 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.174+0000 7fd3a415e640 1 --2- 172.21.15.111:0/1988769218 >> v2:172.21.15.188:3300/0 conn(0x7fd39c159310 0x7fd39c0ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.174+0000 7fd3a415e640 1 -- 172.21.15.111:0/1988769218 >> 172.21.15.111:0/1988769218 conn(0x7fd39c06a0b0 msgr2=0x7fd39c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:55.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.174+0000 7fd3a415e640 1 -- 172.21.15.111:0/1988769218 shutdown_connections 2024-09-13T17:57:55.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.174+0000 7fd3a415e640 1 -- 172.21.15.111:0/1988769218 wait complete. 2024-09-13T17:57:55.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.175+0000 7fd3a415e640 1 Processor -- start 2024-09-13T17:57:55.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.175+0000 7fd3a415e640 1 -- start start 2024-09-13T17:57:55.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.176+0000 7fd3a415e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd39c0ab680 0x7fd39c1e4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.176+0000 7fd3a415e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd39c0b2b40 0x7fd39c150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.176+0000 7fd3a415e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd39c159310 0x7fd39c0acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.176+0000 7fd3a415e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c0c4ad0 con 0x7fd39c0ab680 2024-09-13T17:57:55.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.176+0000 7fd3a415e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c151d20 con 0x7fd39c0b2b40 2024-09-13T17:57:55.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.176+0000 7fd3a415e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c1e5880 con 0x7fd39c159310 2024-09-13T17:57:55.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.176+0000 7fd3a1efa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd39c0ab680 0x7fd39c1e4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.176+0000 7fd3a1efa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd39c0ab680 0x7fd39c1e4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53290/0 (socket says 172.21.15.111:53290) 2024-09-13T17:57:55.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.176+0000 7fd3a1efa640 1 -- 172.21.15.111:0/2635485629 learned_addr learned my addr 172.21.15.111:0/2635485629 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:55.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.176+0000 7fd3a1efa640 1 -- 172.21.15.111:0/2635485629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd39c159310 msgr2=0x7fd39c0acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.176+0000 7fd3a26fb640 1 --2- 172.21.15.111:0/2635485629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd39c159310 0x7fd39c0acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.176+0000 7fd3a16f9640 1 --2- 172.21.15.111:0/2635485629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd39c0b2b40 0x7fd39c150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.176+0000 7fd3a1efa640 1 --2- 172.21.15.111:0/2635485629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd39c159310 0x7fd39c0acf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.176+0000 7fd3a1efa640 1 -- 172.21.15.111:0/2635485629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd39c0b2b40 msgr2=0x7fd39c150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.176+0000 7fd3a1efa640 1 --2- 172.21.15.111:0/2635485629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd39c0b2b40 0x7fd39c150360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.176+0000 7fd3a1efa640 1 -- 172.21.15.111:0/2635485629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c0094d0 con 0x7fd39c0ab680 2024-09-13T17:57:55.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.176+0000 7fd3a1efa640 1 --2- 172.21.15.111:0/2635485629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd39c0ab680 0x7fd39c1e4fa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd39800c820 tx=0x7fd39800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:55.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.177+0000 7fd3927fc640 1 -- 172.21.15.111:0/2635485629 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd398007a30 con 0x7fd39c0ab680 2024-09-13T17:57:55.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.177+0000 7fd3927fc640 1 -- 172.21.15.111:0/2635485629 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd398007b90 con 0x7fd39c0ab680 2024-09-13T17:57:55.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.177+0000 7fd3927fc640 1 -- 172.21.15.111:0/2635485629 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39800ce10 con 0x7fd39c0ab680 2024-09-13T17:57:55.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.177+0000 7fd3a415e640 1 -- 172.21.15.111:0/2635485629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39c1508a0 con 0x7fd39c0ab680 2024-09-13T17:57:55.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.177+0000 7fd3a415e640 1 -- 172.21.15.111:0/2635485629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39c152210 con 0x7fd39c0ab680 2024-09-13T17:57:55.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.178+0000 7fd3a415e640 1 -- 172.21.15.111:0/2635485629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd39c0cee10 con 0x7fd39c0ab680 2024-09-13T17:57:55.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.178+0000 7fd3927fc640 1 -- 172.21.15.111:0/2635485629 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd3980113f0 con 0x7fd39c0ab680 2024-09-13T17:57:55.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.179+0000 7fd3927fc640 1 --2- 172.21.15.111:0/2635485629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd38006ac00 0x7fd38006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.179+0000 7fd3a16f9640 1 --2- 172.21.15.111:0/2635485629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd38006ac00 0x7fd38006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.179+0000 7fd3927fc640 1 -- 172.21.15.111:0/2635485629 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd398089690 con 0x7fd39c0ab680 2024-09-13T17:57:55.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.179+0000 7fd3a16f9640 1 --2- 172.21.15.111:0/2635485629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd38006ac00 0x7fd38006d0c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd38c034000 tx=0x7fd38c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:55.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.183+0000 7fd3927fc640 1 -- 172.21.15.111:0/2635485629 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fd398054580 con 0x7fd39c0ab680 2024-09-13T17:57:55.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.343+0000 7fd3a415e640 1 -- 172.21.15.111:0/2635485629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd39c06e4b0 con 0x7fd39c0ab680 2024-09-13T17:57:55.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.343+0000 7fd3927fc640 1 -- 172.21.15.111:0/2635485629 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd398057ba0 con 0x7fd39c0ab680 2024-09-13T17:57:55.346 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:57:55.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.345+0000 7fd3a415e640 1 -- 172.21.15.111:0/2635485629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd38006ac00 msgr2=0x7fd38006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.345+0000 7fd3a415e640 1 --2- 172.21.15.111:0/2635485629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd38006ac00 0x7fd38006d0c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd38c034000 tx=0x7fd38c02e000 comp rx=0 tx=0).stop 2024-09-13T17:57:55.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.345+0000 7fd3a415e640 1 -- 172.21.15.111:0/2635485629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd39c0ab680 msgr2=0x7fd39c1e4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.345+0000 7fd3a415e640 1 --2- 172.21.15.111:0/2635485629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd39c0ab680 0x7fd39c1e4fa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd39800c820 tx=0x7fd39800cce0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.345+0000 7fd3a415e640 1 -- 172.21.15.111:0/2635485629 shutdown_connections 2024-09-13T17:57:55.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.345+0000 7fd3a415e640 1 --2- 172.21.15.111:0/2635485629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd39c159310 0x7fd39c0acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.345+0000 7fd3a415e640 1 --2- 172.21.15.111:0/2635485629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd38006ac00 0x7fd38006d0c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.345+0000 7fd3a415e640 1 --2- 172.21.15.111:0/2635485629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd39c0b2b40 0x7fd39c150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.345+0000 7fd3a415e640 1 --2- 172.21.15.111:0/2635485629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd39c0ab680 0x7fd39c1e4fa0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.345+0000 7fd3a415e640 1 -- 172.21.15.111:0/2635485629 >> 172.21.15.111:0/2635485629 conn(0x7fd39c06a0b0 msgr2=0x7fd39c0caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:55.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.345+0000 7fd3a415e640 1 -- 172.21.15.111:0/2635485629 shutdown_connections 2024-09-13T17:57:55.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.345+0000 7fd3a415e640 1 -- 172.21.15.111:0/2635485629 wait complete. 2024-09-13T17:57:55.359 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:57:55.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.453+0000 7f253b6c9640 1 Processor -- start 2024-09-13T17:57:55.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.453+0000 7f253b6c9640 1 -- start start 2024-09-13T17:57:55.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.454+0000 7f253b6c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2534071d30 0x7f2534070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.454+0000 7f253b6c9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2534072110 0x7f25340b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.454+0000 7f253b6c9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25340abdf0 con 0x7f25340c8920 2024-09-13T17:57:55.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.454+0000 7f253b6c9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25340ab1e0 con 0x7f2534072110 2024-09-13T17:57:55.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.454+0000 7f253b6c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25340ab4c0 con 0x7f2534071d30 2024-09-13T17:57:55.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.454+0000 7f2539c66640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f25340c8920 0x7f25340ac280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35616/0 (socket says 172.21.15.111:35616) 2024-09-13T17:57:55.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.454+0000 7f2539c66640 1 -- 172.21.15.111:0/1349482386 learned_addr learned my addr 172.21.15.111:0/1349482386 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:55.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.454+0000 7f2539465640 1 --2- 172.21.15.111:0/1349482386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2534071d30 0x7f2534070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.454+0000 7f2538c64640 1 --2- 172.21.15.111:0/1349482386 >> v2:172.21.15.188:3300/0 conn(0x7f2534072110 0x7f25340b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.454+0000 7f252bfff640 1 -- 172.21.15.111:0/1349482386 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828887130 0 0) 0x7f25340abdf0 con 0x7f25340c8920 2024-09-13T17:57:55.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.454+0000 7f252bfff640 1 -- 172.21.15.111:0/1349482386 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2520003520 con 0x7f25340c8920 2024-09-13T17:57:55.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f252bfff640 1 -- 172.21.15.111:0/1349482386 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 671893729 0 0) 0x7f2520003520 con 0x7f25340c8920 2024-09-13T17:57:55.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f252bfff640 1 -- 172.21.15.111:0/1349482386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2534071d30 msgr2=0x7f2534070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:55.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f252bfff640 1 --2- 172.21.15.111:0/1349482386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2534071d30 0x7f2534070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f252bfff640 1 -- 172.21.15.111:0/1349482386 >> v2:172.21.15.188:3300/0 conn(0x7f2534072110 msgr2=0x7f25340b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:55.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f252bfff640 1 --2- 172.21.15.111:0/1349482386 >> v2:172.21.15.188:3300/0 conn(0x7f2534072110 0x7f25340b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f252bfff640 1 -- 172.21.15.111:0/1349482386 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25340ab7a0 con 0x7f25340c8920 2024-09-13T17:57:55.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f252bfff640 1 -- 172.21.15.111:0/1349482386 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f252c0019f0 con 0x7f25340c8920 2024-09-13T17:57:55.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f2538c64640 1 --2- 172.21.15.111:0/1349482386 >> v2:172.21.15.188:3300/0 conn(0x7f2534072110 0x7f25340b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:55.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f252bfff640 1 -- 172.21.15.111:0/1349482386 >> v1:172.21.15.111:6789/0 conn(0x7f25340c8920 legacy=0x7f25340ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f252bfff640 1 --2- 172.21.15.111:0/1349482386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2534071d30 0x7f2520003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f252bfff640 1 --2- 172.21.15.111:0/1349482386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2520003ef0 0x7f25200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f252bfff640 1 --2- 172.21.15.111:0/1349482386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2520004d00 0x7f2520005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f252bfff640 1 -- 172.21.15.111:0/1349482386 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25340ab1e0 con 0x7f2520003ef0 2024-09-13T17:57:55.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f252bfff640 1 -- 172.21.15.111:0/1349482386 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25340ab4c0 con 0x7f2520004d00 2024-09-13T17:57:55.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f252bfff640 1 -- 172.21.15.111:0/1349482386 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252000b980 con 0x7f2534071d30 2024-09-13T17:57:55.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f2538c64640 1 --2- 172.21.15.111:0/1349482386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2520003ef0 0x7f25200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f2539465640 1 --2- 172.21.15.111:0/1349482386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2534071d30 0x7f2520003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f2539c66640 1 --2- 172.21.15.111:0/1349482386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2520004d00 0x7f2520005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f2538c64640 1 -- 172.21.15.111:0/1349482386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2534071d30 msgr2=0x7f2520003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:55.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f2538c64640 1 --2- 172.21.15.111:0/1349482386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2534071d30 0x7f2520003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f2538c64640 1 -- 172.21.15.111:0/1349482386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2520004d00 msgr2=0x7f2520005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:55.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f2538c64640 1 --2- 172.21.15.111:0/1349482386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2520004d00 0x7f2520005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.455+0000 7f2538c64640 1 -- 172.21.15.111:0/1349482386 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2520003520 con 0x7f2520003ef0 2024-09-13T17:57:55.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.456+0000 7f2538c64640 1 --2- 172.21.15.111:0/1349482386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2520003ef0 0x7f25200047c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2534072530 tx=0x7f25200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0f4020363b4268 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:55.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.456+0000 7f252bfff640 1 -- 172.21.15.111:0/1349482386 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2524004020 con 0x7f2520003ef0 2024-09-13T17:57:55.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.456+0000 7f252bfff640 1 -- 172.21.15.111:0/1349482386 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252402d050 con 0x7f2520003ef0 2024-09-13T17:57:55.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.456+0000 7f252bfff640 1 -- 172.21.15.111:0/1349482386 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25240043a0 con 0x7f2520003ef0 2024-09-13T17:57:55.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.456+0000 7f253b6c9640 1 -- 172.21.15.111:0/1349482386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2520003ef0 msgr2=0x7f25200047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.456+0000 7f253b6c9640 1 --2- 172.21.15.111:0/1349482386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2520003ef0 0x7f25200047c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2534072530 tx=0x7f25200057f0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.456+0000 7f2539c66640 1 -- 172.21.15.111:0/1349482386 reap_dead start 2024-09-13T17:57:55.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.457+0000 7f253b6c9640 1 -- 172.21.15.111:0/1349482386 shutdown_connections 2024-09-13T17:57:55.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.457+0000 7f253b6c9640 1 -- 172.21.15.111:0/1349482386 >> 172.21.15.111:0/1349482386 conn(0x7f2534069ed0 msgr2=0x7f253406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:55.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.457+0000 7f253b6c9640 1 -- 172.21.15.111:0/1349482386 shutdown_connections 2024-09-13T17:57:55.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.457+0000 7f253b6c9640 1 -- 172.21.15.111:0/1349482386 wait complete. 2024-09-13T17:57:55.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.458+0000 7f253b6c9640 1 Processor -- start 2024-09-13T17:57:55.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.458+0000 7f253b6c9640 1 -- start start 2024-09-13T17:57:55.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.458+0000 7f253b6c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2520003ef0 0x7f253415aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.458+0000 7f253b6c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25340ad530 0x7f2534153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.458+0000 7f253b6c9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25340af250 0x7f25341f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.458+0000 7f253b6c9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253415c910 con 0x7f25340af250 2024-09-13T17:57:55.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.458+0000 7f253b6c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25340c4e10 con 0x7f2520003ef0 2024-09-13T17:57:55.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.458+0000 7f253b6c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253415a480 con 0x7f25340ad530 2024-09-13T17:57:55.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.459+0000 7f2539c66640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25340af250 0x7f25341f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.459+0000 7f2539c66640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25340af250 0x7f25341f3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53298/0 (socket says 172.21.15.111:53298) 2024-09-13T17:57:55.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.459+0000 7f2539c66640 1 -- 172.21.15.111:0/1912408091 learned_addr learned my addr 172.21.15.111:0/1912408091 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:55.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.459+0000 7f2539465640 1 --2- 172.21.15.111:0/1912408091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2520003ef0 0x7f253415aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.459+0000 7f2538c64640 1 --2- 172.21.15.111:0/1912408091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25340ad530 0x7f2534153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.459+0000 7f2539c66640 1 -- 172.21.15.111:0/1912408091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25340ad530 msgr2=0x7f2534153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.459+0000 7f2539c66640 1 --2- 172.21.15.111:0/1912408091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25340ad530 0x7f2534153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.459+0000 7f2539c66640 1 -- 172.21.15.111:0/1912408091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2520003ef0 msgr2=0x7f253415aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.459+0000 7f2539c66640 1 --2- 172.21.15.111:0/1912408091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2520003ef0 0x7f253415aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.459+0000 7f2539c66640 1 -- 172.21.15.111:0/1912408091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25340c4b30 con 0x7f25340af250 2024-09-13T17:57:55.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.459+0000 7f2539c66640 1 --2- 172.21.15.111:0/1912408091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25340af250 0x7f25341f3210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f25340c8d00 tx=0x7f252c007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:55.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.459+0000 7f2529ffb640 1 -- 172.21.15.111:0/1912408091 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c00ab10 con 0x7f25340af250 2024-09-13T17:57:55.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.459+0000 7f2529ffb640 1 -- 172.21.15.111:0/1912408091 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252c00ac70 con 0x7f25340af250 2024-09-13T17:57:55.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.459+0000 7f253b6c9640 1 -- 172.21.15.111:0/1912408091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25341f3de0 con 0x7f25340af250 2024-09-13T17:57:55.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.459+0000 7f2529ffb640 1 -- 172.21.15.111:0/1912408091 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c0133f0 con 0x7f25340af250 2024-09-13T17:57:55.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.459+0000 7f253b6c9640 1 -- 172.21.15.111:0/1912408091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25341f3750 con 0x7f25340af250 2024-09-13T17:57:55.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.460+0000 7f253b6c9640 1 -- 172.21.15.111:0/1912408091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25340b1b70 con 0x7f25340af250 2024-09-13T17:57:55.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.461+0000 7f2529ffb640 1 -- 172.21.15.111:0/1912408091 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f252c013550 con 0x7f25340af250 2024-09-13T17:57:55.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.461+0000 7f2529ffb640 1 --2- 172.21.15.111:0/1912408091 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f251806ac00 0x7f251806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.461+0000 7f2529ffb640 1 -- 172.21.15.111:0/1912408091 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f252c089b90 con 0x7f25340af250 2024-09-13T17:57:55.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.461+0000 7f2539465640 1 --2- 172.21.15.111:0/1912408091 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f251806ac00 0x7f251806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.461+0000 7f2539465640 1 --2- 172.21.15.111:0/1912408091 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f251806ac00 0x7f251806d0c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2530009a60 tx=0x7f2530009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:55.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.465+0000 7f2529ffb640 1 -- 172.21.15.111:0/1912408091 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f252c054a80 con 0x7f25340af250 2024-09-13T17:57:55.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.660+0000 7f253b6c9640 1 -- 172.21.15.111:0/1912408091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f253415a5f0 con 0x7f25340af250 2024-09-13T17:57:55.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.660+0000 7f2529ffb640 1 -- 172.21.15.111:0/1912408091 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4742 (secure 0 0 0) 0x7f252c0580a0 con 0x7f25340af250 2024-09-13T17:57:55.663 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:55.663 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4001406804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":4001406804},{"type":"v1","addr":"172.21.15.188:6835","nonce":4001406804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2565466103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2565466103},{"type":"v1","addr":"172.21.15.111:6837","nonce":2565466103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/1252690799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1252690799},{"type":"v1","addr":"172.21.15.111:6836","nonce":1252690799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:57:25.702997+0000","modified":"2024-09-13T17:57:46.351493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/1266310843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1266310843},{"type":"v1","addr":"172.21.15.188:6834","nonce":1266310843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:57:55.664 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-09-13T17:57:55.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.662+0000 7f253b6c9640 1 -- 172.21.15.111:0/1912408091 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f251806ac00 msgr2=0x7f251806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.662+0000 7f253b6c9640 1 --2- 172.21.15.111:0/1912408091 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f251806ac00 0x7f251806d0c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2530009a60 tx=0x7f2530009190 comp rx=0 tx=0).stop 2024-09-13T17:57:55.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.662+0000 7f253b6c9640 1 -- 172.21.15.111:0/1912408091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25340af250 msgr2=0x7f25341f3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.662+0000 7f253b6c9640 1 --2- 172.21.15.111:0/1912408091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25340af250 0x7f25341f3210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f25340c8d00 tx=0x7f252c007b10 comp rx=0 tx=0).stop 2024-09-13T17:57:55.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.662+0000 7f253b6c9640 1 -- 172.21.15.111:0/1912408091 shutdown_connections 2024-09-13T17:57:55.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.662+0000 7f253b6c9640 1 --2- 172.21.15.111:0/1912408091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25340af250 0x7f25341f3210 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.662+0000 7f253b6c9640 1 --2- 172.21.15.111:0/1912408091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25340ad530 0x7f2534153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.662+0000 7f253b6c9640 1 --2- 172.21.15.111:0/1912408091 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f251806ac00 0x7f251806d0c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.662+0000 7f253b6c9640 1 --2- 172.21.15.111:0/1912408091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2520003ef0 0x7f253415aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.662+0000 7f253b6c9640 1 -- 172.21.15.111:0/1912408091 >> 172.21.15.111:0/1912408091 conn(0x7f2534069ed0 msgr2=0x7f25340c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:55.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.662+0000 7f253b6c9640 1 -- 172.21.15.111:0/1912408091 shutdown_connections 2024-09-13T17:57:55.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.662+0000 7f253b6c9640 1 -- 172.21.15.111:0/1912408091 wait complete. 2024-09-13T17:57:55.675 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:57:55.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.769+0000 7ff8a47bc640 1 Processor -- start 2024-09-13T17:57:55.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.769+0000 7ff8a47bc640 1 -- start start 2024-09-13T17:57:55.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.770+0000 7ff8a47bc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff89c15a2e0 0x7ff89c0c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.770+0000 7ff8a47bc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89c0c9f80 0x7ff89c0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.770+0000 7ff8a47bc640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff89c0c8920 con 0x7ff89c0cb220 2024-09-13T17:57:55.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.770+0000 7ff8a47bc640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff89c0cb650 con 0x7ff89c15a2e0 2024-09-13T17:57:55.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.770+0000 7ff8a47bc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c15a6c0 con 0x7ff89c0c9f80 2024-09-13T17:57:55.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.770+0000 7ff8a2558640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff89c15a2e0 0x7ff89c0c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.770+0000 7ff8a2d59640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff89c0cb220 0x7ff89c0ce260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35628/0 (socket says 172.21.15.111:35628) 2024-09-13T17:57:55.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.770+0000 7ff8a2d59640 1 -- 172.21.15.111:0/382161591 learned_addr learned my addr 172.21.15.111:0/382161591 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:55.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.770+0000 7ff8a1d57640 1 --2- 172.21.15.111:0/382161591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89c0c9f80 0x7ff89c0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.771+0000 7ff8a2558640 1 -- 172.21.15.111:0/382161591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89c0c9f80 msgr2=0x7ff89c0cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:55.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.771+0000 7ff8a2558640 1 --2- 172.21.15.111:0/382161591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89c0c9f80 0x7ff89c0cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.771+0000 7ff8a2558640 1 -- 172.21.15.111:0/382161591 >> v1:172.21.15.111:6789/0 conn(0x7ff89c0cb220 legacy=0x7ff89c0ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.771+0000 7ff8a2558640 1 -- 172.21.15.111:0/382161591 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89c0ca500 con 0x7ff89c15a2e0 2024-09-13T17:57:55.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.771+0000 7ff8a1556640 1 -- 172.21.15.111:0/382161591 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472884840 0 0) 0x7ff89c0c8920 con 0x7ff89c0cb220 2024-09-13T17:57:55.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.771+0000 7ff8a1d57640 1 --2- 172.21.15.111:0/382161591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89c0c9f80 0x7ff89c0cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:55.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.771+0000 7ff8a2558640 1 --2- 172.21.15.111:0/382161591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff89c15a2e0 0x7ff89c0c9a40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff8980097a0 tx=0x7ff89802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224c20fd5ee3a69c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:55.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.771+0000 7ff8a1556640 1 -- 172.21.15.111:0/382161591 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff898032440 con 0x7ff89c15a2e0 2024-09-13T17:57:55.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.771+0000 7ff8a1556640 1 -- 172.21.15.111:0/382161591 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8980325a0 con 0x7ff89c15a2e0 2024-09-13T17:57:55.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.771+0000 7ff8a1556640 1 -- 172.21.15.111:0/382161591 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff898032930 con 0x7ff89c15a2e0 2024-09-13T17:57:55.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.772+0000 7ff8a47bc640 1 -- 172.21.15.111:0/382161591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff89c15a2e0 msgr2=0x7ff89c0c9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.772+0000 7ff8a47bc640 1 --2- 172.21.15.111:0/382161591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff89c15a2e0 0x7ff89c0c9a40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff8980097a0 tx=0x7ff89802a180 comp rx=0 tx=0).stop 2024-09-13T17:57:55.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.772+0000 7ff8a47bc640 1 -- 172.21.15.111:0/382161591 shutdown_connections 2024-09-13T17:57:55.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.772+0000 7ff8a47bc640 1 --2- 172.21.15.111:0/382161591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89c0c9f80 0x7ff89c0cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.772+0000 7ff8a47bc640 1 --2- 172.21.15.111:0/382161591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff89c15a2e0 0x7ff89c0c9a40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.772+0000 7ff8a47bc640 1 -- 172.21.15.111:0/382161591 >> 172.21.15.111:0/382161591 conn(0x7ff89c069ed0 msgr2=0x7ff89c06a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:55.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.772+0000 7ff8a47bc640 1 -- 172.21.15.111:0/382161591 shutdown_connections 2024-09-13T17:57:55.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.772+0000 7ff8a47bc640 1 -- 172.21.15.111:0/382161591 wait complete. 2024-09-13T17:57:55.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.773+0000 7ff8a47bc640 1 Processor -- start 2024-09-13T17:57:55.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.773+0000 7ff8a47bc640 1 -- start start 2024-09-13T17:57:55.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a47bc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff89c0c9f80 0x7ff89c1ed000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a47bc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff89c0cb220 0x7ff89c1efc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a47bc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89c15a2e0 0x7ff89c0c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a47bc640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c1f0ba0 con 0x7ff89c0cb220 2024-09-13T17:57:55.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a47bc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c1f05e0 con 0x7ff89c0c9f80 2024-09-13T17:57:55.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a47bc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c1f08c0 con 0x7ff89c15a2e0 2024-09-13T17:57:55.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a1d57640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff89c0cb220 0x7ff89c1efc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a1d57640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff89c0cb220 0x7ff89c1efc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53314/0 (socket says 172.21.15.111:53314) 2024-09-13T17:57:55.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a1d57640 1 -- 172.21.15.111:0/3767624646 learned_addr learned my addr 172.21.15.111:0/3767624646 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:55.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a2558640 1 --2- 172.21.15.111:0/3767624646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff89c0c9f80 0x7ff89c1ed000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a2d59640 1 --2- 172.21.15.111:0/3767624646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89c15a2e0 0x7ff89c0c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a1d57640 1 -- 172.21.15.111:0/3767624646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89c15a2e0 msgr2=0x7ff89c0c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a1d57640 1 --2- 172.21.15.111:0/3767624646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89c15a2e0 0x7ff89c0c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a1d57640 1 -- 172.21.15.111:0/3767624646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff89c0c9f80 msgr2=0x7ff89c1ed000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a1d57640 1 --2- 172.21.15.111:0/3767624646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff89c0c9f80 0x7ff89c1ed000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a1d57640 1 -- 172.21.15.111:0/3767624646 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff898009400 con 0x7ff89c0cb220 2024-09-13T17:57:55.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a2d59640 1 --2- 172.21.15.111:0/3767624646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89c15a2e0 0x7ff89c0c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:55.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.774+0000 7ff8a1d57640 1 --2- 172.21.15.111:0/3767624646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff89c0cb220 0x7ff89c1efc20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00c7f0 tx=0x7ff88c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:55.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.775+0000 7ff892ffd640 1 -- 172.21.15.111:0/3767624646 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c007a30 con 0x7ff89c0cb220 2024-09-13T17:57:55.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.775+0000 7ff892ffd640 1 -- 172.21.15.111:0/3767624646 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c007b90 con 0x7ff89c0cb220 2024-09-13T17:57:55.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.775+0000 7ff8a47bc640 1 -- 172.21.15.111:0/3767624646 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff89c15b9f0 con 0x7ff89c0cb220 2024-09-13T17:57:55.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.775+0000 7ff892ffd640 1 -- 172.21.15.111:0/3767624646 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c00ce40 con 0x7ff89c0cb220 2024-09-13T17:57:55.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.775+0000 7ff8a47bc640 1 -- 172.21.15.111:0/3767624646 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff89c1ed540 con 0x7ff89c0cb220 2024-09-13T17:57:55.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.776+0000 7ff8a47bc640 1 -- 172.21.15.111:0/3767624646 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff89c15bf10 con 0x7ff89c0cb220 2024-09-13T17:57:55.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.776+0000 7ff892ffd640 1 -- 172.21.15.111:0/3767624646 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff88c0113f0 con 0x7ff89c0cb220 2024-09-13T17:57:55.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.776+0000 7ff892ffd640 1 --2- 172.21.15.111:0/3767624646 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff88006ac00 0x7ff88006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:55.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.777+0000 7ff892ffd640 1 -- 172.21.15.111:0/3767624646 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff88c089800 con 0x7ff89c0cb220 2024-09-13T17:57:55.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.777+0000 7ff8a2558640 1 --2- 172.21.15.111:0/3767624646 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff88006ac00 0x7ff88006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:55.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.777+0000 7ff8a2558640 1 --2- 172.21.15.111:0/3767624646 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff88006ac00 0x7ff88006d0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff898034000 tx=0x7ff89802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:55.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.781+0000 7ff892ffd640 1 -- 172.21.15.111:0/3767624646 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff89c15bf10 con 0x7ff89c0cb220 2024-09-13T17:57:55.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.944+0000 7ff8a47bc640 1 -- 172.21.15.111:0/3767624646 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff89c15bf10 con 0x7ff89c0cb220 2024-09-13T17:57:55.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.945+0000 7ff892ffd640 1 -- 172.21.15.111:0/3767624646 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7ff89c15bf10 con 0x7ff89c0cb220 2024-09-13T17:57:55.948 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:55.948 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:30.065235+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:57:55.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.947+0000 7ff8a47bc640 1 -- 172.21.15.111:0/3767624646 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff88006ac00 msgr2=0x7ff88006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.947+0000 7ff8a47bc640 1 --2- 172.21.15.111:0/3767624646 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff88006ac00 0x7ff88006d0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff898034000 tx=0x7ff89802e000 comp rx=0 tx=0).stop 2024-09-13T17:57:55.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.947+0000 7ff8a47bc640 1 -- 172.21.15.111:0/3767624646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff89c0cb220 msgr2=0x7ff89c1efc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.947+0000 7ff8a47bc640 1 --2- 172.21.15.111:0/3767624646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff89c0cb220 0x7ff89c1efc20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00c7f0 tx=0x7ff88c00cbf0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.947+0000 7ff8a47bc640 1 -- 172.21.15.111:0/3767624646 shutdown_connections 2024-09-13T17:57:55.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.947+0000 7ff8a47bc640 1 --2- 172.21.15.111:0/3767624646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89c15a2e0 0x7ff89c0c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.947+0000 7ff8a47bc640 1 --2- 172.21.15.111:0/3767624646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff89c0cb220 0x7ff89c1efc20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.947+0000 7ff8a47bc640 1 --2- 172.21.15.111:0/3767624646 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff88006ac00 0x7ff88006d0c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.947+0000 7ff8a47bc640 1 --2- 172.21.15.111:0/3767624646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff89c0c9f80 0x7ff89c1ed000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.947+0000 7ff8a47bc640 1 -- 172.21.15.111:0/3767624646 >> 172.21.15.111:0/3767624646 conn(0x7ff89c069ed0 msgr2=0x7ff89c0af200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:55.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.947+0000 7ff8a47bc640 1 -- 172.21.15.111:0/3767624646 shutdown_connections 2024-09-13T17:57:55.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:55.947+0000 7ff8a47bc640 1 -- 172.21.15.111:0/3767624646 wait complete. 2024-09-13T17:57:55.961 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:57:56.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.055+0000 7f9dd3f7d640 1 Processor -- start 2024-09-13T17:57:56.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.055+0000 7f9dd3f7d640 1 -- start start 2024-09-13T17:57:56.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.056+0000 7f9dd3f7d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9dcc071d30 0x7f9dcc070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.056+0000 7f9dd3f7d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dcc072110 0x7f9dcc0b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.056+0000 7f9dd3f7d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9dcc0c8920 0x7f9dcc0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.056+0000 7f9dd3f7d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dcc0c9310 con 0x7f9dcc0c8920 2024-09-13T17:57:56.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.056+0000 7f9dd3f7d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dcc0abdf0 con 0x7f9dcc071d30 2024-09-13T17:57:56.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.056+0000 7f9dd3f7d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc0ab1e0 con 0x7f9dcc072110 2024-09-13T17:57:56.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.056+0000 7f9dd251a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9dcc0c8920 0x7f9dcc0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.056+0000 7f9dd251a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9dcc0c8920 0x7f9dcc0c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53322/0 (socket says 172.21.15.111:53322) 2024-09-13T17:57:56.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.056+0000 7f9dd251a640 1 -- 172.21.15.111:0/1637000231 learned_addr learned my addr 172.21.15.111:0/1637000231 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:56.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.056+0000 7f9dd1518640 1 --2- 172.21.15.111:0/1637000231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dcc072110 0x7f9dcc0b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.056+0000 7f9dd1d19640 1 --2- 172.21.15.111:0/1637000231 >> v2:172.21.15.188:3300/0 conn(0x7f9dcc071d30 0x7f9dcc070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.056+0000 7f9dd251a640 1 -- 172.21.15.111:0/1637000231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dcc072110 msgr2=0x7f9dcc0b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:56.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.056+0000 7f9dd251a640 1 --2- 172.21.15.111:0/1637000231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dcc072110 0x7f9dcc0b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.056+0000 7f9dd251a640 1 -- 172.21.15.111:0/1637000231 >> v2:172.21.15.188:3300/0 conn(0x7f9dcc071d30 msgr2=0x7f9dcc070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:56.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.056+0000 7f9dd251a640 1 --2- 172.21.15.111:0/1637000231 >> v2:172.21.15.188:3300/0 conn(0x7f9dcc071d30 0x7f9dcc070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.056+0000 7f9dd251a640 1 -- 172.21.15.111:0/1637000231 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dcc0ab4c0 con 0x7f9dcc0c8920 2024-09-13T17:57:56.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.057+0000 7f9dd251a640 1 --2- 172.21.15.111:0/1637000231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dcc0c8920 0x7f9dcc0c8dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009e50 tx=0x7f9dc402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2328cf9f8c0b49f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:56.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.057+0000 7f9dd0d17640 1 -- 172.21.15.111:0/1637000231 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc402aea0 con 0x7f9dcc0c8920 2024-09-13T17:57:56.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.057+0000 7f9dd0d17640 1 -- 172.21.15.111:0/1637000231 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc402d050 con 0x7f9dcc0c8920 2024-09-13T17:57:56.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.057+0000 7f9dd0d17640 1 -- 172.21.15.111:0/1637000231 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4031820 con 0x7f9dcc0c8920 2024-09-13T17:57:56.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.057+0000 7f9dd3f7d640 1 -- 172.21.15.111:0/1637000231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dcc0c8920 msgr2=0x7f9dcc0c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:56.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.057+0000 7f9dd3f7d640 1 --2- 172.21.15.111:0/1637000231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dcc0c8920 0x7f9dcc0c8dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009e50 tx=0x7f9dc402a6c0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.058+0000 7f9dd3f7d640 1 -- 172.21.15.111:0/1637000231 shutdown_connections 2024-09-13T17:57:56.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.058+0000 7f9dd3f7d640 1 --2- 172.21.15.111:0/1637000231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dcc072110 0x7f9dcc0b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.058+0000 7f9dd3f7d640 1 --2- 172.21.15.111:0/1637000231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dcc0c8920 0x7f9dcc0c8dd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.058+0000 7f9dd3f7d640 1 --2- 172.21.15.111:0/1637000231 >> v2:172.21.15.188:3300/0 conn(0x7f9dcc071d30 0x7f9dcc070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.058+0000 7f9dd3f7d640 1 -- 172.21.15.111:0/1637000231 >> 172.21.15.111:0/1637000231 conn(0x7f9dcc069ed0 msgr2=0x7f9dcc06a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:56.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.058+0000 7f9dd3f7d640 1 -- 172.21.15.111:0/1637000231 shutdown_connections 2024-09-13T17:57:56.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.058+0000 7f9dd3f7d640 1 -- 172.21.15.111:0/1637000231 wait complete. 2024-09-13T17:57:56.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.059+0000 7f9dd3f7d640 1 Processor -- start 2024-09-13T17:57:56.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.059+0000 7f9dd3f7d640 1 -- start start 2024-09-13T17:57:56.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd3f7d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dcc071d30 0x7f9dcc152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd3f7d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dcc072110 0x7f9dcc0c2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd3f7d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dcc0c8920 0x7f9dcc0c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd3f7d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc0c34d0 con 0x7f9dcc072110 2024-09-13T17:57:56.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd3f7d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc099220 con 0x7f9dcc0c8920 2024-09-13T17:57:56.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd3f7d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc152a90 con 0x7f9dcc071d30 2024-09-13T17:57:56.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd1518640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dcc072110 0x7f9dcc0c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd1518640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dcc072110 0x7f9dcc0c2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53326/0 (socket says 172.21.15.111:53326) 2024-09-13T17:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd1518640 1 -- 172.21.15.111:0/477801158 learned_addr learned my addr 172.21.15.111:0/477801158 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd251a640 1 --2- 172.21.15.111:0/477801158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dcc0c8920 0x7f9dcc0c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd1d19640 1 --2- 172.21.15.111:0/477801158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dcc071d30 0x7f9dcc152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd1518640 1 -- 172.21.15.111:0/477801158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dcc071d30 msgr2=0x7f9dcc152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd1518640 1 --2- 172.21.15.111:0/477801158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dcc071d30 0x7f9dcc152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd1518640 1 -- 172.21.15.111:0/477801158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dcc0c8920 msgr2=0x7f9dcc0c40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd1518640 1 --2- 172.21.15.111:0/477801158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dcc0c8920 0x7f9dcc0c40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd1518640 1 -- 172.21.15.111:0/477801158 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc4009b30 con 0x7f9dcc072110 2024-09-13T17:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd1518640 1 --2- 172.21.15.111:0/477801158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dcc072110 0x7f9dcc0c2490 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00e8b0 tx=0x7f9dbc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dc27fc640 1 -- 172.21.15.111:0/477801158 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc00c940 con 0x7f9dcc072110 2024-09-13T17:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dc27fc640 1 -- 172.21.15.111:0/477801158 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc00caa0 con 0x7f9dcc072110 2024-09-13T17:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd3f7d640 1 -- 172.21.15.111:0/477801158 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dcc0c2f10 con 0x7f9dcc072110 2024-09-13T17:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dc27fc640 1 -- 172.21.15.111:0/477801158 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc00ce30 con 0x7f9dcc072110 2024-09-13T17:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.060+0000 7f9dd3f7d640 1 -- 172.21.15.111:0/477801158 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dcc0c6520 con 0x7f9dcc072110 2024-09-13T17:57:56.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.062+0000 7f9dd3f7d640 1 -- 172.21.15.111:0/477801158 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dcc0b2f00 con 0x7f9dcc072110 2024-09-13T17:57:56.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.062+0000 7f9dc27fc640 1 -- 172.21.15.111:0/477801158 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9dbc013470 con 0x7f9dcc072110 2024-09-13T17:57:56.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.063+0000 7f9dc27fc640 1 --2- 172.21.15.111:0/477801158 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9db006ac30 0x7f9db006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.063+0000 7f9dc27fc640 1 -- 172.21.15.111:0/477801158 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9dbc089a70 con 0x7f9dcc072110 2024-09-13T17:57:56.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.063+0000 7f9dd1d19640 1 --2- 172.21.15.111:0/477801158 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9db006ac30 0x7f9db006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.063+0000 7f9dd1d19640 1 --2- 172.21.15.111:0/477801158 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9db006ac30 0x7f9db006d0f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009ac0 tx=0x7f9dc8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:56.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.067+0000 7f9dc27fc640 1 -- 172.21.15.111:0/477801158 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f9dbc054aa0 con 0x7f9dcc072110 2024-09-13T17:57:56.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.262+0000 7f9dd3f7d640 1 -- 172.21.15.111:0/477801158 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dcc15cb20 con 0x7f9dcc072110 2024-09-13T17:57:56.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.263+0000 7f9dc27fc640 1 -- 172.21.15.111:0/477801158 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4742 (secure 0 0 0) 0x7f9dbc017090 con 0x7f9dcc072110 2024-09-13T17:57:56.266 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:57:56.266 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4001406804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":4001406804},{"type":"v1","addr":"172.21.15.188:6835","nonce":4001406804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2565466103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2565466103},{"type":"v1","addr":"172.21.15.111:6837","nonce":2565466103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/1252690799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1252690799},{"type":"v1","addr":"172.21.15.111:6836","nonce":1252690799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:57:25.702997+0000","modified":"2024-09-13T17:57:46.351493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/1266310843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1266310843},{"type":"v1","addr":"172.21.15.188:6834","nonce":1266310843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:57:56.266 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-09-13T17:57:56.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.265+0000 7f9dd3f7d640 1 -- 172.21.15.111:0/477801158 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9db006ac30 msgr2=0x7f9db006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:56.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.265+0000 7f9dd3f7d640 1 --2- 172.21.15.111:0/477801158 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9db006ac30 0x7f9db006d0f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009ac0 tx=0x7f9dc8009190 comp rx=0 tx=0).stop 2024-09-13T17:57:56.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.265+0000 7f9dd3f7d640 1 -- 172.21.15.111:0/477801158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dcc072110 msgr2=0x7f9dcc0c2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:56.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.265+0000 7f9dd3f7d640 1 --2- 172.21.15.111:0/477801158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dcc072110 0x7f9dcc0c2490 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00e8b0 tx=0x7f9dbc00ed70 comp rx=0 tx=0).stop 2024-09-13T17:57:56.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.265+0000 7f9dd3f7d640 1 -- 172.21.15.111:0/477801158 shutdown_connections 2024-09-13T17:57:56.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.265+0000 7f9dd3f7d640 1 --2- 172.21.15.111:0/477801158 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9db006ac30 0x7f9db006d0f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.265+0000 7f9dd3f7d640 1 --2- 172.21.15.111:0/477801158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dcc0c8920 0x7f9dcc0c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.265+0000 7f9dd3f7d640 1 --2- 172.21.15.111:0/477801158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dcc072110 0x7f9dcc0c2490 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.265+0000 7f9dd3f7d640 1 --2- 172.21.15.111:0/477801158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dcc071d30 0x7f9dcc152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.265+0000 7f9dd3f7d640 1 -- 172.21.15.111:0/477801158 >> 172.21.15.111:0/477801158 conn(0x7f9dcc069ed0 msgr2=0x7f9dcc0b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:56.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.265+0000 7f9dd3f7d640 1 -- 172.21.15.111:0/477801158 shutdown_connections 2024-09-13T17:57:56.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.265+0000 7f9dd3f7d640 1 -- 172.21.15.111:0/477801158 wait complete. 2024-09-13T17:57:56.391 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:57:56.393 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:57:56.681 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:57:25.702997+0000', 'modified': '2024-09-13T17:57:46.351493+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4291': {'gid': 4291, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6834/1266310843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 1266310843}, {'type': 'v1', 'addr': '172.21.15.188:6834', 'nonce': 1266310843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T17:57:56.681 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T17:57:56.682 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-13T17:57:56.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.779+0000 7fe9326f7640 1 Processor -- start 2024-09-13T17:57:56.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.779+0000 7fe9326f7640 1 -- start start 2024-09-13T17:57:56.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.779+0000 7fe9326f7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe92c0d07b0 0x7fe92c06ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.779+0000 7fe9326f7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe92c0b4660 0x7fe92c06b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.779+0000 7fe9326f7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92c0d1840 con 0x7fe92c15c6e0 2024-09-13T17:57:56.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.779+0000 7fe9326f7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92c0d0c30 con 0x7fe92c0b4660 2024-09-13T17:57:56.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.779+0000 7fe9326f7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0d0f10 con 0x7fe92c0d07b0 2024-09-13T17:57:56.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.779+0000 7fe923fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe92c0d07b0 0x7fe92c06ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.779+0000 7fe930c94640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe92c0b4660 0x7fe92c06b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.779+0000 7fe92bfff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fe92c15c6e0 0x7fe92c0b3f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:33350/0 (socket says 172.21.15.111:33350) 2024-09-13T17:57:56.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.779+0000 7fe92bfff640 1 -- 172.21.15.111:0/643910071 learned_addr learned my addr 172.21.15.111:0/643910071 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:57:56.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.779+0000 7fe923fff640 1 --2- 172.21.15.111:0/643910071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe92c0d07b0 0x7fe92c06ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:57992/0 (socket says 172.21.15.111:57992) 2024-09-13T17:57:56.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.779+0000 7fe930c94640 1 -- 172.21.15.111:0/643910071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe92c0d07b0 msgr2=0x7fe92c06ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:56.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.779+0000 7fe930c94640 1 --2- 172.21.15.111:0/643910071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe92c0d07b0 0x7fe92c06ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.779+0000 7fe930c94640 1 -- 172.21.15.111:0/643910071 >> v1:172.21.15.188:6789/0 conn(0x7fe92c15c6e0 legacy=0x7fe92c0b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:56.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.779+0000 7fe930c94640 1 -- 172.21.15.111:0/643910071 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c0d11f0 con 0x7fe92c0b4660 2024-09-13T17:57:56.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.779+0000 7fe923fff640 1 --2- 172.21.15.111:0/643910071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe92c0d07b0 0x7fe92c06ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:56.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.780+0000 7fe930c94640 1 --2- 172.21.15.111:0/643910071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe92c0b4660 0x7fe92c06b250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe924009d90 tx=0x7fe92402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e2fd07b71833fe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:56.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.780+0000 7fe92b7fe640 1 -- 172.21.15.111:0/643910071 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924031600 con 0x7fe92c0b4660 2024-09-13T17:57:56.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.780+0000 7fe92b7fe640 1 -- 172.21.15.111:0/643910071 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92402d050 con 0x7fe92c0b4660 2024-09-13T17:57:56.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.780+0000 7fe92b7fe640 1 -- 172.21.15.111:0/643910071 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924031980 con 0x7fe92c0b4660 2024-09-13T17:57:56.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.781+0000 7fe9326f7640 1 -- 172.21.15.111:0/643910071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe92c0b4660 msgr2=0x7fe92c06b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:56.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.781+0000 7fe9326f7640 1 --2- 172.21.15.111:0/643910071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe92c0b4660 0x7fe92c06b250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe924009d90 tx=0x7fe92402a5d0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.781+0000 7fe9326f7640 1 -- 172.21.15.111:0/643910071 shutdown_connections 2024-09-13T17:57:56.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.781+0000 7fe9326f7640 1 --2- 172.21.15.111:0/643910071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe92c0b4660 0x7fe92c06b250 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.781+0000 7fe9326f7640 1 --2- 172.21.15.111:0/643910071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe92c0d07b0 0x7fe92c06ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.781+0000 7fe9326f7640 1 -- 172.21.15.111:0/643910071 >> 172.21.15.111:0/643910071 conn(0x7fe92c089f60 msgr2=0x7fe92c0d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:56.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.781+0000 7fe9326f7640 1 -- 172.21.15.111:0/643910071 shutdown_connections 2024-09-13T17:57:56.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.781+0000 7fe9326f7640 1 -- 172.21.15.111:0/643910071 wait complete. 2024-09-13T17:57:56.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.782+0000 7fe9326f7640 1 Processor -- start 2024-09-13T17:57:56.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.782+0000 7fe9326f7640 1 -- start start 2024-09-13T17:57:56.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe9326f7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe92c0b4660 0x7fe92c15cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe9326f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c0d07b0 0x7fe92c15d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe9326f7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe92c15c6e0 0x7fe92c15ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe9326f7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0b07d0 con 0x7fe92c0d07b0 2024-09-13T17:57:56.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe9326f7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0b0ab0 con 0x7fe92c0b4660 2024-09-13T17:57:56.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe9326f7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0b0d90 con 0x7fe92c15c6e0 2024-09-13T17:57:56.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe923fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c0d07b0 0x7fe92c15d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe923fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c0d07b0 0x7fe92c15d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53332/0 (socket says 172.21.15.111:53332) 2024-09-13T17:57:56.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe923fff640 1 -- 172.21.15.111:0/2852936095 learned_addr learned my addr 172.21.15.111:0/2852936095 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:56.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe92bfff640 1 --2- 172.21.15.111:0/2852936095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe92c0b4660 0x7fe92c15cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe930c94640 1 --2- 172.21.15.111:0/2852936095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe92c15c6e0 0x7fe92c15ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe923fff640 1 -- 172.21.15.111:0/2852936095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe92c15c6e0 msgr2=0x7fe92c15ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:56.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe923fff640 1 --2- 172.21.15.111:0/2852936095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe92c15c6e0 0x7fe92c15ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe923fff640 1 -- 172.21.15.111:0/2852936095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe92c0b4660 msgr2=0x7fe92c15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:56.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe923fff640 1 --2- 172.21.15.111:0/2852936095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe92c0b4660 0x7fe92c15cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe923fff640 1 -- 172.21.15.111:0/2852936095 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe914009400 con 0x7fe92c0d07b0 2024-09-13T17:57:56.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe930c94640 1 --2- 172.21.15.111:0/2852936095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe92c15c6e0 0x7fe92c15ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:56.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe923fff640 1 --2- 172.21.15.111:0/2852936095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c0d07b0 0x7fe92c15d000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe91400e940 tx=0x7fe91400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:56.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe9297fa640 1 -- 172.21.15.111:0/2852936095 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91400ca00 con 0x7fe92c0d07b0 2024-09-13T17:57:56.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe9297fa640 1 -- 172.21.15.111:0/2852936095 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91400cb60 con 0x7fe92c0d07b0 2024-09-13T17:57:56.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe9326f7640 1 -- 172.21.15.111:0/2852936095 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe924009a70 con 0x7fe92c0d07b0 2024-09-13T17:57:56.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe9297fa640 1 -- 172.21.15.111:0/2852936095 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe914013470 con 0x7fe92c0d07b0 2024-09-13T17:57:56.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.783+0000 7fe9326f7640 1 -- 172.21.15.111:0/2852936095 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92c0b61f0 con 0x7fe92c0d07b0 2024-09-13T17:57:56.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.785+0000 7fe9297fa640 1 -- 172.21.15.111:0/2852936095 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe9140135d0 con 0x7fe92c0d07b0 2024-09-13T17:57:56.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.785+0000 7fe9297fa640 1 --2- 172.21.15.111:0/2852936095 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe90c06ac00 0x7fe90c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.785+0000 7fe9297fa640 1 -- 172.21.15.111:0/2852936095 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe914089800 con 0x7fe92c0d07b0 2024-09-13T17:57:56.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.785+0000 7fe92bfff640 1 --2- 172.21.15.111:0/2852936095 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe90c06ac00 0x7fe90c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.786+0000 7fe92bfff640 1 --2- 172.21.15.111:0/2852936095 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe90c06ac00 0x7fe90c06d0c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe91c005880 tx=0x7fe91c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:56.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.786+0000 7fe93541c740 1 Processor -- start 2024-09-13T17:57:56.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.786+0000 7fe93541c740 1 -- start start 2024-09-13T17:57:56.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.787+0000 7fe93541c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888ef90a0 0x557888ef9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.787+0000 7fe93541c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888ef6860 0x557888f46570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.787+0000 7fe93541c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888f20550 0x557888f46ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.787+0000 7fe93541c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557888e537c0 con 0x557888ef6860 2024-09-13T17:57:56.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.787+0000 7fe93541c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x557888e62b70 con 0x557888f20550 2024-09-13T17:57:56.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.787+0000 7fe93541c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x557888e60300 con 0x557888ef90a0 2024-09-13T17:57:56.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.787+0000 7fe923fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888ef6860 0x557888f46570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.787+0000 7fe923fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888ef6860 0x557888f46570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53342/0 (socket says 172.21.15.111:53342) 2024-09-13T17:57:56.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.787+0000 7fe923fff640 1 -- 172.21.15.111:0/4261212567 learned_addr learned my addr 172.21.15.111:0/4261212567 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:56.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.787+0000 7fe923fff640 1 -- 172.21.15.111:0/4261212567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888ef90a0 msgr2=0x557888ef9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:56.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.787+0000 7fe923fff640 1 --2- 172.21.15.111:0/4261212567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888ef90a0 0x557888ef9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.787+0000 7fe923fff640 1 -- 172.21.15.111:0/4261212567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888f20550 msgr2=0x557888f46ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:56.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.787+0000 7fe923fff640 1 --2- 172.21.15.111:0/4261212567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888f20550 0x557888f46ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.787+0000 7fe923fff640 1 -- 172.21.15.111:0/4261212567 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557888de84b0 con 0x557888ef6860 2024-09-13T17:57:56.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.787+0000 7fe923fff640 1 --2- 172.21.15.111:0/4261212567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888ef6860 0x557888f46570 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe9140549f0 tx=0x7fe914058130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157c88dbf0087b49 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:56.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.787+0000 7fe921ffb640 1 -- 172.21.15.111:0/4261212567 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe914060830 con 0x557888ef6860 2024-09-13T17:57:56.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.788+0000 7fe921ffb640 1 -- 172.21.15.111:0/4261212567 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91405c020 con 0x557888ef6860 2024-09-13T17:57:56.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.788+0000 7fe921ffb640 1 -- 172.21.15.111:0/4261212567 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe914054c00 con 0x557888ef6860 2024-09-13T17:57:56.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.788+0000 7fe93541c740 1 -- 172.21.15.111:0/4261212567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888ef6860 msgr2=0x557888f46570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:56.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.788+0000 7fe93541c740 1 --2- 172.21.15.111:0/4261212567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888ef6860 0x557888f46570 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe9140549f0 tx=0x7fe914058130 comp rx=0 tx=0).stop 2024-09-13T17:57:56.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.788+0000 7fe93541c740 1 -- 172.21.15.111:0/4261212567 shutdown_connections 2024-09-13T17:57:56.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.788+0000 7fe93541c740 1 --2- 172.21.15.111:0/4261212567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888f20550 0x557888f46ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.788+0000 7fe93541c740 1 --2- 172.21.15.111:0/4261212567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888ef6860 0x557888f46570 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.788+0000 7fe93541c740 1 --2- 172.21.15.111:0/4261212567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888ef90a0 0x557888ef9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.788+0000 7fe93541c740 1 -- 172.21.15.111:0/4261212567 >> 172.21.15.111:0/4261212567 conn(0x557888ef57d0 msgr2=0x557888f3dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:56.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.788+0000 7fe93541c740 1 -- 172.21.15.111:0/4261212567 shutdown_connections 2024-09-13T17:57:56.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.788+0000 7fe93541c740 1 -- 172.21.15.111:0/4261212567 wait complete. 2024-09-13T17:57:56.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.789+0000 7fe93541c740 1 -- start start 2024-09-13T17:57:56.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.789+0000 7fe93541c740 1 Processor -- start 2024-09-13T17:57:56.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe93541c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888ef90a0 0x557888f24030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe93541c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f10720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe93541c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888f45230 0x557888ef3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe93541c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557888e5bbf0 con 0x557888f20550 2024-09-13T17:57:56.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe93541c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x557888e1a250 con 0x557888f45230 2024-09-13T17:57:56.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe93541c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x557888e19ef0 con 0x557888ef90a0 2024-09-13T17:57:56.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe93541c740 10 client.0 _ll_register_callbacks cb 0x557888e4c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:57:56.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe923fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f10720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe923fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f10720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53354/0 (socket says 172.21.15.111:53354) 2024-09-13T17:57:56.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe923fff640 1 -- 172.21.15.111:0/1255670277 learned_addr learned my addr 172.21.15.111:0/1255670277 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:56.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe92bfff640 1 --2- 172.21.15.111:0/1255670277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888ef90a0 0x557888f24030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe930c94640 1 --2- 172.21.15.111:0/1255670277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888f45230 0x557888ef3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe923fff640 1 -- 172.21.15.111:0/1255670277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888ef90a0 msgr2=0x557888f24030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:56.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe923fff640 1 --2- 172.21.15.111:0/1255670277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888ef90a0 0x557888f24030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe923fff640 1 -- 172.21.15.111:0/1255670277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888f45230 msgr2=0x557888ef3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:56.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe923fff640 1 --2- 172.21.15.111:0/1255670277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888f45230 0x557888ef3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe923fff640 1 -- 172.21.15.111:0/1255670277 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92c0b61f0 con 0x557888f20550 2024-09-13T17:57:56.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe930c94640 1 --2- 172.21.15.111:0/1255670277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888f45230 0x557888ef3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:56.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe923fff640 1 --2- 172.21.15.111:0/1255670277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f10720 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe9140581f0 tx=0x7fe914000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413429b30a6bf039 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:56.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe90affd640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-13T17:57:56.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe90affd640 1 -- 172.21.15.111:0/1255670277 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9140605a0 con 0x557888f20550 2024-09-13T17:57:56.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe90affd640 1 -- 172.21.15.111:0/1255670277 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe91405e050 con 0x557888f20550 2024-09-13T17:57:56.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe93541c740 1 -- 172.21.15.111:0/1255670277 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557888b4f3c0 con 0x557888f20550 2024-09-13T17:57:56.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe90affd640 1 -- 172.21.15.111:0/1255670277 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe91405e210 con 0x557888f20550 2024-09-13T17:57:56.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe93541c740 10 client.4827 fetch_fsmap learned FSMap version 9 2024-09-13T17:57:56.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.790+0000 7fe93541c740 1 -- 172.21.15.111:0/1255670277 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x557888e5a270 con 0x557888f20550 2024-09-13T17:57:56.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.791+0000 7fe90affd640 1 -- 172.21.15.111:0/1255670277 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557888b4f3c0 con 0x557888f20550 2024-09-13T17:57:56.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.791+0000 7fe90affd640 1 -- 172.21.15.111:0/1255670277 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fe914011070 con 0x557888f20550 2024-09-13T17:57:56.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.791+0000 7fe93541c740 10 client.4827 fetch_fsmap finished waiting for FSMap version 9 2024-09-13T17:57:56.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.791+0000 7fe93541c740 10 client.4827 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T17:57:56.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.791+0000 7fe93541c740 20 client.4827 populate_metadata read hostname 'smithi111' 2024-09-13T17:57:56.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.791+0000 7fe93541c740 1 --2- 172.21.15.111:0/1255670277 >> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] conn(0x557888f4df90 0x557888f42330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.791+0000 7fe93541c740 4 client.4827 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:57:56.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.791+0000 7fe93541c740 1 -- 172.21.15.111:0/1255670277 --> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557888b606c0 con 0x557888f4df90 2024-09-13T17:57:56.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.791+0000 7fe92bfff640 1 --2- 172.21.15.111:0/1255670277 >> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] conn(0x557888f4df90 0x557888f42330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.791+0000 7fe90affd640 10 client.4827 ms_handle_connect on v2:172.21.15.188:6832/1266310843 2024-09-13T17:57:56.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.791+0000 7fe92bfff640 1 --2- 172.21.15.111:0/1255670277 >> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] conn(0x557888f4df90 0x557888f42330 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5475914a13563a9f server_cookie=4c807dbcdd7d64b4 in_seq=0 out_seq=0 2024-09-13T17:57:56.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.792+0000 7fe90affd640 1 -- 172.21.15.111:0/1255670277 <== mds.0 v2:172.21.15.188:6832/1266310843 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x557888b606c0 con 0x557888f4df90 2024-09-13T17:57:56.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.792+0000 7fe90affd640 10 client.4827 handle_command_reply: tid=0 2024-09-13T17:57:56.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.792+0000 7fe93541c740 1 client.4827 shutdown 2024-09-13T17:57:56.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.792+0000 7fe93541c740 1 -- 172.21.15.111:0/1255670277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 msgr2=0x557888f10720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:56.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.792+0000 7fe93541c740 1 --2- 172.21.15.111:0/1255670277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f10720 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe9140581f0 tx=0x7fe914000bc0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.792+0000 7fe93541c740 1 -- 172.21.15.111:0/1255670277 shutdown_connections 2024-09-13T17:57:56.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.792+0000 7fe93541c740 1 --2- 172.21.15.111:0/1255670277 >> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] conn(0x557888f4df90 0x557888f42330 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.792+0000 7fe93541c740 1 --2- 172.21.15.111:0/1255670277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888f45230 0x557888ef3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.792+0000 7fe90affd640 0 client.4827 ms_handle_reset on v2:172.21.15.188:6832/1266310843 2024-09-13T17:57:56.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.792+0000 7fe93541c740 1 --2- 172.21.15.111:0/1255670277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f10720 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.792+0000 7fe93541c740 1 --2- 172.21.15.111:0/1255670277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888ef90a0 0x557888f24030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.792+0000 7fe93541c740 1 -- 172.21.15.111:0/1255670277 >> 172.21.15.111:0/1255670277 conn(0x557888ef6860 msgr2=0x557888f41960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:56.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.792+0000 7fe93541c740 1 -- 172.21.15.111:0/1255670277 shutdown_connections 2024-09-13T17:57:56.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.792+0000 7fe93541c740 1 -- 172.21.15.111:0/1255670277 wait complete. 2024-09-13T17:57:56.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.792+0000 7fe93541c740 20 client.4827 trim_cache size 0 max 16384 2024-09-13T17:57:56.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.808+0000 7fe93541c740 1 Processor -- start 2024-09-13T17:57:56.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.808+0000 7fe93541c740 1 -- start start 2024-09-13T17:57:56.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.808+0000 7fe93541c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888ef90a0 0x557888fa1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.808+0000 7fe93541c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f4ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.808+0000 7fe93541c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888f45230 0x557888eeea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.808+0000 7fe93541c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557888ddf460 con 0x557888f20550 2024-09-13T17:57:56.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.808+0000 7fe93541c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x557888f60790 con 0x557888ef90a0 2024-09-13T17:57:56.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.808+0000 7fe93541c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x557888f60a30 con 0x557888f45230 2024-09-13T17:57:56.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.808+0000 7fe923fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f4ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.808+0000 7fe923fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f4ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53364/0 (socket says 172.21.15.111:53364) 2024-09-13T17:57:56.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.808+0000 7fe923fff640 1 -- 172.21.15.111:0/3628559500 learned_addr learned my addr 172.21.15.111:0/3628559500 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:56.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe923fff640 1 -- 172.21.15.111:0/3628559500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888f45230 msgr2=0x557888eeea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:56.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe923fff640 1 --2- 172.21.15.111:0/3628559500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888f45230 0x557888eeea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe923fff640 1 -- 172.21.15.111:0/3628559500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888ef90a0 msgr2=0x557888fa1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:56.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe923fff640 1 --2- 172.21.15.111:0/3628559500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888ef90a0 0x557888fa1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe923fff640 1 -- 172.21.15.111:0/3628559500 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557888dfc5f0 con 0x557888f20550 2024-09-13T17:57:56.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe923fff640 1 --2- 172.21.15.111:0/3628559500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f4ceb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe91400cfd0 tx=0x7fe91405b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c99e68c659414a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:56.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe9227fc640 1 -- 172.21.15.111:0/3628559500 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9140582c0 con 0x557888f20550 2024-09-13T17:57:56.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe9227fc640 1 -- 172.21.15.111:0/3628559500 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9140584f0 con 0x557888f20550 2024-09-13T17:57:56.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe9227fc640 1 -- 172.21.15.111:0/3628559500 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9140605c0 con 0x557888f20550 2024-09-13T17:57:56.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe93541c740 1 -- 172.21.15.111:0/3628559500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 msgr2=0x557888f4ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:56.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe93541c740 1 --2- 172.21.15.111:0/3628559500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f4ceb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe91400cfd0 tx=0x7fe91405b070 comp rx=0 tx=0).stop 2024-09-13T17:57:56.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe93541c740 1 -- 172.21.15.111:0/3628559500 shutdown_connections 2024-09-13T17:57:56.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe93541c740 1 --2- 172.21.15.111:0/3628559500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888f45230 0x557888eeea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe93541c740 1 --2- 172.21.15.111:0/3628559500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f4ceb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe93541c740 1 --2- 172.21.15.111:0/3628559500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888ef90a0 0x557888fa1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe93541c740 1 -- 172.21.15.111:0/3628559500 >> 172.21.15.111:0/3628559500 conn(0x557888ef6860 msgr2=0x557888f23e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:56.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe93541c740 1 -- 172.21.15.111:0/3628559500 shutdown_connections 2024-09-13T17:57:56.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.809+0000 7fe93541c740 1 -- 172.21.15.111:0/3628559500 wait complete. 2024-09-13T17:57:56.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.810+0000 7fe93541c740 1 -- start start 2024-09-13T17:57:56.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.810+0000 7fe93541c740 1 Processor -- start 2024-09-13T17:57:56.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe93541c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888ef90a0 0x557888ef2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe93541c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f4d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe93541c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888f45230 0x557888f9dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe93541c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557888f4d580 con 0x557888f20550 2024-09-13T17:57:56.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe93541c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x557888f43be0 con 0x557888ef90a0 2024-09-13T17:57:56.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe93541c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x557888f9e760 con 0x557888f45230 2024-09-13T17:57:56.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe93541c740 10 client.0 _ll_register_callbacks cb 0x7fe9140096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:57:56.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe923fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f4d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe923fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f4d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53374/0 (socket says 172.21.15.111:53374) 2024-09-13T17:57:56.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe923fff640 1 -- 172.21.15.111:0/1792145339 learned_addr learned my addr 172.21.15.111:0/1792145339 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:56.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe92bfff640 1 --2- 172.21.15.111:0/1792145339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888ef90a0 0x557888ef2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe930c94640 1 --2- 172.21.15.111:0/1792145339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888f45230 0x557888f9dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe923fff640 1 -- 172.21.15.111:0/1792145339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888f45230 msgr2=0x557888f9dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:56.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe923fff640 1 --2- 172.21.15.111:0/1792145339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888f45230 0x557888f9dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe923fff640 1 -- 172.21.15.111:0/1792145339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888ef90a0 msgr2=0x557888ef2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:56.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe923fff640 1 --2- 172.21.15.111:0/1792145339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888ef90a0 0x557888ef2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe923fff640 1 -- 172.21.15.111:0/1792145339 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557888e5a270 con 0x557888f20550 2024-09-13T17:57:56.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe923fff640 1 --2- 172.21.15.111:0/1792145339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f4d040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe91400c520 tx=0x7fe914009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa273ddc6407a46 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:56.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe90affd640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-13T17:57:56.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe90affd640 1 -- 172.21.15.111:0/1792145339 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9140605a0 con 0x557888f20550 2024-09-13T17:57:56.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe90affd640 1 -- 172.21.15.111:0/1792145339 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe91405e040 con 0x557888f20550 2024-09-13T17:57:56.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.811+0000 7fe93541c740 1 -- 172.21.15.111:0/1792145339 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe91405c020 con 0x557888f20550 2024-09-13T17:57:56.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.812+0000 7fe90affd640 1 -- 172.21.15.111:0/1792145339 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9140582c0 con 0x557888f20550 2024-09-13T17:57:56.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.812+0000 7fe93541c740 10 client.4833 fetch_fsmap learned FSMap version 9 2024-09-13T17:57:56.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.812+0000 7fe93541c740 1 -- 172.21.15.111:0/1792145339 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x557888e21d00 con 0x557888f20550 2024-09-13T17:57:56.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.812+0000 7fe90affd640 1 -- 172.21.15.111:0/1792145339 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91405c020 con 0x557888f20550 2024-09-13T17:57:56.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.812+0000 7fe90affd640 1 -- 172.21.15.111:0/1792145339 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fe914011070 con 0x557888f20550 2024-09-13T17:57:56.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.812+0000 7fe93541c740 10 client.4833 fetch_fsmap finished waiting for FSMap version 9 2024-09-13T17:57:56.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.812+0000 7fe93541c740 10 client.4833 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T17:57:56.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.812+0000 7fe93541c740 20 client.4833 populate_metadata read hostname 'smithi111' 2024-09-13T17:57:56.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.812+0000 7fe93541c740 1 --2- 172.21.15.111:0/1792145339 >> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] conn(0x557888f4df90 0x557888f43120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:56.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.812+0000 7fe93541c740 4 client.4833 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:57:56.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.812+0000 7fe93541c740 1 -- 172.21.15.111:0/1792145339 --> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557888bb3d10 con 0x557888f4df90 2024-09-13T17:57:56.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.812+0000 7fe92bfff640 1 --2- 172.21.15.111:0/1792145339 >> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] conn(0x557888f4df90 0x557888f43120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:56.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.812+0000 7fe90affd640 10 client.4833 ms_handle_connect on v2:172.21.15.188:6832/1266310843 2024-09-13T17:57:56.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.812+0000 7fe92bfff640 1 --2- 172.21.15.111:0/1792145339 >> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] conn(0x557888f4df90 0x557888f43120 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b26c98832571e313 server_cookie=a9745da40790b86a in_seq=0 out_seq=0 2024-09-13T17:57:56.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe90affd640 1 -- 172.21.15.111:0/1792145339 <== mds.0 v2:172.21.15.188:6832/1266310843 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x557888bb3d10 con 0x557888f4df90 2024-09-13T17:57:56.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe90affd640 10 client.4833 handle_command_reply: tid=0 2024-09-13T17:57:56.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe93541c740 1 client.4833 shutdown 2024-09-13T17:57:56.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe93541c740 1 -- 172.21.15.111:0/1792145339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 msgr2=0x557888f4d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:56.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe93541c740 1 --2- 172.21.15.111:0/1792145339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f4d040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe91400c520 tx=0x7fe914009790 comp rx=0 tx=0).stop 2024-09-13T17:57:56.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe93541c740 1 -- 172.21.15.111:0/1792145339 shutdown_connections 2024-09-13T17:57:56.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe93541c740 1 --2- 172.21.15.111:0/1792145339 >> [v2:172.21.15.188:6832/1266310843,v1:172.21.15.188:6834/1266310843] conn(0x557888f4df90 0x557888f43120 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe93541c740 1 --2- 172.21.15.111:0/1792145339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557888f45230 0x557888f9dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe93541c740 1 --2- 172.21.15.111:0/1792145339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557888f20550 0x557888f4d040 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe93541c740 1 --2- 172.21.15.111:0/1792145339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557888ef90a0 0x557888ef2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe90affd640 0 client.4833 ms_handle_reset on v2:172.21.15.188:6832/1266310843 2024-09-13T17:57:56.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe93541c740 1 -- 172.21.15.111:0/1792145339 >> 172.21.15.111:0/1792145339 conn(0x557888ef6860 msgr2=0x557888ef4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:56.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe93541c740 1 -- 172.21.15.111:0/1792145339 shutdown_connections 2024-09-13T17:57:56.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe93541c740 1 -- 172.21.15.111:0/1792145339 wait complete. 2024-09-13T17:57:56.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe93541c740 20 client.4833 trim_cache size 0 max 16384 2024-09-13T17:57:56.823 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T17:57:56.823 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "2a8c13ef-d427-4b48-90b3-65459ec79a47", 2024-09-13T17:57:56.823 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2024-09-13T17:57:56.823 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4291, 2024-09-13T17:57:56.823 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-09-13T17:57:56.823 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-09-13T17:57:56.823 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-09-13T17:57:56.823 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 31.109495566, 2024-09-13T17:57:56.823 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 9, 2024-09-13T17:57:56.823 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 24, 2024-09-13T17:57:56.823 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 24, 2024-09-13T17:57:56.823 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 37.004180552000001 2024-09-13T17:57:56.823 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T17:57:56.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe9326f7640 1 -- 172.21.15.111:0/2852936095 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe90c06ac00 msgr2=0x7fe90c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:56.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe9326f7640 1 --2- 172.21.15.111:0/2852936095 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe90c06ac00 0x7fe90c06d0c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe91c005880 tx=0x7fe91c009040 comp rx=0 tx=0).stop 2024-09-13T17:57:56.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe9326f7640 1 -- 172.21.15.111:0/2852936095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c0d07b0 msgr2=0x7fe92c15d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:56.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe9326f7640 1 --2- 172.21.15.111:0/2852936095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c0d07b0 0x7fe92c15d000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe91400e940 tx=0x7fe91400ee00 comp rx=0 tx=0).stop 2024-09-13T17:57:56.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe9326f7640 1 -- 172.21.15.111:0/2852936095 shutdown_connections 2024-09-13T17:57:56.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe9326f7640 1 --2- 172.21.15.111:0/2852936095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe92c15c6e0 0x7fe92c15ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe9326f7640 1 --2- 172.21.15.111:0/2852936095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c0d07b0 0x7fe92c15d000 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe9326f7640 1 --2- 172.21.15.111:0/2852936095 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe90c06ac00 0x7fe90c06d0c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe9326f7640 1 --2- 172.21.15.111:0/2852936095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe92c0b4660 0x7fe92c15cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:56.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.813+0000 7fe9326f7640 1 -- 172.21.15.111:0/2852936095 >> 172.21.15.111:0/2852936095 conn(0x7fe92c089f60 msgr2=0x7fe92c1e5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:56.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.814+0000 7fe9326f7640 1 -- 172.21.15.111:0/2852936095 shutdown_connections 2024-09-13T17:57:56.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:56.814+0000 7fe9326f7640 1 -- 172.21.15.111:0/2852936095 wait complete. 2024-09-13T17:57:56.827 INFO:teuthology.run_tasks:Running task sequential... 2024-09-13T17:57:56.836 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-13T17:57:56.837 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-13T17:57:56.837 INFO:teuthology.task.print:Enabling mgr modules 2024-09-13T17:57:56.837 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-13T17:57:56.861 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-13T17:57:56.861 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-13T17:57:56.861 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-13T17:57:56.862 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T17:57:56.862 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T17:57:56.862 INFO:tasks.ceph_fuse:client.2 config is {} 2024-09-13T17:57:56.862 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T17:57:56.862 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-13T17:57:56.862 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-13T17:57:56.862 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T17:57:56.862 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T17:57:56.862 INFO:tasks.ceph_fuse:client.3 config is {} 2024-09-13T17:57:56.862 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T17:57:56.862 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-13T17:57:56.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:56.863 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-09-13T17:57:56.879 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-13T17:57:56.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:56.880 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-13T17:57:56.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:56.945 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link delete ceph-brx 2024-09-13T17:57:57.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:57.044 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-09-13T17:57:57.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:57.063 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-09-13T17:57:57.136 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2024-09-13T17:57:57.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:57:57.138 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-13T17:57:57.138 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T17:57:57.138 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:57:57.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.235+0000 7fb1e8db5640 1 Processor -- start 2024-09-13T17:57:57.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.236+0000 7fb1e8db5640 1 -- start start 2024-09-13T17:57:57.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.236+0000 7fb1e8db5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb1e406c4c0 0x7fb1e406c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:57.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.236+0000 7fb1e8db5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb1e40cbea0 0x7fb1e40cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:57.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.236+0000 7fb1e8db5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e406dab0 con 0x7fb1e40b2590 2024-09-13T17:57:57.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.236+0000 7fb1e8db5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e406deb0 con 0x7fb1e40cbea0 2024-09-13T17:57:57.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.236+0000 7fb1e8db5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e40b2170 con 0x7fb1e406c4c0 2024-09-13T17:57:57.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.237+0000 7fb1e2d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb1e40cbea0 0x7fb1e40cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:57.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.237+0000 7fb1e2d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb1e40cbea0 0x7fb1e40cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53388/0 (socket says 172.21.15.111:53388) 2024-09-13T17:57:57.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.237+0000 7fb1e2d9d640 1 -- 172.21.15.111:0/3651206138 learned_addr learned my addr 172.21.15.111:0/3651206138 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:57.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.237+0000 7fb1e259c640 1 --2- 172.21.15.111:0/3651206138 >> v2:172.21.15.188:3300/0 conn(0x7fb1e406c4c0 0x7fb1e406c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:57.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.237+0000 7fb1e2d9d640 1 -- 172.21.15.111:0/3651206138 >> v2:172.21.15.188:3300/0 conn(0x7fb1e406c4c0 msgr2=0x7fb1e406c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:57.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.237+0000 7fb1e2d9d640 1 --2- 172.21.15.111:0/3651206138 >> v2:172.21.15.188:3300/0 conn(0x7fb1e406c4c0 0x7fb1e406c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:57.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.237+0000 7fb1e2d9d640 1 -- 172.21.15.111:0/3651206138 >> v1:172.21.15.188:6789/0 conn(0x7fb1e40b2590 legacy=0x7fb1e406d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:57.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.237+0000 7fb1e2d9d640 1 -- 172.21.15.111:0/3651206138 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e40cc870 con 0x7fb1e40cbea0 2024-09-13T17:57:57.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.237+0000 7fb1e2d9d640 1 --2- 172.21.15.111:0/3651206138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1e40cbea0 0x7fb1e40cc330 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40cb910 tx=0x7fb1dc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab446abd846be5a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:57.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.238+0000 7fb1e159a640 1 -- 172.21.15.111:0/3651206138 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc031600 con 0x7fb1e40cbea0 2024-09-13T17:57:57.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.238+0000 7fb1e159a640 1 -- 172.21.15.111:0/3651206138 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1dc02d050 con 0x7fb1e40cbea0 2024-09-13T17:57:57.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.238+0000 7fb1e159a640 1 -- 172.21.15.111:0/3651206138 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc031980 con 0x7fb1e40cbea0 2024-09-13T17:57:57.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.238+0000 7fb1e8db5640 1 -- 172.21.15.111:0/3651206138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1e40cbea0 msgr2=0x7fb1e40cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:57.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.238+0000 7fb1e8db5640 1 --2- 172.21.15.111:0/3651206138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1e40cbea0 0x7fb1e40cc330 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40cb910 tx=0x7fb1dc02a730 comp rx=0 tx=0).stop 2024-09-13T17:57:57.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.238+0000 7fb1e8db5640 1 -- 172.21.15.111:0/3651206138 shutdown_connections 2024-09-13T17:57:57.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.238+0000 7fb1e8db5640 1 --2- 172.21.15.111:0/3651206138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1e40cbea0 0x7fb1e40cc330 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:57.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.238+0000 7fb1e8db5640 1 --2- 172.21.15.111:0/3651206138 >> v2:172.21.15.188:3300/0 conn(0x7fb1e406c4c0 0x7fb1e406c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:57.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.238+0000 7fb1e8db5640 1 -- 172.21.15.111:0/3651206138 >> 172.21.15.111:0/3651206138 conn(0x7fb1e4069ed0 msgr2=0x7fb1e406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:57.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.238+0000 7fb1e8db5640 1 -- 172.21.15.111:0/3651206138 shutdown_connections 2024-09-13T17:57:57.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.238+0000 7fb1e8db5640 1 -- 172.21.15.111:0/3651206138 wait complete. 2024-09-13T17:57:57.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.240+0000 7fb1e8db5640 1 Processor -- start 2024-09-13T17:57:57.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.240+0000 7fb1e8db5640 1 -- start start 2024-09-13T17:57:57.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.240+0000 7fb1e8db5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1e406c4c0 0x7fb1e4156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:57.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.240+0000 7fb1e8db5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1e40b2590 0x7fb1e4157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:57.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.240+0000 7fb1e8db5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1e40cbea0 0x7fb1e40ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:57.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.240+0000 7fb1e8db5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e41f1950 con 0x7fb1e406c4c0 2024-09-13T17:57:57.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.240+0000 7fb1e8db5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e4157990 con 0x7fb1e40b2590 2024-09-13T17:57:57.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.240+0000 7fb1e8db5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40ab600 con 0x7fb1e40cbea0 2024-09-13T17:57:57.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.240+0000 7fb1e259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1e406c4c0 0x7fb1e4156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:57.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.240+0000 7fb1e259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1e406c4c0 0x7fb1e4156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53392/0 (socket says 172.21.15.111:53392) 2024-09-13T17:57:57.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.240+0000 7fb1e259c640 1 -- 172.21.15.111:0/3753629045 learned_addr learned my addr 172.21.15.111:0/3753629045 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:57.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.241+0000 7fb1e1d9b640 1 --2- 172.21.15.111:0/3753629045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1e40b2590 0x7fb1e4157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:57.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.241+0000 7fb1e2d9d640 1 --2- 172.21.15.111:0/3753629045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1e40cbea0 0x7fb1e40ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:57.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.241+0000 7fb1e259c640 1 -- 172.21.15.111:0/3753629045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1e40cbea0 msgr2=0x7fb1e40ac680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:57.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.241+0000 7fb1e259c640 1 --2- 172.21.15.111:0/3753629045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1e40cbea0 0x7fb1e40ac680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:57.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.241+0000 7fb1e259c640 1 -- 172.21.15.111:0/3753629045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1e40b2590 msgr2=0x7fb1e4157450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:57.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.241+0000 7fb1e259c640 1 --2- 172.21.15.111:0/3753629045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1e40b2590 0x7fb1e4157450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:57.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.241+0000 7fb1e259c640 1 -- 172.21.15.111:0/3753629045 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc009b30 con 0x7fb1e406c4c0 2024-09-13T17:57:57.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.241+0000 7fb1e1d9b640 1 --2- 172.21.15.111:0/3753629045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1e40b2590 0x7fb1e4157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:57.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.241+0000 7fb1e2d9d640 1 --2- 172.21.15.111:0/3753629045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1e40cbea0 0x7fb1e40ac680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:57.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.241+0000 7fb1e259c640 1 --2- 172.21.15.111:0/3753629045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1e406c4c0 0x7fb1e4156740 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400d7c0 tx=0x7fb1d400dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:57.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.241+0000 7fb1daffd640 1 -- 172.21.15.111:0/3753629045 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400a8c0 con 0x7fb1e406c4c0 2024-09-13T17:57:57.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.241+0000 7fb1daffd640 1 -- 172.21.15.111:0/3753629045 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d400aa20 con 0x7fb1e406c4c0 2024-09-13T17:57:57.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.241+0000 7fb1e8db5640 1 -- 172.21.15.111:0/3753629045 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e40ab8e0 con 0x7fb1e406c4c0 2024-09-13T17:57:57.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.241+0000 7fb1daffd640 1 -- 172.21.15.111:0/3753629045 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400add0 con 0x7fb1e406c4c0 2024-09-13T17:57:57.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.241+0000 7fb1e8db5640 1 -- 172.21.15.111:0/3753629045 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e40abea0 con 0x7fb1e406c4c0 2024-09-13T17:57:57.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.242+0000 7fb1e8db5640 1 -- 172.21.15.111:0/3753629045 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e406c8a0 con 0x7fb1e406c4c0 2024-09-13T17:57:57.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.242+0000 7fb1daffd640 1 -- 172.21.15.111:0/3753629045 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb1d40123f0 con 0x7fb1e406c4c0 2024-09-13T17:57:57.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.243+0000 7fb1daffd640 1 --2- 172.21.15.111:0/3753629045 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb1c406ac00 0x7fb1c406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:57.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.243+0000 7fb1daffd640 1 -- 172.21.15.111:0/3753629045 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb1d4089740 con 0x7fb1e406c4c0 2024-09-13T17:57:57.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.243+0000 7fb1e1d9b640 1 --2- 172.21.15.111:0/3753629045 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb1c406ac00 0x7fb1c406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:57.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.243+0000 7fb1e1d9b640 1 --2- 172.21.15.111:0/3753629045 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb1c406ac00 0x7fb1c406d0c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc007fd0 tx=0x7fb1cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:57.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.247+0000 7fb1daffd640 1 -- 172.21.15.111:0/3753629045 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fb1d4054630 con 0x7fb1e406c4c0 2024-09-13T17:57:57.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.443+0000 7fb1e8db5640 1 -- 172.21.15.111:0/3753629045 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb1e41f1a90 con 0x7fb1e406c4c0 2024-09-13T17:57:57.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.443+0000 7fb1daffd640 1 -- 172.21.15.111:0/3753629045 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb1d4016070 con 0x7fb1e406c4c0 2024-09-13T17:57:57.445 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T17:57:57.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.445+0000 7fb1e8db5640 1 -- 172.21.15.111:0/3753629045 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb1c406ac00 msgr2=0x7fb1c406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:57.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.445+0000 7fb1e8db5640 1 --2- 172.21.15.111:0/3753629045 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb1c406ac00 0x7fb1c406d0c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc007fd0 tx=0x7fb1cc009040 comp rx=0 tx=0).stop 2024-09-13T17:57:57.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.445+0000 7fb1e8db5640 1 -- 172.21.15.111:0/3753629045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1e406c4c0 msgr2=0x7fb1e4156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:57.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.445+0000 7fb1e8db5640 1 --2- 172.21.15.111:0/3753629045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1e406c4c0 0x7fb1e4156740 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400d7c0 tx=0x7fb1d400dc80 comp rx=0 tx=0).stop 2024-09-13T17:57:57.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.445+0000 7fb1e8db5640 1 -- 172.21.15.111:0/3753629045 shutdown_connections 2024-09-13T17:57:57.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.445+0000 7fb1e8db5640 1 --2- 172.21.15.111:0/3753629045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1e40cbea0 0x7fb1e40ac680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:57.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.445+0000 7fb1e8db5640 1 --2- 172.21.15.111:0/3753629045 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb1c406ac00 0x7fb1c406d0c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:57.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.445+0000 7fb1e8db5640 1 --2- 172.21.15.111:0/3753629045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1e40b2590 0x7fb1e4157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:57.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.445+0000 7fb1e8db5640 1 --2- 172.21.15.111:0/3753629045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1e406c4c0 0x7fb1e4156740 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:57.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.445+0000 7fb1e8db5640 1 -- 172.21.15.111:0/3753629045 >> 172.21.15.111:0/3753629045 conn(0x7fb1e4069ed0 msgr2=0x7fb1e406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:57.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.445+0000 7fb1e8db5640 1 -- 172.21.15.111:0/3753629045 shutdown_connections 2024-09-13T17:57:57.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:57.445+0000 7fb1e8db5640 1 -- 172.21.15.111:0/3753629045 wait complete. 2024-09-13T17:57:57.458 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:57:57.458 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:57:57.458 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2024-09-13T17:57:57.458 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T17:57:57.458 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T17:57:57.458 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:57:57.458 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:57:57.459 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:57:57.459 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:57:57.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:57.459 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2024-09-13T17:57:57.475 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:57:57.475 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:57:57.475 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:57:57.475 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:57.475 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-09-13T17:57:57.475 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:57.475 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:57:57.475 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:57:57.476 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2024-09-13T17:57:57.476 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:57:57.476 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:57:57.476 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2024-09-13T17:57:57.476 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:57:57.476 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2024-09-13T17:57:57.476 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:57:57.476 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2024-09-13T17:57:57.476 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-13T17:57:57.476 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42782sec preferred_lft 42782sec 2024-09-13T17:57:57.477 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link noprefixroute 2024-09-13T17:57:57.477 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:57.477 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T17:57:57.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:57.477 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2024-09-13T17:57:57.477 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-13T17:57:57.477 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add name ceph-brx type bridge 2024-09-13T17:57:57.477 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr flush dev ceph-brx 2024-09-13T17:57:57.477 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set ceph-brx up 2024-09-13T17:57:57.477 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T17:57:57.478 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-13T17:57:57.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:57.574 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:57:57.597 INFO:teuthology.orchestra.run.smithi111.stdout:1 2024-09-13T17:57:57.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:57.598 DEBUG:teuthology.orchestra.run.smithi111:> route 2024-09-13T17:57:57.655 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2024-09-13T17:57:57.655 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:57:57.656 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:57:57.656 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:57:57.656 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:57:57.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:57.656 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2024-09-13T17:57:57.656 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-13T17:57:57.656 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:57:57.656 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:57:57.656 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:57:57.656 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-13T17:57:57.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:57.744 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-09-13T17:57:57.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:57.759 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list-id 2024-09-13T17:57:57.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:57.815 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2024-09-13T17:57:57.815 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-13T17:57:57.815 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:57:57.816 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-13T17:57:57.816 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-13T17:57:57.893 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-13T17:57:57.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:57.893 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2024-09-13T17:57:57.893 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-13T17:57:57.893 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-13T17:57:57.893 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T17:57:57.893 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-13T17:57:57.893 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-13T17:57:57.893 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-13T17:57:57.894 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-13T17:57:58.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:58.006 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2024-09-13T17:57:58.006 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-13T17:57:58.006 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2024-09-13T17:57:58.006 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T17:57:58.006 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-13T17:57:58.043 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-13T17:57:58.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:57:58.043 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T17:57:58.099 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:57:58.100 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:57:58.100 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:57:58.155 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2024-09-13T17:57:58.219 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:57:58.220 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:57:58.284 INFO:teuthology.orchestra.run.smithi111.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:57:58.285 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:57:58.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:58.285 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2024-09-13T17:57:58.341 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T17:57:58.341 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-13T17:57:58.384 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2024-09-13T17:57:58.408 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:57:58.408 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:57:58.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:2024-09-13T17:57:58.463+0000 7f265f2d3480 -1 init, newargv = 0x558f1822d690 newargc=15 2024-09-13T17:57:58.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[46945]: starting ceph client 2024-09-13T17:57:58.469 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[46945]: starting fuse 2024-09-13T17:57:58.476 INFO:teuthology.orchestra.run.smithi111.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:57:58.477 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:57:58.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:58.478 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2024-09-13T17:57:58.533 INFO:teuthology.orchestra.run.smithi111.stdout:41 2024-09-13T17:57:58.533 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-13T17:57:58.534 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:57:58.534 DEBUG:teuthology.orchestra.run.smithi111:> import glob 2024-09-13T17:57:58.534 DEBUG:teuthology.orchestra.run.smithi111:> import re 2024-09-13T17:57:58.534 DEBUG:teuthology.orchestra.run.smithi111:> import os 2024-09-13T17:57:58.534 DEBUG:teuthology.orchestra.run.smithi111:> import subprocess 2024-09-13T17:57:58.534 DEBUG:teuthology.orchestra.run.smithi111:> 2024-09-13T17:57:58.534 DEBUG:teuthology.orchestra.run.smithi111:> def _find_admin_socket(client_name): 2024-09-13T17:57:58.534 DEBUG:teuthology.orchestra.run.smithi111:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T17:57:58.534 DEBUG:teuthology.orchestra.run.smithi111:> files = glob.glob(asok_path) 2024-09-13T17:57:58.534 DEBUG:teuthology.orchestra.run.smithi111:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T17:57:58.534 DEBUG:teuthology.orchestra.run.smithi111:> 2024-09-13T17:57:58.534 DEBUG:teuthology.orchestra.run.smithi111:> # Given a non-glob path, it better be there 2024-09-13T17:57:58.534 DEBUG:teuthology.orchestra.run.smithi111:> if "*" not in asok_path: 2024-09-13T17:57:58.534 DEBUG:teuthology.orchestra.run.smithi111:> assert(len(files) == 1) 2024-09-13T17:57:58.534 DEBUG:teuthology.orchestra.run.smithi111:> return files[0] 2024-09-13T17:57:58.534 DEBUG:teuthology.orchestra.run.smithi111:> 2024-09-13T17:57:58.535 DEBUG:teuthology.orchestra.run.smithi111:> for f in files: 2024-09-13T17:57:58.535 DEBUG:teuthology.orchestra.run.smithi111:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:57:58.535 DEBUG:teuthology.orchestra.run.smithi111:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:57:58.535 DEBUG:teuthology.orchestra.run.smithi111:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:57:58.535 DEBUG:teuthology.orchestra.run.smithi111:> contents = proc_f.read() 2024-09-13T17:57:58.535 DEBUG:teuthology.orchestra.run.smithi111:> if mountpoint in contents: 2024-09-13T17:57:58.535 DEBUG:teuthology.orchestra.run.smithi111:> return f 2024-09-13T17:57:58.535 DEBUG:teuthology.orchestra.run.smithi111:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:57:58.535 DEBUG:teuthology.orchestra.run.smithi111:> 2024-09-13T17:57:58.535 DEBUG:teuthology.orchestra.run.smithi111:> print(_find_admin_socket("client.0")) 2024-09-13T17:57:58.535 DEBUG:teuthology.orchestra.run.smithi111:> ' 2024-09-13T17:57:58.647 INFO:teuthology.orchestra.run.smithi111.stdout:/var/run/ceph/ceph-client.0.46945.asok 2024-09-13T17:57:58.833 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46945.asok 2024-09-13T17:57:58.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:58.834 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46945.asok status 2024-09-13T17:57:58.967 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T17:57:58.968 INFO:teuthology.orchestra.run.smithi111.stdout: "metadata": { 2024-09-13T17:57:58.968 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:57:58.968 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:57:58.968 INFO:teuthology.orchestra.run.smithi111.stdout: "entity_id": "0", 2024-09-13T17:57:58.968 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": "smithi111", 2024-09-13T17:57:58.968 INFO:teuthology.orchestra.run.smithi111.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T17:57:58.968 INFO:teuthology.orchestra.run.smithi111.stdout: "pid": "46945", 2024-09-13T17:57:58.968 INFO:teuthology.orchestra.run.smithi111.stdout: "root": "/" 2024-09-13T17:57:58.968 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:57:58.968 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_count": 0, 2024-09-13T17:57:58.968 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_pinned_count": 0, 2024-09-13T17:57:58.968 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4845, 2024-09-13T17:57:58.969 INFO:teuthology.orchestra.run.smithi111.stdout: "inst": { 2024-09-13T17:57:58.969 INFO:teuthology.orchestra.run.smithi111.stdout: "name": { 2024-09-13T17:57:58.969 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "client", 2024-09-13T17:57:58.969 INFO:teuthology.orchestra.run.smithi111.stdout: "num": 4845 2024-09-13T17:57:58.969 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:57:58.969 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2024-09-13T17:57:58.969 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2024-09-13T17:57:58.969 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:57:58.969 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 1459733430 2024-09-13T17:57:58.969 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:57:58.969 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:57:58.969 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2024-09-13T17:57:58.969 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2024-09-13T17:57:58.969 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:57:58.970 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 1459733430 2024-09-13T17:57:58.970 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:57:58.970 INFO:teuthology.orchestra.run.smithi111.stdout: "inst_str": "client.4845 192.168.0.1:0/1459733430", 2024-09-13T17:57:58.970 INFO:teuthology.orchestra.run.smithi111.stdout: "addr_str": "192.168.0.1:0/1459733430", 2024-09-13T17:57:58.970 INFO:teuthology.orchestra.run.smithi111.stdout: "inode_count": 1, 2024-09-13T17:57:58.970 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_epoch": 9, 2024-09-13T17:57:58.970 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch": 24, 2024-09-13T17:57:58.970 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:57:58.970 INFO:teuthology.orchestra.run.smithi111.stdout: "blocklisted": false, 2024-09-13T17:57:58.970 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs" 2024-09-13T17:57:58.970 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T17:57:58.976 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T17:57:58.976 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:57:59.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.072+0000 7fe958945640 1 Processor -- start 2024-09-13T17:57:59.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.072+0000 7fe958945640 1 -- start start 2024-09-13T17:57:59.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.072+0000 7fe958945640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95406c8a0 0x7fe9540b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:59.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.072+0000 7fe958945640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe9540cbea0 0x7fe9540cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:59.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.073+0000 7fe958945640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9540cc890 con 0x7fe95406c4c0 2024-09-13T17:57:59.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.073+0000 7fe958945640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95406deb0 con 0x7fe9540cbea0 2024-09-13T17:57:59.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.073+0000 7fe958945640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9540b2170 con 0x7fe95406c8a0 2024-09-13T17:57:59.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.073+0000 7fe952d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe9540cbea0 0x7fe9540cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:59.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.073+0000 7fe952d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe9540cbea0 0x7fe9540cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53396/0 (socket says 172.21.15.111:53396) 2024-09-13T17:57:59.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.073+0000 7fe952d9d640 1 -- 172.21.15.111:0/2062890985 learned_addr learned my addr 172.21.15.111:0/2062890985 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:59.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.073+0000 7fe951d9b640 1 --2- 172.21.15.111:0/2062890985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95406c8a0 0x7fe9540b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:59.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.073+0000 7fe952d9d640 1 -- 172.21.15.111:0/2062890985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95406c8a0 msgr2=0x7fe9540b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:59.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.073+0000 7fe952d9d640 1 --2- 172.21.15.111:0/2062890985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95406c8a0 0x7fe9540b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:59.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.073+0000 7fe952d9d640 1 -- 172.21.15.111:0/2062890985 >> v1:172.21.15.188:6789/0 conn(0x7fe95406c4c0 legacy=0x7fe95406d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:59.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.073+0000 7fe952d9d640 1 -- 172.21.15.111:0/2062890985 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95406dab0 con 0x7fe9540cbea0 2024-09-13T17:57:59.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.074+0000 7fe952d9d640 1 --2- 172.21.15.111:0/2062890985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9540cbea0 0x7fe9540cc350 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe95406dff0 tx=0x7fe94c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558881f413f6cadd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:59.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.074+0000 7fe95159a640 1 -- 172.21.15.111:0/2062890985 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c031600 con 0x7fe9540cbea0 2024-09-13T17:57:59.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.074+0000 7fe95159a640 1 -- 172.21.15.111:0/2062890985 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c02d050 con 0x7fe9540cbea0 2024-09-13T17:57:59.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.074+0000 7fe95159a640 1 -- 172.21.15.111:0/2062890985 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c031980 con 0x7fe9540cbea0 2024-09-13T17:57:59.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.074+0000 7fe958945640 1 -- 172.21.15.111:0/2062890985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9540cbea0 msgr2=0x7fe9540cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:59.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.074+0000 7fe958945640 1 --2- 172.21.15.111:0/2062890985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9540cbea0 0x7fe9540cc350 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe95406dff0 tx=0x7fe94c02a6f0 comp rx=0 tx=0).stop 2024-09-13T17:57:59.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.074+0000 7fe958945640 1 -- 172.21.15.111:0/2062890985 shutdown_connections 2024-09-13T17:57:59.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.074+0000 7fe958945640 1 --2- 172.21.15.111:0/2062890985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9540cbea0 0x7fe9540cc350 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:59.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.074+0000 7fe958945640 1 --2- 172.21.15.111:0/2062890985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95406c8a0 0x7fe9540b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:59.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.074+0000 7fe958945640 1 -- 172.21.15.111:0/2062890985 >> 172.21.15.111:0/2062890985 conn(0x7fe954069ed0 msgr2=0x7fe95406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:59.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.075+0000 7fe958945640 1 -- 172.21.15.111:0/2062890985 shutdown_connections 2024-09-13T17:57:59.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.075+0000 7fe958945640 1 -- 172.21.15.111:0/2062890985 wait complete. 2024-09-13T17:57:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.076+0000 7fe958945640 1 Processor -- start 2024-09-13T17:57:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.076+0000 7fe958945640 1 -- start start 2024-09-13T17:57:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe958945640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe95406c4c0 0x7fe954158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe958945640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95406c8a0 0x7fe954152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe958945640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9540cbea0 0x7fe9540c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe958945640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe954153f30 con 0x7fe95406c4c0 2024-09-13T17:57:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe958945640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe954072180 con 0x7fe9540cbea0 2024-09-13T17:57:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe958945640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe954153c50 con 0x7fe95406c8a0 2024-09-13T17:57:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe95259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe95406c4c0 0x7fe954158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe95259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe95406c4c0 0x7fe954158f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53398/0 (socket says 172.21.15.111:53398) 2024-09-13T17:57:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe95259c640 1 -- 172.21.15.111:0/134208959 learned_addr learned my addr 172.21.15.111:0/134208959 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:57:59.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe952d9d640 1 --2- 172.21.15.111:0/134208959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9540cbea0 0x7fe9540c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:59.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe951d9b640 1 --2- 172.21.15.111:0/134208959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95406c8a0 0x7fe954152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:59.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe95259c640 1 -- 172.21.15.111:0/134208959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95406c8a0 msgr2=0x7fe954152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:59.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe95259c640 1 --2- 172.21.15.111:0/134208959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95406c8a0 0x7fe954152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:59.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe95259c640 1 -- 172.21.15.111:0/134208959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9540cbea0 msgr2=0x7fe9540c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:59.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe95259c640 1 --2- 172.21.15.111:0/134208959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9540cbea0 0x7fe9540c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:59.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe95259c640 1 -- 172.21.15.111:0/134208959 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c009b30 con 0x7fe95406c4c0 2024-09-13T17:57:59.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe951d9b640 1 --2- 172.21.15.111:0/134208959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95406c8a0 0x7fe954152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:59.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe95259c640 1 --2- 172.21.15.111:0/134208959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe95406c4c0 0x7fe954158f00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe94400e760 tx=0x7fe94400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:59.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe94affd640 1 -- 172.21.15.111:0/134208959 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9440041f0 con 0x7fe95406c4c0 2024-09-13T17:57:59.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe94affd640 1 -- 172.21.15.111:0/134208959 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe944004370 con 0x7fe95406c4c0 2024-09-13T17:57:59.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe958945640 1 -- 172.21.15.111:0/134208959 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe954070230 con 0x7fe95406c4c0 2024-09-13T17:57:59.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe94affd640 1 -- 172.21.15.111:0/134208959 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9440116b0 con 0x7fe95406c4c0 2024-09-13T17:57:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.077+0000 7fe958945640 1 -- 172.21.15.111:0/134208959 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9540707f0 con 0x7fe95406c4c0 2024-09-13T17:57:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.078+0000 7fe958945640 1 -- 172.21.15.111:0/134208959 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe954070090 con 0x7fe95406c4c0 2024-09-13T17:57:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.079+0000 7fe94affd640 1 -- 172.21.15.111:0/134208959 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe944011810 con 0x7fe95406c4c0 2024-09-13T17:57:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.079+0000 7fe94affd640 1 --2- 172.21.15.111:0/134208959 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe93406ac00 0x7fe93406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:59.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.080+0000 7fe951d9b640 1 --2- 172.21.15.111:0/134208959 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe93406ac00 0x7fe93406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:59.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.080+0000 7fe94affd640 1 -- 172.21.15.111:0/134208959 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe94408bc10 con 0x7fe95406c4c0 2024-09-13T17:57:59.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.080+0000 7fe951d9b640 1 --2- 172.21.15.111:0/134208959 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe93406ac00 0x7fe93406d0c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe93c006fd0 tx=0x7fe93c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:59.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.084+0000 7fe94affd640 1 -- 172.21.15.111:0/134208959 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fe954070090 con 0x7fe95406c4c0 2024-09-13T17:57:59.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.277+0000 7fe958945640 1 -- 172.21.15.111:0/134208959 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe954070090 con 0x7fe95406c4c0 2024-09-13T17:57:59.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.278+0000 7fe94affd640 1 -- 172.21.15.111:0/134208959 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fe954070090 con 0x7fe95406c4c0 2024-09-13T17:57:59.280 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T17:57:59.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.280+0000 7fe958945640 1 -- 172.21.15.111:0/134208959 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe93406ac00 msgr2=0x7fe93406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:59.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.280+0000 7fe958945640 1 --2- 172.21.15.111:0/134208959 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe93406ac00 0x7fe93406d0c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe93c006fd0 tx=0x7fe93c008040 comp rx=0 tx=0).stop 2024-09-13T17:57:59.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.280+0000 7fe958945640 1 -- 172.21.15.111:0/134208959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe95406c4c0 msgr2=0x7fe954158f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:59.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.280+0000 7fe958945640 1 --2- 172.21.15.111:0/134208959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe95406c4c0 0x7fe954158f00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe94400e760 tx=0x7fe94400ec20 comp rx=0 tx=0).stop 2024-09-13T17:57:59.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.280+0000 7fe958945640 1 -- 172.21.15.111:0/134208959 shutdown_connections 2024-09-13T17:57:59.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.280+0000 7fe958945640 1 --2- 172.21.15.111:0/134208959 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe93406ac00 0x7fe93406d0c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:59.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.280+0000 7fe958945640 1 --2- 172.21.15.111:0/134208959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9540cbea0 0x7fe9540c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:59.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.280+0000 7fe958945640 1 --2- 172.21.15.111:0/134208959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95406c8a0 0x7fe954152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:59.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.280+0000 7fe958945640 1 --2- 172.21.15.111:0/134208959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe95406c4c0 0x7fe954158f00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:59.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.280+0000 7fe958945640 1 -- 172.21.15.111:0/134208959 >> 172.21.15.111:0/134208959 conn(0x7fe954069ed0 msgr2=0x7fe95406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:59.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.280+0000 7fe958945640 1 -- 172.21.15.111:0/134208959 shutdown_connections 2024-09-13T17:57:59.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:57:59.280+0000 7fe958945640 1 -- 172.21.15.111:0/134208959 wait complete. 2024-09-13T17:57:59.293 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:57:59.293 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:57:59.293 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2024-09-13T17:57:59.293 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-13T17:57:59.293 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-13T17:57:59.293 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:57:59.293 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:57:59.293 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:57:59.293 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-13T17:57:59.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:59.294 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2024-09-13T17:57:59.309 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:57:59.309 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:57:59.310 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:57:59.310 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:59.310 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-09-13T17:57:59.310 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:59.310 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:57:59.310 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:57:59.310 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2024-09-13T17:57:59.310 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:57:59.310 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:57:59.310 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2024-09-13T17:57:59.310 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:57:59.310 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2024-09-13T17:57:59.310 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:57:59.311 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2024-09-13T17:57:59.311 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-13T17:57:59.311 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42780sec preferred_lft 42780sec 2024-09-13T17:57:59.311 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link noprefixroute 2024-09-13T17:57:59.311 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:59.311 INFO:teuthology.orchestra.run.smithi111.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:57:59.311 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether da:3f:2a:41:1d:dc brd ff:ff:ff:ff:ff:ff 2024-09-13T17:57:59.311 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T17:57:59.311 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:59.311 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::d4ae:99ff:fea4:a069/64 scope link tentative 2024-09-13T17:57:59.311 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:59.311 INFO:teuthology.orchestra.run.smithi111.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:57:59.311 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether da:3f:2a:41:1d:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:57:59.311 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::d83f:2aff:fe41:1ddc/64 scope link tentative 2024-09-13T17:57:59.312 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:59.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:59.312 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:57:59.373 INFO:teuthology.orchestra.run.smithi111.stdout:1 2024-09-13T17:57:59.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:59.374 DEBUG:teuthology.orchestra.run.smithi111:> route 2024-09-13T17:57:59.430 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2024-09-13T17:57:59.430 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:57:59.431 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:57:59.431 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:57:59.431 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:57:59.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:59.431 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2024-09-13T17:57:59.431 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-13T17:57:59.431 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:57:59.431 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:57:59.431 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:57:59.431 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-13T17:57:59.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:59.518 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-09-13T17:57:59.533 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:57:59.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:59.533 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list-id 2024-09-13T17:57:59.589 INFO:teuthology.orchestra.run.smithi111.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-13T17:57:59.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:59.589 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2024-09-13T17:57:59.589 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-13T17:57:59.590 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T17:57:59.590 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-13T17:57:59.590 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-13T17:57:59.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:59.667 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-13T17:57:59.732 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:57:59.732 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:57:59.732 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:57:59.732 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:59.732 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-09-13T17:57:59.732 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:59.732 INFO:teuthology.orchestra.run.smithi111.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:57:59.732 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether c6:0b:9c:d1:27:52 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T17:57:59.732 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T17:57:59.733 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:59.733 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::c40b:9cff:fed1:2752/64 scope link 2024-09-13T17:57:59.733 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:59.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:59.734 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-13T17:57:59.798 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:57:59.799 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:57:59.799 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:57:59.799 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:59.799 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-09-13T17:57:59.799 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:59.799 INFO:teuthology.orchestra.run.smithi111.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:57:59.799 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether c6:0b:9c:d1:27:52 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T17:57:59.799 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T17:57:59.799 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:59.799 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::c40b:9cff:fed1:2752/64 scope link 2024-09-13T17:57:59.799 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:57:59.800 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-13T17:57:59.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:59.801 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2024-09-13T17:57:59.801 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-13T17:57:59.801 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-13T17:57:59.801 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-13T17:57:59.801 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-13T17:57:59.801 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-13T17:57:59.801 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-13T17:57:59.801 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-13T17:57:59.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:57:59.913 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2024-09-13T17:57:59.913 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-13T17:57:59.913 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.1 up 2024-09-13T17:57:59.913 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set dev brx.1 master ceph-brx 2024-09-13T17:57:59.913 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-13T17:57:59.950 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-09-13T17:57:59.951 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:57:59.951 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-13T17:58:00.006 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-13T17:58:00.006 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:58:00.006 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-13T17:58:00.061 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2024-09-13T17:58:00.126 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:58:00.127 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:58:00.190 INFO:teuthology.orchestra.run.smithi111.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:58:00.191 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:58:00.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:00.192 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2024-09-13T17:58:00.247 INFO:teuthology.orchestra.run.smithi111.stdout:41 2024-09-13T17:58:00.247 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-09-13T17:58:00.248 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-09-13T17:58:00.290 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2024-09-13T17:58:00.315 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:58:00.315 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:58:00.371 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi111.stderr:2024-09-13T17:58:00.369+0000 7f71f5a12480 -1 init, newargv = 0x55dc6c1a09f0 newargc=15 2024-09-13T17:58:00.371 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi111.stderr:ceph-fuse[47440]: starting ceph client 2024-09-13T17:58:00.376 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi111.stderr:ceph-fuse[47440]: starting fuse 2024-09-13T17:58:00.381 INFO:teuthology.orchestra.run.smithi111.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:58:00.382 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:58:00.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:00.383 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2024-09-13T17:58:00.438 INFO:teuthology.orchestra.run.smithi111.stdout:41 2024-09-13T17:58:00.438 INFO:teuthology.orchestra.run.smithi111.stdout:44 2024-09-13T17:58:00.438 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2024-09-13T17:58:00.439 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:58:00.439 DEBUG:teuthology.orchestra.run.smithi111:> import glob 2024-09-13T17:58:00.439 DEBUG:teuthology.orchestra.run.smithi111:> import re 2024-09-13T17:58:00.439 DEBUG:teuthology.orchestra.run.smithi111:> import os 2024-09-13T17:58:00.439 DEBUG:teuthology.orchestra.run.smithi111:> import subprocess 2024-09-13T17:58:00.439 DEBUG:teuthology.orchestra.run.smithi111:> 2024-09-13T17:58:00.439 DEBUG:teuthology.orchestra.run.smithi111:> def _find_admin_socket(client_name): 2024-09-13T17:58:00.439 DEBUG:teuthology.orchestra.run.smithi111:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-09-13T17:58:00.439 DEBUG:teuthology.orchestra.run.smithi111:> files = glob.glob(asok_path) 2024-09-13T17:58:00.439 DEBUG:teuthology.orchestra.run.smithi111:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-09-13T17:58:00.439 DEBUG:teuthology.orchestra.run.smithi111:> 2024-09-13T17:58:00.439 DEBUG:teuthology.orchestra.run.smithi111:> # Given a non-glob path, it better be there 2024-09-13T17:58:00.440 DEBUG:teuthology.orchestra.run.smithi111:> if "*" not in asok_path: 2024-09-13T17:58:00.440 DEBUG:teuthology.orchestra.run.smithi111:> assert(len(files) == 1) 2024-09-13T17:58:00.440 DEBUG:teuthology.orchestra.run.smithi111:> return files[0] 2024-09-13T17:58:00.440 DEBUG:teuthology.orchestra.run.smithi111:> 2024-09-13T17:58:00.440 DEBUG:teuthology.orchestra.run.smithi111:> for f in files: 2024-09-13T17:58:00.440 DEBUG:teuthology.orchestra.run.smithi111:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:58:00.440 DEBUG:teuthology.orchestra.run.smithi111:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:58:00.440 DEBUG:teuthology.orchestra.run.smithi111:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:58:00.440 DEBUG:teuthology.orchestra.run.smithi111:> contents = proc_f.read() 2024-09-13T17:58:00.440 DEBUG:teuthology.orchestra.run.smithi111:> if mountpoint in contents: 2024-09-13T17:58:00.440 DEBUG:teuthology.orchestra.run.smithi111:> return f 2024-09-13T17:58:00.440 DEBUG:teuthology.orchestra.run.smithi111:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:58:00.440 DEBUG:teuthology.orchestra.run.smithi111:> 2024-09-13T17:58:00.440 DEBUG:teuthology.orchestra.run.smithi111:> print(_find_admin_socket("client.2")) 2024-09-13T17:58:00.440 DEBUG:teuthology.orchestra.run.smithi111:> ' 2024-09-13T17:58:00.552 INFO:teuthology.orchestra.run.smithi111.stdout:/var/run/ceph/ceph-client.2.47440.asok 2024-09-13T17:58:00.739 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.47440.asok 2024-09-13T17:58:00.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:00.739 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.47440.asok status 2024-09-13T17:58:00.873 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T17:58:00.873 INFO:teuthology.orchestra.run.smithi111.stdout: "metadata": { 2024-09-13T17:58:00.873 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:58:00.873 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:58:00.873 INFO:teuthology.orchestra.run.smithi111.stdout: "entity_id": "2", 2024-09-13T17:58:00.873 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": "smithi111", 2024-09-13T17:58:00.873 INFO:teuthology.orchestra.run.smithi111.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-09-13T17:58:00.873 INFO:teuthology.orchestra.run.smithi111.stdout: "pid": "47440", 2024-09-13T17:58:00.873 INFO:teuthology.orchestra.run.smithi111.stdout: "root": "/" 2024-09-13T17:58:00.873 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:00.874 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_count": 0, 2024-09-13T17:58:00.874 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_pinned_count": 0, 2024-09-13T17:58:00.874 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4857, 2024-09-13T17:58:00.874 INFO:teuthology.orchestra.run.smithi111.stdout: "inst": { 2024-09-13T17:58:00.874 INFO:teuthology.orchestra.run.smithi111.stdout: "name": { 2024-09-13T17:58:00.874 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "client", 2024-09-13T17:58:00.874 INFO:teuthology.orchestra.run.smithi111.stdout: "num": 4857 2024-09-13T17:58:00.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:00.874 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2024-09-13T17:58:00.874 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2024-09-13T17:58:00.874 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:58:00.874 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 2979777286 2024-09-13T17:58:00.874 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:00.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:00.875 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2024-09-13T17:58:00.875 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2024-09-13T17:58:00.875 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:58:00.875 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 2979777286 2024-09-13T17:58:00.875 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:00.875 INFO:teuthology.orchestra.run.smithi111.stdout: "inst_str": "client.4857 192.168.0.2:0/2979777286", 2024-09-13T17:58:00.875 INFO:teuthology.orchestra.run.smithi111.stdout: "addr_str": "192.168.0.2:0/2979777286", 2024-09-13T17:58:00.875 INFO:teuthology.orchestra.run.smithi111.stdout: "inode_count": 1, 2024-09-13T17:58:00.875 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_epoch": 9, 2024-09-13T17:58:00.875 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch": 24, 2024-09-13T17:58:00.875 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:58:00.875 INFO:teuthology.orchestra.run.smithi111.stdout: "blocklisted": false, 2024-09-13T17:58:00.875 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs" 2024-09-13T17:58:00.875 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T17:58:00.881 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T17:58:00.881 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:58:00.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.977+0000 7ffb4869c640 1 Processor -- start 2024-09-13T17:58:00.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.977+0000 7ffb4869c640 1 -- start start 2024-09-13T17:58:00.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.978+0000 7ffb4869c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb4006c4c0 0x7ffb4006c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:00.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.978+0000 7ffb4869c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400b2590 0x7ffb4006d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:00.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.978+0000 7ffb4869c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4006e0c0 con 0x7ffb400cbea0 2024-09-13T17:58:00.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.978+0000 7ffb4869c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4006deb0 con 0x7ffb4006c4c0 2024-09-13T17:58:00.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.978+0000 7ffb4869c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb400b2170 con 0x7ffb400b2590 2024-09-13T17:58:00.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.978+0000 7ffb45c37640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400b2590 0x7ffb4006d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:00.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.978+0000 7ffb46438640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb4006c4c0 0x7ffb4006c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:00.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.978+0000 7ffb45c37640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400b2590 0x7ffb4006d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:58078/0 (socket says 172.21.15.111:58078) 2024-09-13T17:58:00.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.978+0000 7ffb45c37640 1 -- 172.21.15.111:0/3180536136 learned_addr learned my addr 172.21.15.111:0/3180536136 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:00.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.978+0000 7ffb46438640 1 -- 172.21.15.111:0/3180536136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400b2590 msgr2=0x7ffb4006d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:00.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.978+0000 7ffb46438640 1 --2- 172.21.15.111:0/3180536136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400b2590 0x7ffb4006d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:00.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.978+0000 7ffb46438640 1 -- 172.21.15.111:0/3180536136 >> v1:172.21.15.188:6789/0 conn(0x7ffb400cbea0 legacy=0x7ffb400cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:00.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.978+0000 7ffb46438640 1 -- 172.21.15.111:0/3180536136 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4006e3a0 con 0x7ffb4006c4c0 2024-09-13T17:58:00.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.978+0000 7ffb45c37640 1 --2- 172.21.15.111:0/3180536136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400b2590 0x7ffb4006d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:00.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.979+0000 7ffb46438640 1 --2- 172.21.15.111:0/3180536136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb4006c4c0 0x7ffb4006c8a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0097d0 tx=0x7ffb3c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2f5f89b99543435 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:00.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.979+0000 7ffb45436640 1 -- 172.21.15.111:0/3180536136 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c032440 con 0x7ffb4006c4c0 2024-09-13T17:58:00.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.979+0000 7ffb45436640 1 -- 172.21.15.111:0/3180536136 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c0325a0 con 0x7ffb4006c4c0 2024-09-13T17:58:00.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.979+0000 7ffb45436640 1 -- 172.21.15.111:0/3180536136 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c032930 con 0x7ffb4006c4c0 2024-09-13T17:58:00.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.980+0000 7ffb4869c640 1 -- 172.21.15.111:0/3180536136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb4006c4c0 msgr2=0x7ffb4006c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:00.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.980+0000 7ffb4869c640 1 --2- 172.21.15.111:0/3180536136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb4006c4c0 0x7ffb4006c8a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0097d0 tx=0x7ffb3c02a1c0 comp rx=0 tx=0).stop 2024-09-13T17:58:00.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.980+0000 7ffb4869c640 1 -- 172.21.15.111:0/3180536136 shutdown_connections 2024-09-13T17:58:00.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.980+0000 7ffb4869c640 1 --2- 172.21.15.111:0/3180536136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400b2590 0x7ffb4006d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:00.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.980+0000 7ffb4869c640 1 --2- 172.21.15.111:0/3180536136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb4006c4c0 0x7ffb4006c8a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:00.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.980+0000 7ffb4869c640 1 -- 172.21.15.111:0/3180536136 >> 172.21.15.111:0/3180536136 conn(0x7ffb40069ed0 msgr2=0x7ffb4006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:00.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.980+0000 7ffb4869c640 1 -- 172.21.15.111:0/3180536136 shutdown_connections 2024-09-13T17:58:00.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.980+0000 7ffb4869c640 1 -- 172.21.15.111:0/3180536136 wait complete. 2024-09-13T17:58:00.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.981+0000 7ffb4869c640 1 Processor -- start 2024-09-13T17:58:00.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.981+0000 7ffb4869c640 1 -- start start 2024-09-13T17:58:00.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb4869c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb4006c4c0 0x7ffb400b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:00.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb4869c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb400b2590 0x7ffb401f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:00.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb4869c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400cbea0 0x7ffb401f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:00.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb4869c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb401f2360 con 0x7ffb400b2590 2024-09-13T17:58:00.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb4869c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb401f2640 con 0x7ffb4006c4c0 2024-09-13T17:58:00.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb4869c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb401f2920 con 0x7ffb400cbea0 2024-09-13T17:58:00.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb45c37640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb400b2590 0x7ffb401f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:00.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb45c37640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb400b2590 0x7ffb401f2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53408/0 (socket says 172.21.15.111:53408) 2024-09-13T17:58:00.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb45c37640 1 -- 172.21.15.111:0/3234086246 learned_addr learned my addr 172.21.15.111:0/3234086246 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:00.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb46438640 1 --2- 172.21.15.111:0/3234086246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb4006c4c0 0x7ffb400b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:00.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb46c39640 1 --2- 172.21.15.111:0/3234086246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400cbea0 0x7ffb401f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:00.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb45c37640 1 -- 172.21.15.111:0/3234086246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400cbea0 msgr2=0x7ffb401f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:00.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb45c37640 1 --2- 172.21.15.111:0/3234086246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400cbea0 0x7ffb401f4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:00.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb45c37640 1 -- 172.21.15.111:0/3234086246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb4006c4c0 msgr2=0x7ffb400b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:00.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb45c37640 1 --2- 172.21.15.111:0/3234086246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb4006c4c0 0x7ffb400b0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:00.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb45c37640 1 -- 172.21.15.111:0/3234086246 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3000b760 con 0x7ffb400b2590 2024-09-13T17:58:00.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb45c37640 1 --2- 172.21.15.111:0/3234086246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb400b2590 0x7ffb401f2f70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000de30 tx=0x7ffb3000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:00.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb36ffd640 1 -- 172.21.15.111:0/3234086246 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb300136d0 con 0x7ffb400b2590 2024-09-13T17:58:00.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb36ffd640 1 -- 172.21.15.111:0/3234086246 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb30013830 con 0x7ffb400b2590 2024-09-13T17:58:00.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.982+0000 7ffb4869c640 1 -- 172.21.15.111:0/3234086246 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3c009400 con 0x7ffb400b2590 2024-09-13T17:58:00.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.983+0000 7ffb36ffd640 1 -- 172.21.15.111:0/3234086246 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30013be0 con 0x7ffb400b2590 2024-09-13T17:58:00.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.983+0000 7ffb4869c640 1 -- 172.21.15.111:0/3234086246 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb401f2050 con 0x7ffb400b2590 2024-09-13T17:58:00.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.984+0000 7ffb4869c640 1 -- 172.21.15.111:0/3234086246 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4006c8a0 con 0x7ffb400b2590 2024-09-13T17:58:00.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.984+0000 7ffb36ffd640 1 -- 172.21.15.111:0/3234086246 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ffb30013d40 con 0x7ffb400b2590 2024-09-13T17:58:00.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.984+0000 7ffb36ffd640 1 --2- 172.21.15.111:0/3234086246 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb2406ac00 0x7ffb2406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:00.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.985+0000 7ffb36ffd640 1 -- 172.21.15.111:0/3234086246 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ffb3000b050 con 0x7ffb400b2590 2024-09-13T17:58:00.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.985+0000 7ffb46438640 1 --2- 172.21.15.111:0/3234086246 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb2406ac00 0x7ffb2406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:00.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.985+0000 7ffb46438640 1 --2- 172.21.15.111:0/3234086246 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb2406ac00 0x7ffb2406d0c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c034000 tx=0x7ffb3c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:00.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:00.989+0000 7ffb36ffd640 1 -- 172.21.15.111:0/3234086246 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ffb30054d50 con 0x7ffb400b2590 2024-09-13T17:58:01.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:01.181+0000 7ffb4869c640 1 -- 172.21.15.111:0/3234086246 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffb400b3100 con 0x7ffb400b2590 2024-09-13T17:58:01.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:01.182+0000 7ffb36ffd640 1 -- 172.21.15.111:0/3234086246 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7ffb3001b020 con 0x7ffb400b2590 2024-09-13T17:58:01.184 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T17:58:01.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:01.184+0000 7ffb4869c640 1 -- 172.21.15.111:0/3234086246 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb2406ac00 msgr2=0x7ffb2406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:01.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:01.184+0000 7ffb4869c640 1 --2- 172.21.15.111:0/3234086246 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb2406ac00 0x7ffb2406d0c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c034000 tx=0x7ffb3c02e000 comp rx=0 tx=0).stop 2024-09-13T17:58:01.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:01.184+0000 7ffb4869c640 1 -- 172.21.15.111:0/3234086246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb400b2590 msgr2=0x7ffb401f2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:01.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:01.184+0000 7ffb4869c640 1 --2- 172.21.15.111:0/3234086246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb400b2590 0x7ffb401f2f70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000de30 tx=0x7ffb3000c7b0 comp rx=0 tx=0).stop 2024-09-13T17:58:01.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:01.184+0000 7ffb4869c640 1 -- 172.21.15.111:0/3234086246 shutdown_connections 2024-09-13T17:58:01.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:01.184+0000 7ffb4869c640 1 --2- 172.21.15.111:0/3234086246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400cbea0 0x7ffb401f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:01.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:01.184+0000 7ffb4869c640 1 --2- 172.21.15.111:0/3234086246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb400b2590 0x7ffb401f2f70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:01.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:01.184+0000 7ffb4869c640 1 --2- 172.21.15.111:0/3234086246 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb2406ac00 0x7ffb2406d0c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:01.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:01.184+0000 7ffb4869c640 1 --2- 172.21.15.111:0/3234086246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb4006c4c0 0x7ffb400b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:01.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:01.184+0000 7ffb4869c640 1 -- 172.21.15.111:0/3234086246 >> 172.21.15.111:0/3234086246 conn(0x7ffb40069ed0 msgr2=0x7ffb401571c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:01.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:01.184+0000 7ffb4869c640 1 -- 172.21.15.111:0/3234086246 shutdown_connections 2024-09-13T17:58:01.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:01.184+0000 7ffb4869c640 1 -- 172.21.15.111:0/3234086246 wait complete. 2024-09-13T17:58:01.196 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:58:01.196 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:58:01.196 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-09-13T17:58:01.197 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-13T17:58:01.197 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T17:58:01.197 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:58:01.197 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:58:01.197 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:58:01.197 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:58:01.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:01.197 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-09-13T17:58:01.216 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:58:01.217 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:58:01.217 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:58:01.217 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:01.217 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-09-13T17:58:01.217 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:01.217 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:58:01.217 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:58:01.217 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-09-13T17:58:01.217 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:58:01.217 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:58:01.217 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-09-13T17:58:01.218 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:58:01.218 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:58:01.218 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:58:01.218 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:58:01.218 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-13T17:58:01.218 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42758sec preferred_lft 42758sec 2024-09-13T17:58:01.218 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link noprefixroute 2024-09-13T17:58:01.218 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:01.218 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T17:58:01.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:01.219 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-09-13T17:58:01.219 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-09-13T17:58:01.219 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add name ceph-brx type bridge 2024-09-13T17:58:01.219 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr flush dev ceph-brx 2024-09-13T17:58:01.219 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set ceph-brx up 2024-09-13T17:58:01.219 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T17:58:01.219 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-09-13T17:58:01.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:01.403 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:58:01.427 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-09-13T17:58:01.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:01.429 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-09-13T17:58:01.497 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-09-13T17:58:01.497 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:58:01.497 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:58:01.497 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:58:01.497 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:58:01.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:01.498 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-09-13T17:58:01.498 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-09-13T17:58:01.498 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:58:01.498 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:58:01.498 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:58:01.498 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-09-13T17:58:01.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:01.809 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-09-13T17:58:01.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:01.825 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2024-09-13T17:58:01.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:01.881 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-09-13T17:58:01.881 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-09-13T17:58:01.881 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T17:58:01.881 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-13T17:58:01.881 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-09-13T17:58:01.959 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-13T17:58:01.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:01.959 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-09-13T17:58:01.959 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-09-13T17:58:01.959 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-13T17:58:01.960 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T17:58:01.960 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-13T17:58:01.960 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-13T17:58:01.960 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-13T17:58:01.960 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-09-13T17:58:02.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:02.104 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-09-13T17:58:02.104 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-09-13T17:58:02.104 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2024-09-13T17:58:02.104 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T17:58:02.104 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-09-13T17:58:02.141 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-13T17:58:02.141 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:58:02.141 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-13T17:58:02.197 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:58:02.197 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:58:02.198 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-13T17:58:02.253 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2024-09-13T17:58:02.319 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:58:02.319 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:58:02.383 INFO:teuthology.orchestra.run.smithi188.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:58:02.384 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:58:02.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:02.385 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2024-09-13T17:58:02.439 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T17:58:02.439 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-13T17:58:02.481 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2024-09-13T17:58:02.506 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:58:02.506 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:58:02.571 INFO:teuthology.orchestra.run.smithi188.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:58:02.572 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:58:02.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:02.573 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2024-09-13T17:58:02.584 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:2024-09-13T17:58:02.583+0000 7f76042f9480 -1 init, newargv = 0x55ef4d55d1b0 newargc=15 2024-09-13T17:58:02.585 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[42883]: starting ceph client 2024-09-13T17:58:02.589 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[42883]: starting fuse 2024-09-13T17:58:02.600 INFO:teuthology.orchestra.run.smithi188.stdout:42 2024-09-13T17:58:02.601 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-13T17:58:02.601 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:58:02.601 DEBUG:teuthology.orchestra.run.smithi188:> import glob 2024-09-13T17:58:02.601 DEBUG:teuthology.orchestra.run.smithi188:> import re 2024-09-13T17:58:02.601 DEBUG:teuthology.orchestra.run.smithi188:> import os 2024-09-13T17:58:02.602 DEBUG:teuthology.orchestra.run.smithi188:> import subprocess 2024-09-13T17:58:02.602 DEBUG:teuthology.orchestra.run.smithi188:> 2024-09-13T17:58:02.602 DEBUG:teuthology.orchestra.run.smithi188:> def _find_admin_socket(client_name): 2024-09-13T17:58:02.602 DEBUG:teuthology.orchestra.run.smithi188:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-13T17:58:02.602 DEBUG:teuthology.orchestra.run.smithi188:> files = glob.glob(asok_path) 2024-09-13T17:58:02.603 DEBUG:teuthology.orchestra.run.smithi188:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-13T17:58:02.603 DEBUG:teuthology.orchestra.run.smithi188:> 2024-09-13T17:58:02.603 DEBUG:teuthology.orchestra.run.smithi188:> # Given a non-glob path, it better be there 2024-09-13T17:58:02.603 DEBUG:teuthology.orchestra.run.smithi188:> if "*" not in asok_path: 2024-09-13T17:58:02.603 DEBUG:teuthology.orchestra.run.smithi188:> assert(len(files) == 1) 2024-09-13T17:58:02.603 DEBUG:teuthology.orchestra.run.smithi188:> return files[0] 2024-09-13T17:58:02.604 DEBUG:teuthology.orchestra.run.smithi188:> 2024-09-13T17:58:02.604 DEBUG:teuthology.orchestra.run.smithi188:> for f in files: 2024-09-13T17:58:02.604 DEBUG:teuthology.orchestra.run.smithi188:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:58:02.604 DEBUG:teuthology.orchestra.run.smithi188:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:58:02.604 DEBUG:teuthology.orchestra.run.smithi188:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:58:02.604 DEBUG:teuthology.orchestra.run.smithi188:> contents = proc_f.read() 2024-09-13T17:58:02.605 DEBUG:teuthology.orchestra.run.smithi188:> if mountpoint in contents: 2024-09-13T17:58:02.605 DEBUG:teuthology.orchestra.run.smithi188:> return f 2024-09-13T17:58:02.605 DEBUG:teuthology.orchestra.run.smithi188:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:58:02.605 DEBUG:teuthology.orchestra.run.smithi188:> 2024-09-13T17:58:02.605 DEBUG:teuthology.orchestra.run.smithi188:> print(_find_admin_socket("client.1")) 2024-09-13T17:58:02.605 DEBUG:teuthology.orchestra.run.smithi188:> ' 2024-09-13T17:58:02.714 INFO:teuthology.orchestra.run.smithi188.stdout:/var/run/ceph/ceph-client.1.42883.asok 2024-09-13T17:58:02.901 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.42883.asok 2024-09-13T17:58:02.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:02.901 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.42883.asok status 2024-09-13T17:58:03.035 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-09-13T17:58:03.035 INFO:teuthology.orchestra.run.smithi188.stdout: "metadata": { 2024-09-13T17:58:03.035 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:58:03.035 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:58:03.035 INFO:teuthology.orchestra.run.smithi188.stdout: "entity_id": "1", 2024-09-13T17:58:03.035 INFO:teuthology.orchestra.run.smithi188.stdout: "hostname": "smithi188", 2024-09-13T17:58:03.035 INFO:teuthology.orchestra.run.smithi188.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-13T17:58:03.035 INFO:teuthology.orchestra.run.smithi188.stdout: "pid": "42883", 2024-09-13T17:58:03.035 INFO:teuthology.orchestra.run.smithi188.stdout: "root": "/" 2024-09-13T17:58:03.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-09-13T17:58:03.036 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_count": 0, 2024-09-13T17:58:03.036 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_pinned_count": 0, 2024-09-13T17:58:03.036 INFO:teuthology.orchestra.run.smithi188.stdout: "id": 4780, 2024-09-13T17:58:03.036 INFO:teuthology.orchestra.run.smithi188.stdout: "inst": { 2024-09-13T17:58:03.036 INFO:teuthology.orchestra.run.smithi188.stdout: "name": { 2024-09-13T17:58:03.036 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "client", 2024-09-13T17:58:03.036 INFO:teuthology.orchestra.run.smithi188.stdout: "num": 4780 2024-09-13T17:58:03.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-09-13T17:58:03.036 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2024-09-13T17:58:03.036 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2024-09-13T17:58:03.036 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:58:03.036 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 3993434510 2024-09-13T17:58:03.037 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-09-13T17:58:03.037 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-09-13T17:58:03.037 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2024-09-13T17:58:03.037 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2024-09-13T17:58:03.037 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:58:03.037 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 3993434510 2024-09-13T17:58:03.037 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-09-13T17:58:03.037 INFO:teuthology.orchestra.run.smithi188.stdout: "inst_str": "client.4780 192.168.0.1:0/3993434510", 2024-09-13T17:58:03.037 INFO:teuthology.orchestra.run.smithi188.stdout: "addr_str": "192.168.0.1:0/3993434510", 2024-09-13T17:58:03.037 INFO:teuthology.orchestra.run.smithi188.stdout: "inode_count": 1, 2024-09-13T17:58:03.038 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_epoch": 9, 2024-09-13T17:58:03.038 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch": 24, 2024-09-13T17:58:03.038 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:58:03.038 INFO:teuthology.orchestra.run.smithi188.stdout: "blocklisted": false, 2024-09-13T17:58:03.038 INFO:teuthology.orchestra.run.smithi188.stdout: "fs_name": "cephfs" 2024-09-13T17:58:03.038 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-09-13T17:58:03.043 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T17:58:03.044 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:58:03.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.152+0000 7f0845ded640 1 Processor -- start 2024-09-13T17:58:03.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.152+0000 7f0845ded640 1 -- start start 2024-09-13T17:58:03.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.152+0000 7f0845ded640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f084015c6e0 0x7f08400b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:03.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.152+0000 7f0845ded640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08400d07b0 0x7f08400b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:03.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f0845ded640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08400d1840 con 0x7f08400ccb20 2024-09-13T17:58:03.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f0845ded640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f084006bd20 con 0x7f084015c6e0 2024-09-13T17:58:03.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f0845ded640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08400d0c30 con 0x7f08400d07b0 2024-09-13T17:58:03.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083ffff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f08400ccb20 0x7f0840160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58766/0 (socket says 172.21.15.111:58766) 2024-09-13T17:58:03.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083ffff640 1 -- 172.21.15.111:0/3525489164 learned_addr learned my addr 172.21.15.111:0/3525489164 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:03.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f0837fff640 1 --2- 172.21.15.111:0/3525489164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08400d07b0 0x7f08400b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:03.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083f7fe640 1 --2- 172.21.15.111:0/3525489164 >> v2:172.21.15.188:3300/0 conn(0x7f084015c6e0 0x7f08400b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:03.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083effd640 1 -- 172.21.15.111:0/3525489164 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048458880 0 0) 0x7f08400d1840 con 0x7f08400ccb20 2024-09-13T17:58:03.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083effd640 1 -- 172.21.15.111:0/3525489164 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f082c003520 con 0x7f08400ccb20 2024-09-13T17:58:03.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083effd640 1 -- 172.21.15.111:0/3525489164 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1114472710 0 0) 0x7f082c003520 con 0x7f08400ccb20 2024-09-13T17:58:03.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083effd640 1 -- 172.21.15.111:0/3525489164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08400d07b0 msgr2=0x7f08400b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:03.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083effd640 1 --2- 172.21.15.111:0/3525489164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08400d07b0 0x7f08400b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:03.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083effd640 1 -- 172.21.15.111:0/3525489164 >> v2:172.21.15.188:3300/0 conn(0x7f084015c6e0 msgr2=0x7f08400b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:03.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083effd640 1 --2- 172.21.15.111:0/3525489164 >> v2:172.21.15.188:3300/0 conn(0x7f084015c6e0 0x7f08400b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083effd640 1 -- 172.21.15.111:0/3525489164 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08400d0f10 con 0x7f08400ccb20 2024-09-13T17:58:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083effd640 1 -- 172.21.15.111:0/3525489164 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f08380019f0 con 0x7f08400ccb20 2024-09-13T17:58:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f0837fff640 1 --2- 172.21.15.111:0/3525489164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08400d07b0 0x7f08400b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083effd640 1 -- 172.21.15.111:0/3525489164 >> v1:172.21.15.111:6789/0 conn(0x7f08400ccb20 legacy=0x7f0840160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083effd640 1 --2- 172.21.15.111:0/3525489164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f082c003970 0x7f082c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083effd640 1 --2- 172.21.15.111:0/3525489164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08400d07b0 0x7f082c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083effd640 1 --2- 172.21.15.111:0/3525489164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f082c004d00 0x7f082c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083effd640 1 -- 172.21.15.111:0/3525489164 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084006bd20 con 0x7f082c003970 2024-09-13T17:58:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083effd640 1 -- 172.21.15.111:0/3525489164 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08400d0c30 con 0x7f082c004d00 2024-09-13T17:58:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083effd640 1 -- 172.21.15.111:0/3525489164 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082c00b980 con 0x7f08400d07b0 2024-09-13T17:58:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.153+0000 7f083f7fe640 1 --2- 172.21.15.111:0/3525489164 >> v2:172.21.15.188:3300/0 conn(0x7f084015c6e0 0x7f08400b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.154+0000 7f0837fff640 1 --2- 172.21.15.111:0/3525489164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08400d07b0 0x7f082c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.154+0000 7f083f7fe640 1 --2- 172.21.15.111:0/3525489164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f082c003970 0x7f082c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.154+0000 7f083ffff640 1 --2- 172.21.15.111:0/3525489164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f082c004d00 0x7f082c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:03.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.154+0000 7f0837fff640 1 -- 172.21.15.111:0/3525489164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f082c004d00 msgr2=0x7f082c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:03.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.154+0000 7f0837fff640 1 --2- 172.21.15.111:0/3525489164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f082c004d00 0x7f082c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:03.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.154+0000 7f0837fff640 1 -- 172.21.15.111:0/3525489164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f082c003970 msgr2=0x7f082c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:03.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.154+0000 7f0837fff640 1 --2- 172.21.15.111:0/3525489164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f082c003970 0x7f082c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:03.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.154+0000 7f0837fff640 1 -- 172.21.15.111:0/3525489164 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0828009400 con 0x7f08400d07b0 2024-09-13T17:58:03.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.154+0000 7f083f7fe640 1 --2- 172.21.15.111:0/3525489164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f082c003970 0x7f082c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:03.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.155+0000 7f0837fff640 1 --2- 172.21.15.111:0/3525489164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08400d07b0 0x7f082c0047c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f084015ff90 tx=0x7f082c0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a6ab0721b24b72 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:03.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.155+0000 7f083effd640 1 -- 172.21.15.111:0/3525489164 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828031400 con 0x7f08400d07b0 2024-09-13T17:58:03.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.155+0000 7f083effd640 1 -- 172.21.15.111:0/3525489164 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082802d050 con 0x7f08400d07b0 2024-09-13T17:58:03.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.155+0000 7f083effd640 1 -- 172.21.15.111:0/3525489164 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828031760 con 0x7f08400d07b0 2024-09-13T17:58:03.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.155+0000 7f0845ded640 1 -- 172.21.15.111:0/3525489164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08400d07b0 msgr2=0x7f082c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:03.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.155+0000 7f0845ded640 1 --2- 172.21.15.111:0/3525489164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08400d07b0 0x7f082c0047c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f084015ff90 tx=0x7f082c0057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:03.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.155+0000 7f083ffff640 1 -- 172.21.15.111:0/3525489164 reap_dead start 2024-09-13T17:58:03.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.155+0000 7f0845ded640 1 -- 172.21.15.111:0/3525489164 shutdown_connections 2024-09-13T17:58:03.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.155+0000 7f0845ded640 1 -- 172.21.15.111:0/3525489164 >> 172.21.15.111:0/3525489164 conn(0x7f0840089f60 msgr2=0x7f08400d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:03.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.155+0000 7f0845ded640 1 -- 172.21.15.111:0/3525489164 shutdown_connections 2024-09-13T17:58:03.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.155+0000 7f0845ded640 1 -- 172.21.15.111:0/3525489164 wait complete. 2024-09-13T17:58:03.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.157+0000 7f0845ded640 1 Processor -- start 2024-09-13T17:58:03.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.157+0000 7f0845ded640 1 -- start start 2024-09-13T17:58:03.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.157+0000 7f0845ded640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08400d07b0 0x7f084015cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:03.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.157+0000 7f0845ded640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0840154a00 0x7f084015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:03.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.157+0000 7f0845ded640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0840155840 0x7f0840155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:03.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.157+0000 7f0845ded640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084015fdb0 con 0x7f08400d07b0 2024-09-13T17:58:03.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.157+0000 7f0845ded640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0840156210 con 0x7f0840155840 2024-09-13T17:58:03.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.157+0000 7f0845ded640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08400b59c0 con 0x7f0840154a00 2024-09-13T17:58:03.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.158+0000 7f083f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08400d07b0 0x7f084015cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:03.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.158+0000 7f083f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08400d07b0 0x7f084015cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50830/0 (socket says 172.21.15.111:50830) 2024-09-13T17:58:03.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.158+0000 7f083f7fe640 1 -- 172.21.15.111:0/318058853 learned_addr learned my addr 172.21.15.111:0/318058853 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:03.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.158+0000 7f083ffff640 1 --2- 172.21.15.111:0/318058853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0840155840 0x7f0840155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:03.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.158+0000 7f0837fff640 1 --2- 172.21.15.111:0/318058853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0840154a00 0x7f084015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:03.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.158+0000 7f083f7fe640 1 -- 172.21.15.111:0/318058853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0840154a00 msgr2=0x7f084015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:03.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.158+0000 7f083f7fe640 1 --2- 172.21.15.111:0/318058853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0840154a00 0x7f084015efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:03.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.158+0000 7f083f7fe640 1 -- 172.21.15.111:0/318058853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0840155840 msgr2=0x7f0840155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:03.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.158+0000 7f083f7fe640 1 --2- 172.21.15.111:0/318058853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0840155840 0x7f0840155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:03.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.158+0000 7f083f7fe640 1 -- 172.21.15.111:0/318058853 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084015f7f0 con 0x7f08400d07b0 2024-09-13T17:58:03.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.158+0000 7f083f7fe640 1 --2- 172.21.15.111:0/318058853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08400d07b0 0x7f084015cc80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0830011ca0 tx=0x7f083000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:03.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.158+0000 7f083cff9640 1 -- 172.21.15.111:0/318058853 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830011ee0 con 0x7f08400d07b0 2024-09-13T17:58:03.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.158+0000 7f083cff9640 1 -- 172.21.15.111:0/318058853 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083000fe40 con 0x7f08400d07b0 2024-09-13T17:58:03.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.158+0000 7f0845ded640 1 -- 172.21.15.111:0/318058853 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f084015fad0 con 0x7f08400d07b0 2024-09-13T17:58:03.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.158+0000 7f083cff9640 1 -- 172.21.15.111:0/318058853 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830014680 con 0x7f08400d07b0 2024-09-13T17:58:03.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.158+0000 7f0845ded640 1 -- 172.21.15.111:0/318058853 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f084015f4e0 con 0x7f08400d07b0 2024-09-13T17:58:03.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.159+0000 7f0845ded640 1 -- 172.21.15.111:0/318058853 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08400b54b0 con 0x7f08400d07b0 2024-09-13T17:58:03.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.160+0000 7f083cff9640 1 -- 172.21.15.111:0/318058853 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f08300147e0 con 0x7f08400d07b0 2024-09-13T17:58:03.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.160+0000 7f083cff9640 1 --2- 172.21.15.111:0/318058853 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f082006ac30 0x7f082006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:03.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.160+0000 7f083cff9640 1 -- 172.21.15.111:0/318058853 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f083008a1d0 con 0x7f08400d07b0 2024-09-13T17:58:03.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.160+0000 7f0837fff640 1 --2- 172.21.15.111:0/318058853 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f082006ac30 0x7f082006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:03.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.161+0000 7f0837fff640 1 --2- 172.21.15.111:0/318058853 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f082006ac30 0x7f082006d0f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0828033000 tx=0x7f0828008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:03.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.165+0000 7f083cff9640 1 -- 172.21.15.111:0/318058853 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f08400b54b0 con 0x7f08400d07b0 2024-09-13T17:58:03.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.358+0000 7f0845ded640 1 -- 172.21.15.111:0/318058853 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f08400b54b0 con 0x7f08400d07b0 2024-09-13T17:58:03.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.359+0000 7f083cff9640 1 -- 172.21.15.111:0/318058853 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f08400b54b0 con 0x7f08400d07b0 2024-09-13T17:58:03.361 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T17:58:03.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.360+0000 7f0845ded640 1 -- 172.21.15.111:0/318058853 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f082006ac30 msgr2=0x7f082006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:03.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.361+0000 7f0845ded640 1 --2- 172.21.15.111:0/318058853 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f082006ac30 0x7f082006d0f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0828033000 tx=0x7f0828008030 comp rx=0 tx=0).stop 2024-09-13T17:58:03.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.361+0000 7f0845ded640 1 -- 172.21.15.111:0/318058853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08400d07b0 msgr2=0x7f084015cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:03.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.361+0000 7f0845ded640 1 --2- 172.21.15.111:0/318058853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08400d07b0 0x7f084015cc80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0830011ca0 tx=0x7f083000f610 comp rx=0 tx=0).stop 2024-09-13T17:58:03.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.361+0000 7f0845ded640 1 -- 172.21.15.111:0/318058853 shutdown_connections 2024-09-13T17:58:03.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.361+0000 7f0845ded640 1 --2- 172.21.15.111:0/318058853 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f082006ac30 0x7f082006d0f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:03.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.361+0000 7f0845ded640 1 --2- 172.21.15.111:0/318058853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0840155840 0x7f0840155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:03.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.361+0000 7f0845ded640 1 --2- 172.21.15.111:0/318058853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0840154a00 0x7f084015efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:03.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.361+0000 7f0845ded640 1 --2- 172.21.15.111:0/318058853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08400d07b0 0x7f084015cc80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:03.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.361+0000 7f0845ded640 1 -- 172.21.15.111:0/318058853 >> 172.21.15.111:0/318058853 conn(0x7f0840089f60 msgr2=0x7f08400b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:03.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.361+0000 7f0845ded640 1 -- 172.21.15.111:0/318058853 shutdown_connections 2024-09-13T17:58:03.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:03.361+0000 7f0845ded640 1 -- 172.21.15.111:0/318058853 wait complete. 2024-09-13T17:58:03.373 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:58:03.373 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:58:03.373 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-09-13T17:58:03.373 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-13T17:58:03.374 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-13T17:58:03.374 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:58:03.374 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:58:03.374 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:58:03.374 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-13T17:58:03.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:03.374 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-09-13T17:58:03.391 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:58:03.391 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:58:03.391 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:58:03.391 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:03.391 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-09-13T17:58:03.391 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:03.391 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:58:03.391 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:58:03.391 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-09-13T17:58:03.391 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:58:03.391 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:58:03.392 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-09-13T17:58:03.392 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:58:03.392 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:58:03.392 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:58:03.392 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:58:03.392 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-13T17:58:03.392 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42756sec preferred_lft 42756sec 2024-09-13T17:58:03.392 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link noprefixroute 2024-09-13T17:58:03.392 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:03.392 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:58:03.392 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ae:a5:13:5a:21:52 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:58:03.393 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T17:58:03.393 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:03.393 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::34a0:38ff:fe2c:f782/64 scope link 2024-09-13T17:58:03.393 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:03.393 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:58:03.393 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ae:a5:13:5a:21:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T17:58:03.393 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::aca5:13ff:fe5a:2152/64 scope link 2024-09-13T17:58:03.393 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:03.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:03.394 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:58:03.454 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-09-13T17:58:03.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:03.456 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-09-13T17:58:03.512 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-09-13T17:58:03.512 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:58:03.512 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:58:03.512 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:58:03.512 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:58:03.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:03.513 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-09-13T17:58:03.513 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-09-13T17:58:03.513 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:58:03.513 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:58:03.513 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:58:03.513 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-09-13T17:58:03.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:03.600 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-09-13T17:58:03.615 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-13T17:58:03.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:03.616 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2024-09-13T17:58:03.671 INFO:teuthology.orchestra.run.smithi188.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-13T17:58:03.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:03.671 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-09-13T17:58:03.671 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-09-13T17:58:03.671 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-13T17:58:03.671 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-13T17:58:03.671 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-09-13T17:58:03.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:03.750 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-13T17:58:03.813 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:58:03.814 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:58:03.814 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:58:03.814 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:03.814 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-09-13T17:58:03.814 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:03.814 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:58:03.814 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 8a:b4:39:49:e2:a2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T17:58:03.814 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T17:58:03.814 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:03.814 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::88b4:39ff:fe49:e2a2/64 scope link 2024-09-13T17:58:03.814 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:03.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:03.815 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-13T17:58:03.880 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:58:03.880 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:58:03.880 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:58:03.880 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:03.880 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-09-13T17:58:03.880 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:03.880 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:58:03.880 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 8a:b4:39:49:e2:a2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T17:58:03.880 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T17:58:03.881 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:03.881 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::88b4:39ff:fe49:e2a2/64 scope link 2024-09-13T17:58:03.881 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:03.882 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-13T17:58:03.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:03.882 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-09-13T17:58:03.882 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-09-13T17:58:03.882 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-13T17:58:03.882 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-13T17:58:03.882 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-13T17:58:03.882 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-13T17:58:03.882 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-13T17:58:03.883 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-09-13T17:58:03.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:03.995 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-09-13T17:58:03.995 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-09-13T17:58:03.995 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.1 up 2024-09-13T17:58:03.995 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.1 master ceph-brx 2024-09-13T17:58:03.995 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-09-13T17:58:04.033 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-09-13T17:58:04.034 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:58:04.034 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-13T17:58:04.089 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-13T17:58:04.089 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:58:04.089 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-13T17:58:04.144 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2024-09-13T17:58:04.209 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:58:04.210 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:58:04.275 INFO:teuthology.orchestra.run.smithi188.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:58:04.276 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:58:04.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:04.277 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2024-09-13T17:58:04.333 INFO:teuthology.orchestra.run.smithi188.stdout:42 2024-09-13T17:58:04.334 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-09-13T17:58:04.334 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-09-13T17:58:04.376 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2024-09-13T17:58:04.400 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:58:04.400 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:58:04.457 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi188.stderr:2024-09-13T17:58:04.455+0000 7f8c78d22480 -1 init, newargv = 0x56458cf9e100 newargc=15 2024-09-13T17:58:04.457 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi188.stderr:ceph-fuse[43321]: starting ceph client 2024-09-13T17:58:04.462 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi188.stderr:ceph-fuse[43321]: starting fuse 2024-09-13T17:58:04.467 INFO:teuthology.orchestra.run.smithi188.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:58:04.468 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:58:04.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:04.468 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2024-09-13T17:58:04.524 INFO:teuthology.orchestra.run.smithi188.stdout:42 2024-09-13T17:58:04.525 INFO:teuthology.orchestra.run.smithi188.stdout:44 2024-09-13T17:58:04.525 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 44] 2024-09-13T17:58:04.525 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:58:04.525 DEBUG:teuthology.orchestra.run.smithi188:> import glob 2024-09-13T17:58:04.525 DEBUG:teuthology.orchestra.run.smithi188:> import re 2024-09-13T17:58:04.525 DEBUG:teuthology.orchestra.run.smithi188:> import os 2024-09-13T17:58:04.525 DEBUG:teuthology.orchestra.run.smithi188:> import subprocess 2024-09-13T17:58:04.525 DEBUG:teuthology.orchestra.run.smithi188:> 2024-09-13T17:58:04.525 DEBUG:teuthology.orchestra.run.smithi188:> def _find_admin_socket(client_name): 2024-09-13T17:58:04.525 DEBUG:teuthology.orchestra.run.smithi188:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-09-13T17:58:04.526 DEBUG:teuthology.orchestra.run.smithi188:> files = glob.glob(asok_path) 2024-09-13T17:58:04.526 DEBUG:teuthology.orchestra.run.smithi188:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-09-13T17:58:04.526 DEBUG:teuthology.orchestra.run.smithi188:> 2024-09-13T17:58:04.526 DEBUG:teuthology.orchestra.run.smithi188:> # Given a non-glob path, it better be there 2024-09-13T17:58:04.526 DEBUG:teuthology.orchestra.run.smithi188:> if "*" not in asok_path: 2024-09-13T17:58:04.526 DEBUG:teuthology.orchestra.run.smithi188:> assert(len(files) == 1) 2024-09-13T17:58:04.526 DEBUG:teuthology.orchestra.run.smithi188:> return files[0] 2024-09-13T17:58:04.526 DEBUG:teuthology.orchestra.run.smithi188:> 2024-09-13T17:58:04.526 DEBUG:teuthology.orchestra.run.smithi188:> for f in files: 2024-09-13T17:58:04.526 DEBUG:teuthology.orchestra.run.smithi188:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:58:04.526 DEBUG:teuthology.orchestra.run.smithi188:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:58:04.526 DEBUG:teuthology.orchestra.run.smithi188:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:58:04.526 DEBUG:teuthology.orchestra.run.smithi188:> contents = proc_f.read() 2024-09-13T17:58:04.526 DEBUG:teuthology.orchestra.run.smithi188:> if mountpoint in contents: 2024-09-13T17:58:04.527 DEBUG:teuthology.orchestra.run.smithi188:> return f 2024-09-13T17:58:04.527 DEBUG:teuthology.orchestra.run.smithi188:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:58:04.527 DEBUG:teuthology.orchestra.run.smithi188:> 2024-09-13T17:58:04.527 DEBUG:teuthology.orchestra.run.smithi188:> print(_find_admin_socket("client.3")) 2024-09-13T17:58:04.527 DEBUG:teuthology.orchestra.run.smithi188:> ' 2024-09-13T17:58:04.637 INFO:teuthology.orchestra.run.smithi188.stdout:/var/run/ceph/ceph-client.3.43321.asok 2024-09-13T17:58:04.825 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.43321.asok 2024-09-13T17:58:04.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:04.826 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.43321.asok status 2024-09-13T17:58:04.960 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-09-13T17:58:04.960 INFO:teuthology.orchestra.run.smithi188.stdout: "metadata": { 2024-09-13T17:58:04.960 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:58:04.960 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:58:04.960 INFO:teuthology.orchestra.run.smithi188.stdout: "entity_id": "3", 2024-09-13T17:58:04.960 INFO:teuthology.orchestra.run.smithi188.stdout: "hostname": "smithi188", 2024-09-13T17:58:04.960 INFO:teuthology.orchestra.run.smithi188.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-09-13T17:58:04.961 INFO:teuthology.orchestra.run.smithi188.stdout: "pid": "43321", 2024-09-13T17:58:04.961 INFO:teuthology.orchestra.run.smithi188.stdout: "root": "/" 2024-09-13T17:58:04.961 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-09-13T17:58:04.961 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_count": 0, 2024-09-13T17:58:04.961 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_pinned_count": 0, 2024-09-13T17:58:04.961 INFO:teuthology.orchestra.run.smithi188.stdout: "id": 4616, 2024-09-13T17:58:04.961 INFO:teuthology.orchestra.run.smithi188.stdout: "inst": { 2024-09-13T17:58:04.961 INFO:teuthology.orchestra.run.smithi188.stdout: "name": { 2024-09-13T17:58:04.961 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "client", 2024-09-13T17:58:04.961 INFO:teuthology.orchestra.run.smithi188.stdout: "num": 4616 2024-09-13T17:58:04.961 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-09-13T17:58:04.961 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2024-09-13T17:58:04.961 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2024-09-13T17:58:04.962 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:58:04.962 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 444593605 2024-09-13T17:58:04.962 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-09-13T17:58:04.962 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-09-13T17:58:04.962 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2024-09-13T17:58:04.962 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2024-09-13T17:58:04.962 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:58:04.962 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 444593605 2024-09-13T17:58:04.962 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-09-13T17:58:04.962 INFO:teuthology.orchestra.run.smithi188.stdout: "inst_str": "client.4616 192.168.0.2:0/444593605", 2024-09-13T17:58:04.962 INFO:teuthology.orchestra.run.smithi188.stdout: "addr_str": "192.168.0.2:0/444593605", 2024-09-13T17:58:04.962 INFO:teuthology.orchestra.run.smithi188.stdout: "inode_count": 1, 2024-09-13T17:58:04.962 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_epoch": 9, 2024-09-13T17:58:04.962 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch": 24, 2024-09-13T17:58:04.963 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:58:04.963 INFO:teuthology.orchestra.run.smithi188.stdout: "blocklisted": false, 2024-09-13T17:58:04.963 INFO:teuthology.orchestra.run.smithi188.stdout: "fs_name": "cephfs" 2024-09-13T17:58:04.963 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-09-13T17:58:04.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:04.968 DEBUG:teuthology.orchestra.run.smithi111:> stat --file-system '--printf=%T 2024-09-13T17:58:04.968 DEBUG:teuthology.orchestra.run.smithi111:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T17:58:04.988 INFO:teuthology.orchestra.run.smithi111.stdout:fuseblk 2024-09-13T17:58:04.988 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T17:58:04.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:04.989 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:58:05.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:05.058 DEBUG:teuthology.orchestra.run.smithi111:> stat --file-system '--printf=%T 2024-09-13T17:58:05.058 DEBUG:teuthology.orchestra.run.smithi111:> ' -- /home/ubuntu/cephtest/mnt.2 2024-09-13T17:58:05.116 INFO:teuthology.orchestra.run.smithi111.stdout:fuseblk 2024-09-13T17:58:05.116 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-09-13T17:58:05.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:05.116 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-13T17:58:05.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:05.185 DEBUG:teuthology.orchestra.run.smithi188:> stat --file-system '--printf=%T 2024-09-13T17:58:05.185 DEBUG:teuthology.orchestra.run.smithi188:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-13T17:58:05.201 INFO:teuthology.orchestra.run.smithi188.stdout:fuseblk 2024-09-13T17:58:05.201 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-13T17:58:05.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:05.202 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-13T17:58:05.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:05.267 DEBUG:teuthology.orchestra.run.smithi188:> stat --file-system '--printf=%T 2024-09-13T17:58:05.267 DEBUG:teuthology.orchestra.run.smithi188:> ' -- /home/ubuntu/cephtest/mnt.3 2024-09-13T17:58:05.324 INFO:teuthology.orchestra.run.smithi188.stdout:fuseblk 2024-09-13T17:58:05.324 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-09-13T17:58:05.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:05.324 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-13T17:58:05.390 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-13T17:58:05.408 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:05.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.514+0000 7f22abb1f640 1 Processor -- start 2024-09-13T17:58:05.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.515+0000 7f22abb1f640 1 -- start start 2024-09-13T17:58:05.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.515+0000 7f22abb1f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f22a415c6e0 0x7f22a40b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:05.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.515+0000 7f22abb1f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f22a40ccb20 0x7f22a40b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:05.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.515+0000 7f22abb1f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a40d1840 con 0x7f22a40d07b0 2024-09-13T17:58:05.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.515+0000 7f22abb1f640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a406bd20 con 0x7f22a40ccb20 2024-09-13T17:58:05.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.515+0000 7f22abb1f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a40d0c30 con 0x7f22a415c6e0 2024-09-13T17:58:05.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.516+0000 7f22aa0bc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f22a40ccb20 0x7f22a40b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.516+0000 7f22aa0bc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f22a40ccb20 0x7f22a40b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50844/0 (socket says 172.21.15.111:50844) 2024-09-13T17:58:05.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.516+0000 7f22aa0bc640 1 -- 172.21.15.111:0/3874852268 learned_addr learned my addr 172.21.15.111:0/3874852268 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:05.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.516+0000 7f22a98bb640 1 --2- 172.21.15.111:0/3874852268 >> v2:172.21.15.188:3300/0 conn(0x7f22a415c6e0 0x7f22a40b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.516+0000 7f22aa0bc640 1 -- 172.21.15.111:0/3874852268 >> v2:172.21.15.188:3300/0 conn(0x7f22a415c6e0 msgr2=0x7f22a40b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:05.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.516+0000 7f22aa0bc640 1 --2- 172.21.15.111:0/3874852268 >> v2:172.21.15.188:3300/0 conn(0x7f22a415c6e0 0x7f22a40b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.516+0000 7f22aa0bc640 1 -- 172.21.15.111:0/3874852268 >> v1:172.21.15.188:6789/0 conn(0x7f22a40d07b0 legacy=0x7f22a4160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:05.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.516+0000 7f22aa0bc640 1 -- 172.21.15.111:0/3874852268 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a40d0f10 con 0x7f22a40ccb20 2024-09-13T17:58:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.517+0000 7f22aa0bc640 1 --2- 172.21.15.111:0/3874852268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a40ccb20 0x7f22a40b4490 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f22a40b4c10 tx=0x7f229c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ffeb97518b05b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.517+0000 7f22a88b9640 1 -- 172.21.15.111:0/3874852268 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c031600 con 0x7f22a40ccb20 2024-09-13T17:58:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.517+0000 7f22a88b9640 1 -- 172.21.15.111:0/3874852268 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c02d050 con 0x7f22a40ccb20 2024-09-13T17:58:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.517+0000 7f22a88b9640 1 -- 172.21.15.111:0/3874852268 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c031980 con 0x7f22a40ccb20 2024-09-13T17:58:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.517+0000 7f22abb1f640 1 -- 172.21.15.111:0/3874852268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a40ccb20 msgr2=0x7f22a40b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.517+0000 7f22abb1f640 1 --2- 172.21.15.111:0/3874852268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a40ccb20 0x7f22a40b4490 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f22a40b4c10 tx=0x7f229c02a660 comp rx=0 tx=0).stop 2024-09-13T17:58:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.517+0000 7f22abb1f640 1 -- 172.21.15.111:0/3874852268 shutdown_connections 2024-09-13T17:58:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.517+0000 7f22abb1f640 1 --2- 172.21.15.111:0/3874852268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a40ccb20 0x7f22a40b4490 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.517+0000 7f22abb1f640 1 --2- 172.21.15.111:0/3874852268 >> v2:172.21.15.188:3300/0 conn(0x7f22a415c6e0 0x7f22a40b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.517+0000 7f22abb1f640 1 -- 172.21.15.111:0/3874852268 >> 172.21.15.111:0/3874852268 conn(0x7f22a4089f60 msgr2=0x7f22a40d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.517+0000 7f22abb1f640 1 -- 172.21.15.111:0/3874852268 shutdown_connections 2024-09-13T17:58:05.521 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.518+0000 7f22abb1f640 1 -- 172.21.15.111:0/3874852268 wait complete. 2024-09-13T17:58:05.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.519+0000 7f22abb1f640 1 Processor -- start 2024-09-13T17:58:05.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.519+0000 7f22abb1f640 1 -- start start 2024-09-13T17:58:05.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.519+0000 7f22abb1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a40ccb20 0x7f22a40cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:05.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.519+0000 7f22abb1f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22a40d07b0 0x7f22a4156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:05.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.519+0000 7f22abb1f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22a415c6e0 0x7f22a4156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:05.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.519+0000 7f22abb1f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40cc580 con 0x7f22a40ccb20 2024-09-13T17:58:05.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.519+0000 7f22abb1f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40b79d0 con 0x7f22a40d07b0 2024-09-13T17:58:05.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.519+0000 7f22abb1f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40b7cb0 con 0x7f22a415c6e0 2024-09-13T17:58:05.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.519+0000 7f22a98bb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a40ccb20 0x7f22a40cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.519+0000 7f22a98bb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a40ccb20 0x7f22a40cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50848/0 (socket says 172.21.15.111:50848) 2024-09-13T17:58:05.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.519+0000 7f22a98bb640 1 -- 172.21.15.111:0/281444877 learned_addr learned my addr 172.21.15.111:0/281444877 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:05.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.520+0000 7f22a90ba640 1 --2- 172.21.15.111:0/281444877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22a40d07b0 0x7f22a4156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.520+0000 7f22aa0bc640 1 --2- 172.21.15.111:0/281444877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22a415c6e0 0x7f22a4156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.520+0000 7f22a98bb640 1 -- 172.21.15.111:0/281444877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22a415c6e0 msgr2=0x7f22a4156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:05.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.520+0000 7f22a98bb640 1 --2- 172.21.15.111:0/281444877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22a415c6e0 0x7f22a4156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.520+0000 7f22a98bb640 1 -- 172.21.15.111:0/281444877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22a40d07b0 msgr2=0x7f22a4156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:05.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.520+0000 7f22a98bb640 1 --2- 172.21.15.111:0/281444877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22a40d07b0 0x7f22a4156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.520+0000 7f22a98bb640 1 -- 172.21.15.111:0/281444877 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229c009b30 con 0x7f22a40ccb20 2024-09-13T17:58:05.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.520+0000 7f22a90ba640 1 --2- 172.21.15.111:0/281444877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22a40d07b0 0x7f22a4156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:05.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.520+0000 7f22a98bb640 1 --2- 172.21.15.111:0/281444877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a40ccb20 0x7f22a40cbfa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f22a000d800 tx=0x7f22a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:05.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.520+0000 7f229a7fc640 1 -- 172.21.15.111:0/281444877 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a000b6c0 con 0x7f22a40ccb20 2024-09-13T17:58:05.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.520+0000 7f229a7fc640 1 -- 172.21.15.111:0/281444877 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a000b820 con 0x7f22a40ccb20 2024-09-13T17:58:05.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.520+0000 7f22abb1f640 1 -- 172.21.15.111:0/281444877 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a40b7f90 con 0x7f22a40ccb20 2024-09-13T17:58:05.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.520+0000 7f229a7fc640 1 -- 172.21.15.111:0/281444877 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a000bbb0 con 0x7f22a40ccb20 2024-09-13T17:58:05.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.520+0000 7f22abb1f640 1 -- 172.21.15.111:0/281444877 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a41f20b0 con 0x7f22a40ccb20 2024-09-13T17:58:05.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.521+0000 7f22abb1f640 1 -- 172.21.15.111:0/281444877 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a40b5370 con 0x7f22a40ccb20 2024-09-13T17:58:05.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.522+0000 7f229a7fc640 1 -- 172.21.15.111:0/281444877 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f22a000bd10 con 0x7f22a40ccb20 2024-09-13T17:58:05.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.522+0000 7f229a7fc640 1 --2- 172.21.15.111:0/281444877 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f228806ac30 0x7f228806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:05.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.522+0000 7f229a7fc640 1 -- 172.21.15.111:0/281444877 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f22a0089a60 con 0x7f22a40ccb20 2024-09-13T17:58:05.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.522+0000 7f22a90ba640 1 --2- 172.21.15.111:0/281444877 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f228806ac30 0x7f228806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.523+0000 7f22a90ba640 1 --2- 172.21.15.111:0/281444877 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f228806ac30 0x7f228806d0f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2294006fd0 tx=0x7f2294008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:05.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.526+0000 7f229a7fc640 1 -- 172.21.15.111:0/281444877 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f22a40b5370 con 0x7f22a40ccb20 2024-09-13T17:58:05.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.691+0000 7f22abb1f640 1 -- 172.21.15.111:0/281444877 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22a40b5370 con 0x7f22a40ccb20 2024-09-13T17:58:05.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.692+0000 7f229a7fc640 1 -- 172.21.15.111:0/281444877 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f22a40b5370 con 0x7f22a40ccb20 2024-09-13T17:58:05.694 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:57:30.065235+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:05.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.694+0000 7f22abb1f640 1 -- 172.21.15.111:0/281444877 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f228806ac30 msgr2=0x7f228806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:05.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.694+0000 7f22abb1f640 1 --2- 172.21.15.111:0/281444877 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f228806ac30 0x7f228806d0f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2294006fd0 tx=0x7f2294008040 comp rx=0 tx=0).stop 2024-09-13T17:58:05.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.694+0000 7f22abb1f640 1 -- 172.21.15.111:0/281444877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a40ccb20 msgr2=0x7f22a40cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:05.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.694+0000 7f22abb1f640 1 --2- 172.21.15.111:0/281444877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a40ccb20 0x7f22a40cbfa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f22a000d800 tx=0x7f22a000dcc0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.694+0000 7f22abb1f640 1 -- 172.21.15.111:0/281444877 shutdown_connections 2024-09-13T17:58:05.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.694+0000 7f22abb1f640 1 --2- 172.21.15.111:0/281444877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22a415c6e0 0x7f22a4156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.694+0000 7f22abb1f640 1 --2- 172.21.15.111:0/281444877 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f228806ac30 0x7f228806d0f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.694+0000 7f22abb1f640 1 --2- 172.21.15.111:0/281444877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22a40d07b0 0x7f22a4156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.694+0000 7f22abb1f640 1 --2- 172.21.15.111:0/281444877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a40ccb20 0x7f22a40cbfa0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.694+0000 7f22abb1f640 1 -- 172.21.15.111:0/281444877 >> 172.21.15.111:0/281444877 conn(0x7f22a4089f60 msgr2=0x7f22a415b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:05.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.694+0000 7f22abb1f640 1 -- 172.21.15.111:0/281444877 shutdown_connections 2024-09-13T17:58:05.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.694+0000 7f22abb1f640 1 -- 172.21.15.111:0/281444877 wait complete. 2024-09-13T17:58:05.707 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:58:05.708 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:58:05.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.808+0000 7f7188c44640 1 Processor -- start 2024-09-13T17:58:05.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.808+0000 7f7188c44640 1 -- start start 2024-09-13T17:58:05.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.809+0000 7f7188c44640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7184159310 0x7f71840ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:05.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.809+0000 7f7188c44640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f71840ce4f0 0x7f71840ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:05.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.809+0000 7f7188c44640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71840ccbe0 con 0x7f71840b2b40 2024-09-13T17:58:05.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.809+0000 7f7188c44640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71840ad940 con 0x7f7184159310 2024-09-13T17:58:05.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.809+0000 7f7188c44640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71840b1810 con 0x7f71840ce4f0 2024-09-13T17:58:05.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.809+0000 7f718259c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7184159310 0x7f71840ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.809+0000 7f718259c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7184159310 0x7f71840ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50862/0 (socket says 172.21.15.111:50862) 2024-09-13T17:58:05.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.809+0000 7f7181d9b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f71840b2b40 0x7f71840ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58770/0 (socket says 172.21.15.111:58770) 2024-09-13T17:58:05.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.809+0000 7f718259c640 1 -- 172.21.15.111:0/2452569634 learned_addr learned my addr 172.21.15.111:0/2452569634 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:05.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.809+0000 7f7182d9d640 1 --2- 172.21.15.111:0/2452569634 >> v2:172.21.15.188:3300/0 conn(0x7f71840ce4f0 0x7f71840ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.809+0000 7f718259c640 1 -- 172.21.15.111:0/2452569634 >> v2:172.21.15.188:3300/0 conn(0x7f71840ce4f0 msgr2=0x7f71840ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:05.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.809+0000 7f718259c640 1 --2- 172.21.15.111:0/2452569634 >> v2:172.21.15.188:3300/0 conn(0x7f71840ce4f0 0x7f71840ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.809+0000 7f718259c640 1 -- 172.21.15.111:0/2452569634 >> v1:172.21.15.111:6789/0 conn(0x7f71840b2b40 legacy=0x7f71840ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:05.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.809+0000 7f718159a640 1 -- 172.21.15.111:0/2452569634 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386317456 0 0) 0x7f71840ccbe0 con 0x7f71840b2b40 2024-09-13T17:58:05.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.809+0000 7f718259c640 1 -- 172.21.15.111:0/2452569634 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71840cee10 con 0x7f7184159310 2024-09-13T17:58:05.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.810+0000 7f718259c640 1 --2- 172.21.15.111:0/2452569634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7184159310 0x7f71840ad400 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f71840b2fc0 tx=0x7f717c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99eaa66591280b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:05.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.810+0000 7f718159a640 1 -- 172.21.15.111:0/2452569634 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c032440 con 0x7f7184159310 2024-09-13T17:58:05.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.810+0000 7f718159a640 1 -- 172.21.15.111:0/2452569634 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c0325a0 con 0x7f7184159310 2024-09-13T17:58:05.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.810+0000 7f718159a640 1 -- 172.21.15.111:0/2452569634 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c032930 con 0x7f7184159310 2024-09-13T17:58:05.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.811+0000 7f7188c44640 1 -- 172.21.15.111:0/2452569634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7184159310 msgr2=0x7f71840ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:05.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.811+0000 7f7188c44640 1 --2- 172.21.15.111:0/2452569634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7184159310 0x7f71840ad400 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f71840b2fc0 tx=0x7f717c02a1e0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.811+0000 7f7188c44640 1 -- 172.21.15.111:0/2452569634 shutdown_connections 2024-09-13T17:58:05.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.811+0000 7f7188c44640 1 --2- 172.21.15.111:0/2452569634 >> v2:172.21.15.188:3300/0 conn(0x7f71840ce4f0 0x7f71840ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.811+0000 7f7188c44640 1 --2- 172.21.15.111:0/2452569634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7184159310 0x7f71840ad400 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.811+0000 7f7188c44640 1 -- 172.21.15.111:0/2452569634 >> 172.21.15.111:0/2452569634 conn(0x7f718406a0b0 msgr2=0x7f718406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:05.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.811+0000 7f7188c44640 1 -- 172.21.15.111:0/2452569634 shutdown_connections 2024-09-13T17:58:05.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.811+0000 7f7188c44640 1 -- 172.21.15.111:0/2452569634 wait complete. 2024-09-13T17:58:05.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.812+0000 7f7188c44640 1 Processor -- start 2024-09-13T17:58:05.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.812+0000 7f7188c44640 1 -- start start 2024-09-13T17:58:05.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.813+0000 7f7188c44640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71840b2b40 0x7f7184097820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:05.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.813+0000 7f7188c44640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71840ce4f0 0x7f71840c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:05.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.813+0000 7f7188c44640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7184159310 0x7f71840cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:05.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.813+0000 7f7188c44640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71840c9070 con 0x7f7184159310 2024-09-13T17:58:05.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.813+0000 7f7188c44640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71840c7590 con 0x7f71840ce4f0 2024-09-13T17:58:05.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.813+0000 7f7188c44640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71840c7870 con 0x7f71840b2b40 2024-09-13T17:58:05.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.813+0000 7f7182d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7184159310 0x7f71840cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.813+0000 7f7182d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7184159310 0x7f71840cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50864/0 (socket says 172.21.15.111:50864) 2024-09-13T17:58:05.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.813+0000 7f7182d9d640 1 -- 172.21.15.111:0/2132696245 learned_addr learned my addr 172.21.15.111:0/2132696245 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:05.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.813+0000 7f718259c640 1 --2- 172.21.15.111:0/2132696245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71840b2b40 0x7f7184097820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.813+0000 7f7181d9b640 1 --2- 172.21.15.111:0/2132696245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71840ce4f0 0x7f71840c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.813+0000 7f7182d9d640 1 -- 172.21.15.111:0/2132696245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71840b2b40 msgr2=0x7f7184097820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:05.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.813+0000 7f7182d9d640 1 --2- 172.21.15.111:0/2132696245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71840b2b40 0x7f7184097820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.813+0000 7f7182d9d640 1 -- 172.21.15.111:0/2132696245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71840ce4f0 msgr2=0x7f71840c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.813+0000 7f7182d9d640 1 --2- 172.21.15.111:0/2132696245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71840ce4f0 0x7f71840c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.813+0000 7f7182d9d640 1 -- 172.21.15.111:0/2132696245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c0094d0 con 0x7f7184159310 2024-09-13T17:58:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.814+0000 7f7182d9d640 1 --2- 172.21.15.111:0/2132696245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7184159310 0x7f71840cad30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7174007e40 tx=0x7f717400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.814+0000 7f717affd640 1 -- 172.21.15.111:0/2132696245 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7174014620 con 0x7f7184159310 2024-09-13T17:58:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.814+0000 7f717affd640 1 -- 172.21.15.111:0/2132696245 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717400ee90 con 0x7f7184159310 2024-09-13T17:58:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.814+0000 7f7188c44640 1 -- 172.21.15.111:0/2132696245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71840c92a0 con 0x7f7184159310 2024-09-13T17:58:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.814+0000 7f717affd640 1 -- 172.21.15.111:0/2132696245 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71740149e0 con 0x7f7184159310 2024-09-13T17:58:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.814+0000 7f7188c44640 1 -- 172.21.15.111:0/2132696245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71840c7b50 con 0x7f7184159310 2024-09-13T17:58:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.815+0000 7f7188c44640 1 -- 172.21.15.111:0/2132696245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71840c97c0 con 0x7f7184159310 2024-09-13T17:58:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.815+0000 7f717affd640 1 -- 172.21.15.111:0/2132696245 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7174014b40 con 0x7f7184159310 2024-09-13T17:58:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.816+0000 7f717affd640 1 --2- 172.21.15.111:0/2132696245 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f716406ac00 0x7f716406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.816+0000 7f717affd640 1 -- 172.21.15.111:0/2132696245 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7174089e80 con 0x7f7184159310 2024-09-13T17:58:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.816+0000 7f718259c640 1 --2- 172.21.15.111:0/2132696245 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f716406ac00 0x7f716406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.816+0000 7f718259c640 1 --2- 172.21.15.111:0/2132696245 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f716406ac00 0x7f716406d0c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f717c034000 tx=0x7f717c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:05.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.820+0000 7f717affd640 1 -- 172.21.15.111:0/2132696245 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f71840c97c0 con 0x7f7184159310 2024-09-13T17:58:05.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.979+0000 7f7188c44640 1 -- 172.21.15.111:0/2132696245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71840c97c0 con 0x7f7184159310 2024-09-13T17:58:05.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.980+0000 7f717affd640 1 -- 172.21.15.111:0/2132696245 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71840c97c0 con 0x7f7184159310 2024-09-13T17:58:05.982 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:58:05.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.982+0000 7f7188c44640 1 -- 172.21.15.111:0/2132696245 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f716406ac00 msgr2=0x7f716406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:05.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.982+0000 7f7188c44640 1 --2- 172.21.15.111:0/2132696245 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f716406ac00 0x7f716406d0c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f717c034000 tx=0x7f717c02e000 comp rx=0 tx=0).stop 2024-09-13T17:58:05.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.982+0000 7f7188c44640 1 -- 172.21.15.111:0/2132696245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7184159310 msgr2=0x7f71840cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:05.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.982+0000 7f7188c44640 1 --2- 172.21.15.111:0/2132696245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7184159310 0x7f71840cad30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7174007e40 tx=0x7f717400e4b0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.982+0000 7f7188c44640 1 -- 172.21.15.111:0/2132696245 shutdown_connections 2024-09-13T17:58:05.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.982+0000 7f7188c44640 1 --2- 172.21.15.111:0/2132696245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7184159310 0x7f71840cad30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.982+0000 7f7188c44640 1 --2- 172.21.15.111:0/2132696245 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f716406ac00 0x7f716406d0c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.982+0000 7f7188c44640 1 --2- 172.21.15.111:0/2132696245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71840ce4f0 0x7f71840c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.982+0000 7f7188c44640 1 --2- 172.21.15.111:0/2132696245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71840b2b40 0x7f7184097820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:05.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.982+0000 7f7188c44640 1 -- 172.21.15.111:0/2132696245 >> 172.21.15.111:0/2132696245 conn(0x7f718406a0b0 msgr2=0x7f71840ae9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:05.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.982+0000 7f7188c44640 1 -- 172.21.15.111:0/2132696245 shutdown_connections 2024-09-13T17:58:05.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:05.982+0000 7f7188c44640 1 -- 172.21.15.111:0/2132696245 wait complete. 2024-09-13T17:58:05.995 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:58:06.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.093+0000 7f8104da0640 1 Processor -- start 2024-09-13T17:58:06.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.093+0000 7f8104da0640 1 -- start start 2024-09-13T17:58:06.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.094+0000 7f8104da0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8100071730 0x7f810015fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.094+0000 7f8104da0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f81000b20b0 0x7f81000b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.094+0000 7f8104da0640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f810015a1e0 con 0x7f8100159e00 2024-09-13T17:58:06.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.094+0000 7f8104da0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8100160d90 con 0x7f8100071730 2024-09-13T17:58:06.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.094+0000 7f8104da0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f810015c7b0 con 0x7f81000b20b0 2024-09-13T17:58:06.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.094+0000 7f80feffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8100071730 0x7f810015fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.094+0000 7f80ff7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f8100159e00 0x7f810015cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58784/0 (socket says 172.21.15.111:58784) 2024-09-13T17:58:06.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.094+0000 7f80ff7fe640 1 -- 172.21.15.111:0/744306653 learned_addr learned my addr 172.21.15.111:0/744306653 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:06.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.094+0000 7f80fffff640 1 --2- 172.21.15.111:0/744306653 >> v2:172.21.15.188:3300/0 conn(0x7f81000b20b0 0x7f81000b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.094+0000 7f80feffd640 1 -- 172.21.15.111:0/744306653 >> v2:172.21.15.188:3300/0 conn(0x7f81000b20b0 msgr2=0x7f81000b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:06.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.094+0000 7f80feffd640 1 --2- 172.21.15.111:0/744306653 >> v2:172.21.15.188:3300/0 conn(0x7f81000b20b0 0x7f81000b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.094+0000 7f80feffd640 1 -- 172.21.15.111:0/744306653 >> v1:172.21.15.111:6789/0 conn(0x7f8100159e00 legacy=0x7f810015cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.094+0000 7f80feffd640 1 -- 172.21.15.111:0/744306653 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81001604f0 con 0x7f8100071730 2024-09-13T17:58:06.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.094+0000 7f80fe7fc640 1 -- 172.21.15.111:0/744306653 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361640387 0 0) 0x7f810015a1e0 con 0x7f8100159e00 2024-09-13T17:58:06.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.095+0000 7f80feffd640 1 --2- 172.21.15.111:0/744306653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8100071730 0x7f810015fda0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f81000b4130 tx=0x7f80f402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a06e00b2aa31296 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:06.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.095+0000 7f80fe7fc640 1 -- 172.21.15.111:0/744306653 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f402ae60 con 0x7f8100071730 2024-09-13T17:58:06.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.095+0000 7f80fe7fc640 1 -- 172.21.15.111:0/744306653 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f4032440 con 0x7f8100071730 2024-09-13T17:58:06.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.095+0000 7f80fe7fc640 1 -- 172.21.15.111:0/744306653 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f40327d0 con 0x7f8100071730 2024-09-13T17:58:06.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.096+0000 7f8104da0640 1 -- 172.21.15.111:0/744306653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8100071730 msgr2=0x7f810015fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.096+0000 7f8104da0640 1 --2- 172.21.15.111:0/744306653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8100071730 0x7f810015fda0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f81000b4130 tx=0x7f80f402a110 comp rx=0 tx=0).stop 2024-09-13T17:58:06.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.096+0000 7f8104da0640 1 -- 172.21.15.111:0/744306653 shutdown_connections 2024-09-13T17:58:06.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.096+0000 7f8104da0640 1 --2- 172.21.15.111:0/744306653 >> v2:172.21.15.188:3300/0 conn(0x7f81000b20b0 0x7f81000b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.096+0000 7f8104da0640 1 --2- 172.21.15.111:0/744306653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8100071730 0x7f810015fda0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.096+0000 7f8104da0640 1 -- 172.21.15.111:0/744306653 >> 172.21.15.111:0/744306653 conn(0x7f81000cf000 msgr2=0x7f81000cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:06.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.096+0000 7f8104da0640 1 -- 172.21.15.111:0/744306653 shutdown_connections 2024-09-13T17:58:06.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.096+0000 7f8104da0640 1 -- 172.21.15.111:0/744306653 wait complete. 2024-09-13T17:58:06.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.097+0000 7f8104da0640 1 Processor -- start 2024-09-13T17:58:06.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.097+0000 7f8104da0640 1 -- start start 2024-09-13T17:58:06.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f8104da0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8100071730 0x7f810014d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f8104da0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81000b20b0 0x7f81001549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f8104da0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8100159e00 0x7f810015d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f8104da0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810014d870 con 0x7f8100159e00 2024-09-13T17:58:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f8104da0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8100156370 con 0x7f81000b20b0 2024-09-13T17:58:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f8104da0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81001552d0 con 0x7f8100071730 2024-09-13T17:58:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f80fffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8100159e00 0x7f810015d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f80fffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8100159e00 0x7f810015d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50884/0 (socket says 172.21.15.111:50884) 2024-09-13T17:58:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f80fffff640 1 -- 172.21.15.111:0/1136239201 learned_addr learned my addr 172.21.15.111:0/1136239201 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f80feffd640 1 --2- 172.21.15.111:0/1136239201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81000b20b0 0x7f81001549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f80ff7fe640 1 --2- 172.21.15.111:0/1136239201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8100071730 0x7f810014d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f80fffff640 1 -- 172.21.15.111:0/1136239201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8100071730 msgr2=0x7f810014d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f80fffff640 1 --2- 172.21.15.111:0/1136239201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8100071730 0x7f810014d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f80fffff640 1 -- 172.21.15.111:0/1136239201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81000b20b0 msgr2=0x7f81001549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f80fffff640 1 --2- 172.21.15.111:0/1136239201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81000b20b0 0x7f81001549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f80fffff640 1 -- 172.21.15.111:0/1136239201 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f4009400 con 0x7f8100159e00 2024-09-13T17:58:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f80feffd640 1 --2- 172.21.15.111:0/1136239201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81000b20b0 0x7f81001549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f80fffff640 1 --2- 172.21.15.111:0/1136239201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8100159e00 0x7f810015d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f80f800e820 tx=0x7f80f800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f80e7fff640 1 -- 172.21.15.111:0/1136239201 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f8009510 con 0x7f8100159e00 2024-09-13T17:58:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f80e7fff640 1 -- 172.21.15.111:0/1136239201 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f800ee70 con 0x7f8100159e00 2024-09-13T17:58:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f80e7fff640 1 -- 172.21.15.111:0/1136239201 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f800cb20 con 0x7f8100159e00 2024-09-13T17:58:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.098+0000 7f8104da0640 1 -- 172.21.15.111:0/1136239201 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81001555b0 con 0x7f8100159e00 2024-09-13T17:58:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.099+0000 7f8104da0640 1 -- 172.21.15.111:0/1136239201 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8100155b70 con 0x7f8100159e00 2024-09-13T17:58:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.100+0000 7f8104da0640 1 -- 172.21.15.111:0/1136239201 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81000b2560 con 0x7f8100159e00 2024-09-13T17:58:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.100+0000 7f80e7fff640 1 -- 172.21.15.111:0/1136239201 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f80f800cc80 con 0x7f8100159e00 2024-09-13T17:58:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.100+0000 7f80e7fff640 1 --2- 172.21.15.111:0/1136239201 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f80ec06ac00 0x7f80ec06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.101+0000 7f80e7fff640 1 -- 172.21.15.111:0/1136239201 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f80f8089750 con 0x7f8100159e00 2024-09-13T17:58:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.101+0000 7f80ff7fe640 1 --2- 172.21.15.111:0/1136239201 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f80ec06ac00 0x7f80ec06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.101+0000 7f80ff7fe640 1 --2- 172.21.15.111:0/1136239201 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f80ec06ac00 0x7f80ec06d0c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f80f00012e0 tx=0x7f80f0007ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:06.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.105+0000 7f80e7fff640 1 -- 172.21.15.111:0/1136239201 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f80f80545b0 con 0x7f8100159e00 2024-09-13T17:58:06.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.264+0000 7f8104da0640 1 -- 172.21.15.111:0/1136239201 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f81000aed00 con 0x7f8100159e00 2024-09-13T17:58:06.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.264+0000 7f80e7fff640 1 -- 172.21.15.111:0/1136239201 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f80f8057bd0 con 0x7f8100159e00 2024-09-13T17:58:06.266 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:06.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.266+0000 7f8104da0640 1 -- 172.21.15.111:0/1136239201 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f80ec06ac00 msgr2=0x7f80ec06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.266+0000 7f8104da0640 1 --2- 172.21.15.111:0/1136239201 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f80ec06ac00 0x7f80ec06d0c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f80f00012e0 tx=0x7f80f0007ed0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.266+0000 7f8104da0640 1 -- 172.21.15.111:0/1136239201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8100159e00 msgr2=0x7f810015d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.266+0000 7f8104da0640 1 --2- 172.21.15.111:0/1136239201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8100159e00 0x7f810015d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f80f800e820 tx=0x7f80f800ec20 comp rx=0 tx=0).stop 2024-09-13T17:58:06.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.266+0000 7f8104da0640 1 -- 172.21.15.111:0/1136239201 shutdown_connections 2024-09-13T17:58:06.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.266+0000 7f8104da0640 1 --2- 172.21.15.111:0/1136239201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8100159e00 0x7f810015d990 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.266+0000 7f8104da0640 1 --2- 172.21.15.111:0/1136239201 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f80ec06ac00 0x7f80ec06d0c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.266+0000 7f8104da0640 1 --2- 172.21.15.111:0/1136239201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81000b20b0 0x7f81001549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.266+0000 7f8104da0640 1 --2- 172.21.15.111:0/1136239201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8100071730 0x7f810014d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.266+0000 7f8104da0640 1 -- 172.21.15.111:0/1136239201 >> 172.21.15.111:0/1136239201 conn(0x7f81000cf000 msgr2=0x7f81000b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:06.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.266+0000 7f8104da0640 1 -- 172.21.15.111:0/1136239201 shutdown_connections 2024-09-13T17:58:06.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.266+0000 7f8104da0640 1 -- 172.21.15.111:0/1136239201 wait complete. 2024-09-13T17:58:06.280 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:58:06.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.378+0000 7fee50a61640 1 Processor -- start 2024-09-13T17:58:06.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.378+0000 7fee50a61640 1 -- start start 2024-09-13T17:58:06.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.379+0000 7fee50a61640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee4c0cdf10 0x7fee4c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.379+0000 7fee50a61640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fee4c06b600 0x7fee4c065c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.379+0000 7fee50a61640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fee4c0ce2f0 0x7fee4c0cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.379+0000 7fee50a61640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee4c15c9a0 con 0x7fee4c0ce2f0 2024-09-13T17:58:06.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.379+0000 7fee50a61640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee4c06b170 con 0x7fee4c06b600 2024-09-13T17:58:06.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.379+0000 7fee50a61640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0661a0 con 0x7fee4c0cdf10 2024-09-13T17:58:06.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.379+0000 7fee4ad9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fee4c0ce2f0 0x7fee4c0cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.379+0000 7fee4ad9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fee4c0ce2f0 0x7fee4c0cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50894/0 (socket says 172.21.15.111:50894) 2024-09-13T17:58:06.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.379+0000 7fee4ad9d640 1 -- 172.21.15.111:0/3608189039 learned_addr learned my addr 172.21.15.111:0/3608189039 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:06.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.379+0000 7fee4a59c640 1 --2- 172.21.15.111:0/3608189039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee4c0cdf10 0x7fee4c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.379+0000 7fee41d9b640 1 --2- 172.21.15.111:0/3608189039 >> v2:172.21.15.188:3300/0 conn(0x7fee4c06b600 0x7fee4c065c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.379+0000 7fee4ad9d640 1 -- 172.21.15.111:0/3608189039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee4c0cdf10 msgr2=0x7fee4c069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:06.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.379+0000 7fee4ad9d640 1 --2- 172.21.15.111:0/3608189039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee4c0cdf10 0x7fee4c069ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.379+0000 7fee4ad9d640 1 -- 172.21.15.111:0/3608189039 >> v2:172.21.15.188:3300/0 conn(0x7fee4c06b600 msgr2=0x7fee4c065c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:06.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.379+0000 7fee4ad9d640 1 --2- 172.21.15.111:0/3608189039 >> v2:172.21.15.188:3300/0 conn(0x7fee4c06b600 0x7fee4c065c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.379+0000 7fee4ad9d640 1 -- 172.21.15.111:0/3608189039 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c06a560 con 0x7fee4c0ce2f0 2024-09-13T17:58:06.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.380+0000 7fee4ad9d640 1 --2- 172.21.15.111:0/3608189039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee4c0ce2f0 0x7fee4c0cc700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fee4c067980 tx=0x7fee4402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffcaa914a5494ae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:06.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.380+0000 7fee49d9b640 1 -- 172.21.15.111:0/3608189039 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee44031600 con 0x7fee4c0ce2f0 2024-09-13T17:58:06.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.380+0000 7fee49d9b640 1 -- 172.21.15.111:0/3608189039 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4402d050 con 0x7fee4c0ce2f0 2024-09-13T17:58:06.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.380+0000 7fee49d9b640 1 -- 172.21.15.111:0/3608189039 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee44031980 con 0x7fee4c0ce2f0 2024-09-13T17:58:06.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.381+0000 7fee50a61640 1 -- 172.21.15.111:0/3608189039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee4c0ce2f0 msgr2=0x7fee4c0cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.381+0000 7fee50a61640 1 --2- 172.21.15.111:0/3608189039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee4c0ce2f0 0x7fee4c0cc700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fee4c067980 tx=0x7fee4402a6a0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.381+0000 7fee50a61640 1 -- 172.21.15.111:0/3608189039 shutdown_connections 2024-09-13T17:58:06.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.381+0000 7fee50a61640 1 --2- 172.21.15.111:0/3608189039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee4c0ce2f0 0x7fee4c0cc700 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.381+0000 7fee50a61640 1 --2- 172.21.15.111:0/3608189039 >> v2:172.21.15.188:3300/0 conn(0x7fee4c06b600 0x7fee4c065c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.381+0000 7fee50a61640 1 --2- 172.21.15.111:0/3608189039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee4c0cdf10 0x7fee4c069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.381+0000 7fee50a61640 1 -- 172.21.15.111:0/3608189039 >> 172.21.15.111:0/3608189039 conn(0x7fee4c0cb5b0 msgr2=0x7fee4c0cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:06.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.381+0000 7fee50a61640 1 -- 172.21.15.111:0/3608189039 shutdown_connections 2024-09-13T17:58:06.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.381+0000 7fee50a61640 1 -- 172.21.15.111:0/3608189039 wait complete. 2024-09-13T17:58:06.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.382+0000 7fee50a61640 1 Processor -- start 2024-09-13T17:58:06.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.382+0000 7fee50a61640 1 -- start start 2024-09-13T17:58:06.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.383+0000 7fee50a61640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee4c06b600 0x7fee4c15f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.383+0000 7fee50a61640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee4c0cdf10 0x7fee4c0c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.383+0000 7fee50a61640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee4c0ce2f0 0x7fee4c0c2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.383+0000 7fee50a61640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0c6bd0 con 0x7fee4c06b600 2024-09-13T17:58:06.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.383+0000 7fee50a61640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0c65e0 con 0x7fee4c0ce2f0 2024-09-13T17:58:06.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.383+0000 7fee50a61640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0afbd0 con 0x7fee4c0cdf10 2024-09-13T17:58:06.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.383+0000 7fee4a59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee4c06b600 0x7fee4c15f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.383+0000 7fee4a59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee4c06b600 0x7fee4c15f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50908/0 (socket says 172.21.15.111:50908) 2024-09-13T17:58:06.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.383+0000 7fee4a59c640 1 -- 172.21.15.111:0/3239393389 learned_addr learned my addr 172.21.15.111:0/3239393389 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:06.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.383+0000 7fee4ad9d640 1 --2- 172.21.15.111:0/3239393389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee4c0ce2f0 0x7fee4c0c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.383+0000 7fee41d9b640 1 --2- 172.21.15.111:0/3239393389 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee4c0cdf10 0x7fee4c0c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.383+0000 7fee4a59c640 1 -- 172.21.15.111:0/3239393389 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee4c0cdf10 msgr2=0x7fee4c0c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.383+0000 7fee4a59c640 1 --2- 172.21.15.111:0/3239393389 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee4c0cdf10 0x7fee4c0c7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.383+0000 7fee4a59c640 1 -- 172.21.15.111:0/3239393389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee4c0ce2f0 msgr2=0x7fee4c0c2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.383+0000 7fee4a59c640 1 --2- 172.21.15.111:0/3239393389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee4c0ce2f0 0x7fee4c0c2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.383+0000 7fee4a59c640 1 -- 172.21.15.111:0/3239393389 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee44009b30 con 0x7fee4c06b600 2024-09-13T17:58:06.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.384+0000 7fee4a59c640 1 --2- 172.21.15.111:0/3239393389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee4c06b600 0x7fee4c15f570 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00c820 tx=0x7fee3c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:06.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.384+0000 7fee437fe640 1 -- 172.21.15.111:0/3239393389 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c004020 con 0x7fee4c06b600 2024-09-13T17:58:06.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.384+0000 7fee437fe640 1 -- 172.21.15.111:0/3239393389 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3c004180 con 0x7fee4c06b600 2024-09-13T17:58:06.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.384+0000 7fee50a61640 1 -- 172.21.15.111:0/3239393389 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4c0b2e90 con 0x7fee4c06b600 2024-09-13T17:58:06.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.384+0000 7fee437fe640 1 -- 172.21.15.111:0/3239393389 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c007830 con 0x7fee4c06b600 2024-09-13T17:58:06.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.384+0000 7fee50a61640 1 -- 172.21.15.111:0/3239393389 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4c0c6860 con 0x7fee4c06b600 2024-09-13T17:58:06.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.385+0000 7fee50a61640 1 -- 172.21.15.111:0/3239393389 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4c0b33b0 con 0x7fee4c06b600 2024-09-13T17:58:06.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.385+0000 7fee437fe640 1 -- 172.21.15.111:0/3239393389 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fee3c007990 con 0x7fee4c06b600 2024-09-13T17:58:06.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.386+0000 7fee437fe640 1 --2- 172.21.15.111:0/3239393389 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fee2c06ac00 0x7fee2c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.386+0000 7fee41d9b640 1 --2- 172.21.15.111:0/3239393389 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fee2c06ac00 0x7fee2c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.386+0000 7fee437fe640 1 -- 172.21.15.111:0/3239393389 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fee3c089320 con 0x7fee4c06b600 2024-09-13T17:58:06.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.386+0000 7fee41d9b640 1 --2- 172.21.15.111:0/3239393389 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fee2c06ac00 0x7fee2c06d0c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fee4c15b300 tx=0x7fee34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:06.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.390+0000 7fee437fe640 1 -- 172.21.15.111:0/3239393389 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fee3c054210 con 0x7fee4c06b600 2024-09-13T17:58:06.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.550+0000 7fee50a61640 1 -- 172.21.15.111:0/3239393389 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fee4c06aae0 con 0x7fee4c06b600 2024-09-13T17:58:06.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.550+0000 7fee437fe640 1 -- 172.21.15.111:0/3239393389 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fee3c057830 con 0x7fee4c06b600 2024-09-13T17:58:06.552 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:58:06.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.552+0000 7fee50a61640 1 -- 172.21.15.111:0/3239393389 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fee2c06ac00 msgr2=0x7fee2c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.552+0000 7fee50a61640 1 --2- 172.21.15.111:0/3239393389 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fee2c06ac00 0x7fee2c06d0c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fee4c15b300 tx=0x7fee34008040 comp rx=0 tx=0).stop 2024-09-13T17:58:06.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.552+0000 7fee50a61640 1 -- 172.21.15.111:0/3239393389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee4c06b600 msgr2=0x7fee4c15f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.552+0000 7fee50a61640 1 --2- 172.21.15.111:0/3239393389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee4c06b600 0x7fee4c15f570 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00c820 tx=0x7fee3c00cce0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.552+0000 7fee50a61640 1 -- 172.21.15.111:0/3239393389 shutdown_connections 2024-09-13T17:58:06.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.552+0000 7fee50a61640 1 --2- 172.21.15.111:0/3239393389 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fee2c06ac00 0x7fee2c06d0c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.552+0000 7fee50a61640 1 --2- 172.21.15.111:0/3239393389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee4c0ce2f0 0x7fee4c0c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.552+0000 7fee50a61640 1 --2- 172.21.15.111:0/3239393389 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee4c0cdf10 0x7fee4c0c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.552+0000 7fee50a61640 1 --2- 172.21.15.111:0/3239393389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee4c06b600 0x7fee4c15f570 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.552+0000 7fee50a61640 1 -- 172.21.15.111:0/3239393389 >> 172.21.15.111:0/3239393389 conn(0x7fee4c0cb5b0 msgr2=0x7fee4c09e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:06.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.552+0000 7fee50a61640 1 -- 172.21.15.111:0/3239393389 shutdown_connections 2024-09-13T17:58:06.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.552+0000 7fee50a61640 1 -- 172.21.15.111:0/3239393389 wait complete. 2024-09-13T17:58:06.565 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:58:06.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.662+0000 7f0c93ad7640 1 Processor -- start 2024-09-13T17:58:06.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.663+0000 7f0c93ad7640 1 -- start start 2024-09-13T17:58:06.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.663+0000 7f0c93ad7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0c8c159310 0x7f0c8c0ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.663+0000 7f0c93ad7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0ce4f0 0x7f0c8c0ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.663+0000 7f0c93ad7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c8c0ccbe0 con 0x7f0c8c0b2b40 2024-09-13T17:58:06.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.663+0000 7f0c93ad7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c8c0ad940 con 0x7f0c8c159310 2024-09-13T17:58:06.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.663+0000 7f0c93ad7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c0b1810 con 0x7f0c8c0ce4f0 2024-09-13T17:58:06.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.664+0000 7f0c91072640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f0c8c0b2b40 0x7f0c8c0ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58798/0 (socket says 172.21.15.111:58798) 2024-09-13T17:58:06.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.664+0000 7f0c91072640 1 -- 172.21.15.111:0/2944240118 learned_addr learned my addr 172.21.15.111:0/2944240118 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:06.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.664+0000 7f0c92074640 1 --2- 172.21.15.111:0/2944240118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0ce4f0 0x7f0c8c0ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.664+0000 7f0c91873640 1 --2- 172.21.15.111:0/2944240118 >> v2:172.21.15.188:3300/0 conn(0x7f0c8c159310 0x7f0c8c0ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.664+0000 7f0c90871640 1 -- 172.21.15.111:0/2944240118 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959275430 0 0) 0x7f0c8c0ccbe0 con 0x7f0c8c0b2b40 2024-09-13T17:58:06.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.664+0000 7f0c90871640 1 -- 172.21.15.111:0/2944240118 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c78003520 con 0x7f0c8c0b2b40 2024-09-13T17:58:06.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.664+0000 7f0c92074640 1 -- 172.21.15.111:0/2944240118 >> v2:172.21.15.188:3300/0 conn(0x7f0c8c159310 msgr2=0x7f0c8c0ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:06.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.664+0000 7f0c92074640 1 --2- 172.21.15.111:0/2944240118 >> v2:172.21.15.188:3300/0 conn(0x7f0c8c159310 0x7f0c8c0ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.664+0000 7f0c92074640 1 -- 172.21.15.111:0/2944240118 >> v1:172.21.15.111:6789/0 conn(0x7f0c8c0b2b40 legacy=0x7f0c8c0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.664+0000 7f0c90871640 1 -- 172.21.15.111:0/2944240118 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3917503712 0 0) 0x7f0c78003520 con 0x7f0c8c0b2b40 2024-09-13T17:58:06.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.664+0000 7f0c92074640 1 -- 172.21.15.111:0/2944240118 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c0cee40 con 0x7f0c8c0ce4f0 2024-09-13T17:58:06.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.665+0000 7f0c92074640 1 --2- 172.21.15.111:0/2944240118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0ce4f0 0x7f0c8c0ce900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0c84009d90 tx=0x7f0c8402c770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89f0cdac18dcd0c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:06.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.665+0000 7f0c90871640 1 -- 172.21.15.111:0/2944240118 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c001670 con 0x7f0c8c0ce4f0 2024-09-13T17:58:06.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.665+0000 7f0c90871640 1 -- 172.21.15.111:0/2944240118 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8402f050 con 0x7f0c8c0ce4f0 2024-09-13T17:58:06.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.665+0000 7f0c90871640 1 -- 172.21.15.111:0/2944240118 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c84007740 con 0x7f0c8c0ce4f0 2024-09-13T17:58:06.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.666+0000 7f0c93ad7640 1 -- 172.21.15.111:0/2944240118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0ce4f0 msgr2=0x7f0c8c0ce900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.666+0000 7f0c93ad7640 1 --2- 172.21.15.111:0/2944240118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0ce4f0 0x7f0c8c0ce900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0c84009d90 tx=0x7f0c8402c770 comp rx=0 tx=0).stop 2024-09-13T17:58:06.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.666+0000 7f0c93ad7640 1 -- 172.21.15.111:0/2944240118 shutdown_connections 2024-09-13T17:58:06.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.666+0000 7f0c93ad7640 1 --2- 172.21.15.111:0/2944240118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0ce4f0 0x7f0c8c0ce900 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.666+0000 7f0c93ad7640 1 --2- 172.21.15.111:0/2944240118 >> v2:172.21.15.188:3300/0 conn(0x7f0c8c159310 0x7f0c8c0ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.666+0000 7f0c93ad7640 1 -- 172.21.15.111:0/2944240118 >> 172.21.15.111:0/2944240118 conn(0x7f0c8c06a0b0 msgr2=0x7f0c8c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:06.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.666+0000 7f0c93ad7640 1 -- 172.21.15.111:0/2944240118 shutdown_connections 2024-09-13T17:58:06.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.666+0000 7f0c93ad7640 1 -- 172.21.15.111:0/2944240118 wait complete. 2024-09-13T17:58:06.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.667+0000 7f0c93ad7640 1 Processor -- start 2024-09-13T17:58:06.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.667+0000 7f0c93ad7640 1 -- start start 2024-09-13T17:58:06.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c93ad7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c8c0b2b40 0x7f0c8c0c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c93ad7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0ce4f0 0x7f0c8c09b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c93ad7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8c159310 0x7f0c8c0b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c93ad7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c1ede50 con 0x7f0c8c0b2b40 2024-09-13T17:58:06.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c93ad7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c0983d0 con 0x7f0c8c159310 2024-09-13T17:58:06.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c93ad7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c0986b0 con 0x7f0c8c0ce4f0 2024-09-13T17:58:06.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c91873640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c8c0b2b40 0x7f0c8c0c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c91873640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c8c0b2b40 0x7f0c8c0c4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50922/0 (socket says 172.21.15.111:50922) 2024-09-13T17:58:06.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c91873640 1 -- 172.21.15.111:0/4194960194 learned_addr learned my addr 172.21.15.111:0/4194960194 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:06.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c91072640 1 --2- 172.21.15.111:0/4194960194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0ce4f0 0x7f0c8c09b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c92074640 1 --2- 172.21.15.111:0/4194960194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8c159310 0x7f0c8c0b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c91873640 1 -- 172.21.15.111:0/4194960194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0ce4f0 msgr2=0x7f0c8c09b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c91873640 1 --2- 172.21.15.111:0/4194960194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0ce4f0 0x7f0c8c09b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c91873640 1 -- 172.21.15.111:0/4194960194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8c159310 msgr2=0x7f0c8c0b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c91873640 1 --2- 172.21.15.111:0/4194960194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8c159310 0x7f0c8c0b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c91873640 1 -- 172.21.15.111:0/4194960194 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c84009a70 con 0x7f0c8c0b2b40 2024-09-13T17:58:06.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c91072640 1 --2- 172.21.15.111:0/4194960194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0ce4f0 0x7f0c8c09b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:06.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c91873640 1 --2- 172.21.15.111:0/4194960194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c8c0b2b40 0x7f0c8c0c4a70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800e870 tx=0x7f0c8800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:06.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c827fc640 1 -- 172.21.15.111:0/4194960194 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c880094f0 con 0x7f0c8c0b2b40 2024-09-13T17:58:06.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c827fc640 1 -- 172.21.15.111:0/4194960194 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8800c940 con 0x7f0c8c0b2b40 2024-09-13T17:58:06.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c93ad7640 1 -- 172.21.15.111:0/4194960194 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8c098990 con 0x7f0c8c0b2b40 2024-09-13T17:58:06.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c827fc640 1 -- 172.21.15.111:0/4194960194 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8800ccf0 con 0x7f0c8c0b2b40 2024-09-13T17:58:06.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.668+0000 7f0c93ad7640 1 -- 172.21.15.111:0/4194960194 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8c0c4fb0 con 0x7f0c8c0b2b40 2024-09-13T17:58:06.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.669+0000 7f0c93ad7640 1 -- 172.21.15.111:0/4194960194 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8c0b1a40 con 0x7f0c8c0b2b40 2024-09-13T17:58:06.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.670+0000 7f0c827fc640 1 -- 172.21.15.111:0/4194960194 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0c88013470 con 0x7f0c8c0b2b40 2024-09-13T17:58:06.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.670+0000 7f0c827fc640 1 --2- 172.21.15.111:0/4194960194 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0c7006ac00 0x7f0c7006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.670+0000 7f0c91072640 1 --2- 172.21.15.111:0/4194960194 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0c7006ac00 0x7f0c7006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.670+0000 7f0c827fc640 1 -- 172.21.15.111:0/4194960194 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0c88089780 con 0x7f0c8c0b2b40 2024-09-13T17:58:06.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.671+0000 7f0c91072640 1 --2- 172.21.15.111:0/4194960194 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0c7006ac00 0x7f0c7006d0c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0011e0 tx=0x7f0c7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:06.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.674+0000 7f0c827fc640 1 -- 172.21.15.111:0/4194960194 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f0c880545e0 con 0x7f0c8c0b2b40 2024-09-13T17:58:06.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.842+0000 7f0c93ad7640 1 -- 172.21.15.111:0/4194960194 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0c8c0ab800 con 0x7f0c8c0b2b40 2024-09-13T17:58:06.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.842+0000 7f0c827fc640 1 -- 172.21.15.111:0/4194960194 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0c88057c00 con 0x7f0c8c0b2b40 2024-09-13T17:58:06.845 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:06.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.844+0000 7f0c93ad7640 1 -- 172.21.15.111:0/4194960194 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0c7006ac00 msgr2=0x7f0c7006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.844+0000 7f0c93ad7640 1 --2- 172.21.15.111:0/4194960194 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0c7006ac00 0x7f0c7006d0c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0011e0 tx=0x7f0c7c009040 comp rx=0 tx=0).stop 2024-09-13T17:58:06.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.844+0000 7f0c93ad7640 1 -- 172.21.15.111:0/4194960194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c8c0b2b40 msgr2=0x7f0c8c0c4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.844+0000 7f0c93ad7640 1 --2- 172.21.15.111:0/4194960194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c8c0b2b40 0x7f0c8c0c4a70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800e870 tx=0x7f0c8800ed30 comp rx=0 tx=0).stop 2024-09-13T17:58:06.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.844+0000 7f0c93ad7640 1 -- 172.21.15.111:0/4194960194 shutdown_connections 2024-09-13T17:58:06.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.844+0000 7f0c93ad7640 1 --2- 172.21.15.111:0/4194960194 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0c7006ac00 0x7f0c7006d0c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.844+0000 7f0c93ad7640 1 --2- 172.21.15.111:0/4194960194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8c159310 0x7f0c8c0b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.844+0000 7f0c93ad7640 1 --2- 172.21.15.111:0/4194960194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0ce4f0 0x7f0c8c09b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.844+0000 7f0c93ad7640 1 --2- 172.21.15.111:0/4194960194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c8c0b2b40 0x7f0c8c0c4a70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.844+0000 7f0c93ad7640 1 -- 172.21.15.111:0/4194960194 >> 172.21.15.111:0/4194960194 conn(0x7f0c8c06a0b0 msgr2=0x7f0c8c157910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:06.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.845+0000 7f0c93ad7640 1 -- 172.21.15.111:0/4194960194 shutdown_connections 2024-09-13T17:58:06.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.845+0000 7f0c93ad7640 1 -- 172.21.15.111:0/4194960194 wait complete. 2024-09-13T17:58:06.857 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:06.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.953+0000 7f9e21c12640 1 Processor -- start 2024-09-13T17:58:06.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.953+0000 7f9e21c12640 1 -- start start 2024-09-13T17:58:06.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.953+0000 7f9e21c12640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e1c15c6e0 0x7f9e1c0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.953+0000 7f9e21c12640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1c0d1840 con 0x7f9e1c0ccb20 2024-09-13T17:58:06.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.953+0000 7f9e21c12640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1c06bd20 con 0x7f9e1c0d07b0 2024-09-13T17:58:06.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.953+0000 7f9e21c12640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c160e50 con 0x7f9e1c15c6e0 2024-09-13T17:58:06.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1bfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f9e1c0ccb20 0x7f9e1c160740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58814/0 (socket says 172.21.15.111:58814) 2024-09-13T17:58:06.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1bfff640 1 -- 172.21.15.111:0/476486496 learned_addr learned my addr 172.21.15.111:0/476486496 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:06.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1b7fe640 1 --2- 172.21.15.111:0/476486496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e1c15c6e0 0x7f9e1c0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1affd640 1 -- 172.21.15.111:0/476486496 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341015092 0 0) 0x7f9e1c0d1840 con 0x7f9e1c0ccb20 2024-09-13T17:58:06.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1affd640 1 -- 172.21.15.111:0/476486496 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e08003520 con 0x7f9e1c0ccb20 2024-09-13T17:58:06.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1affd640 1 -- 172.21.15.111:0/476486496 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 931181889 0 0) 0x7f9e08003520 con 0x7f9e1c0ccb20 2024-09-13T17:58:06.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1affd640 1 -- 172.21.15.111:0/476486496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e1c15c6e0 msgr2=0x7f9e1c0b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:06.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1affd640 1 --2- 172.21.15.111:0/476486496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e1c15c6e0 0x7f9e1c0b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1affd640 1 -- 172.21.15.111:0/476486496 >> v1:172.21.15.188:6789/0 conn(0x7f9e1c0d07b0 legacy=0x7f9e1c160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1affd640 1 -- 172.21.15.111:0/476486496 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1c0d0c30 con 0x7f9e1c0ccb20 2024-09-13T17:58:06.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1affd640 1 -- 172.21.15.111:0/476486496 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f9e140019f0 con 0x7f9e1c0ccb20 2024-09-13T17:58:06.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1affd640 1 -- 172.21.15.111:0/476486496 >> v1:172.21.15.111:6789/0 conn(0x7f9e1c0ccb20 legacy=0x7f9e1c160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1affd640 1 --2- 172.21.15.111:0/476486496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e080039a0 0x7f9e08003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1affd640 1 --2- 172.21.15.111:0/476486496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e08004350 0x7f9e08004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1affd640 1 --2- 172.21.15.111:0/476486496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e1c15c6e0 0x7f9e08005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1affd640 1 -- 172.21.15.111:0/476486496 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c160e50 con 0x7f9e08004350 2024-09-13T17:58:06.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1affd640 1 -- 172.21.15.111:0/476486496 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0800ba30 con 0x7f9e080039a0 2024-09-13T17:58:06.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1affd640 1 -- 172.21.15.111:0/476486496 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0800bbf0 con 0x7f9e1c15c6e0 2024-09-13T17:58:06.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e13fff640 1 --2- 172.21.15.111:0/476486496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e08004350 0x7f9e08004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1b7fe640 1 --2- 172.21.15.111:0/476486496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e080039a0 0x7f9e08003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.954+0000 7f9e1bfff640 1 --2- 172.21.15.111:0/476486496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e1c15c6e0 0x7f9e08005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.955+0000 7f9e13fff640 1 -- 172.21.15.111:0/476486496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e1c15c6e0 msgr2=0x7f9e08005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:06.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.955+0000 7f9e13fff640 1 --2- 172.21.15.111:0/476486496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e1c15c6e0 0x7f9e08005120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.955+0000 7f9e13fff640 1 -- 172.21.15.111:0/476486496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e080039a0 msgr2=0x7f9e08003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:06.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.955+0000 7f9e13fff640 1 --2- 172.21.15.111:0/476486496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e080039a0 0x7f9e08003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.955+0000 7f9e13fff640 1 -- 172.21.15.111:0/476486496 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e08003520 con 0x7f9e08004350 2024-09-13T17:58:06.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.955+0000 7f9e13fff640 1 --2- 172.21.15.111:0/476486496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e08004350 0x7f9e08004820 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c15ff90 tx=0x7f9e080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768c490a0deefcbc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:06.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.955+0000 7f9e1affd640 1 -- 172.21.15.111:0/476486496 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0402ab70 con 0x7f9e08004350 2024-09-13T17:58:06.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.955+0000 7f9e1affd640 1 -- 172.21.15.111:0/476486496 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0402acd0 con 0x7f9e08004350 2024-09-13T17:58:06.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.955+0000 7f9e1affd640 1 -- 172.21.15.111:0/476486496 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e04032640 con 0x7f9e08004350 2024-09-13T17:58:06.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.956+0000 7f9e21c12640 1 -- 172.21.15.111:0/476486496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e08004350 msgr2=0x7f9e08004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.956+0000 7f9e21c12640 1 --2- 172.21.15.111:0/476486496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e08004350 0x7f9e08004820 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c15ff90 tx=0x7f9e080057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.956+0000 7f9e1bfff640 1 -- 172.21.15.111:0/476486496 reap_dead start 2024-09-13T17:58:06.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.956+0000 7f9e21c12640 1 -- 172.21.15.111:0/476486496 shutdown_connections 2024-09-13T17:58:06.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.956+0000 7f9e21c12640 1 -- 172.21.15.111:0/476486496 >> 172.21.15.111:0/476486496 conn(0x7f9e1c089f60 msgr2=0x7f9e1c0d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:06.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.956+0000 7f9e21c12640 1 -- 172.21.15.111:0/476486496 shutdown_connections 2024-09-13T17:58:06.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.956+0000 7f9e21c12640 1 -- 172.21.15.111:0/476486496 wait complete. 2024-09-13T17:58:06.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.957+0000 7f9e21c12640 1 Processor -- start 2024-09-13T17:58:06.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.957+0000 7f9e21c12640 1 -- start start 2024-09-13T17:58:06.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.958+0000 7f9e21c12640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e08004350 0x7f9e1c1f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.958+0000 7f9e21c12640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e1c15d4a0 0x7f9e1c1ef480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.958+0000 7f9e21c12640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e1c15f470 0x7f9e1c0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.958+0000 7f9e21c12640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c15efe0 con 0x7f9e1c15f470 2024-09-13T17:58:06.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.958+0000 7f9e21c12640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c15dc90 con 0x7f9e08004350 2024-09-13T17:58:06.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.958+0000 7f9e21c12640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c15e6b0 con 0x7f9e1c15d4a0 2024-09-13T17:58:06.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.958+0000 7f9e1bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e1c15f470 0x7f9e1c0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.958+0000 7f9e1bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e1c15f470 0x7f9e1c0b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50944/0 (socket says 172.21.15.111:50944) 2024-09-13T17:58:06.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.958+0000 7f9e1bfff640 1 -- 172.21.15.111:0/2021901373 learned_addr learned my addr 172.21.15.111:0/2021901373 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:06.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.958+0000 7f9e1b7fe640 1 --2- 172.21.15.111:0/2021901373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e08004350 0x7f9e1c1f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.958+0000 7f9e13fff640 1 --2- 172.21.15.111:0/2021901373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e1c15d4a0 0x7f9e1c1ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.958+0000 7f9e1bfff640 1 -- 172.21.15.111:0/2021901373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e1c15d4a0 msgr2=0x7f9e1c1ef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.958+0000 7f9e1bfff640 1 --2- 172.21.15.111:0/2021901373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e1c15d4a0 0x7f9e1c1ef480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.958+0000 7f9e1bfff640 1 -- 172.21.15.111:0/2021901373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e08004350 msgr2=0x7f9e1c1f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:06.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.958+0000 7f9e1bfff640 1 --2- 172.21.15.111:0/2021901373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e08004350 0x7f9e1c1f2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:06.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.958+0000 7f9e1bfff640 1 -- 172.21.15.111:0/2021901373 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1c15e990 con 0x7f9e1c15f470 2024-09-13T17:58:06.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.958+0000 7f9e1bfff640 1 --2- 172.21.15.111:0/2021901373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e1c15f470 0x7f9e1c0b4000 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9e14000f80 tx=0x7f9e14007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:06.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.959+0000 7f9e18ff9640 1 -- 172.21.15.111:0/2021901373 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1400ab60 con 0x7f9e1c15f470 2024-09-13T17:58:06.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.959+0000 7f9e18ff9640 1 -- 172.21.15.111:0/2021901373 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1400ace0 con 0x7f9e1c15f470 2024-09-13T17:58:06.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.959+0000 7f9e21c12640 1 -- 172.21.15.111:0/2021901373 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e1c15ec70 con 0x7f9e1c15f470 2024-09-13T17:58:06.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.959+0000 7f9e18ff9640 1 -- 172.21.15.111:0/2021901373 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14013630 con 0x7f9e1c15f470 2024-09-13T17:58:06.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.959+0000 7f9e21c12640 1 -- 172.21.15.111:0/2021901373 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1c15d920 con 0x7f9e1c15f470 2024-09-13T17:58:06.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.960+0000 7f9e21c12640 1 -- 172.21.15.111:0/2021901373 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e1c0b4c80 con 0x7f9e1c15f470 2024-09-13T17:58:06.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.960+0000 7f9e18ff9640 1 -- 172.21.15.111:0/2021901373 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9e14013790 con 0x7f9e1c15f470 2024-09-13T17:58:06.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.961+0000 7f9e18ff9640 1 --2- 172.21.15.111:0/2021901373 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9dfc06ac00 0x7f9dfc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:06.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.961+0000 7f9e1b7fe640 1 --2- 172.21.15.111:0/2021901373 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9dfc06ac00 0x7f9dfc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.961+0000 7f9e18ff9640 1 -- 172.21.15.111:0/2021901373 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9e14089d10 con 0x7f9e1c15f470 2024-09-13T17:58:06.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.961+0000 7f9e1b7fe640 1 --2- 172.21.15.111:0/2021901373 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9dfc06ac00 0x7f9dfc06d0c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c15b6e0 tx=0x7f9e0c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:06.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:06.965+0000 7f9e18ff9640 1 -- 172.21.15.111:0/2021901373 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f9e14054b30 con 0x7f9e1c15f470 2024-09-13T17:58:07.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.128+0000 7f9e21c12640 1 -- 172.21.15.111:0/2021901373 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e1c15f120 con 0x7f9e1c15f470 2024-09-13T17:58:07.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.129+0000 7f9e18ff9640 1 -- 172.21.15.111:0/2021901373 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18794 (secure 0 0 0) 0x7f9e14058150 con 0x7f9e1c15f470 2024-09-13T17:58:07.131 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:07.131 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:06.744803+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:07.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.131+0000 7f9e21c12640 1 -- 172.21.15.111:0/2021901373 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9dfc06ac00 msgr2=0x7f9dfc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.131+0000 7f9e21c12640 1 --2- 172.21.15.111:0/2021901373 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9dfc06ac00 0x7f9dfc06d0c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c15b6e0 tx=0x7f9e0c009190 comp rx=0 tx=0).stop 2024-09-13T17:58:07.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.131+0000 7f9e21c12640 1 -- 172.21.15.111:0/2021901373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e1c15f470 msgr2=0x7f9e1c0b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.131+0000 7f9e21c12640 1 --2- 172.21.15.111:0/2021901373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e1c15f470 0x7f9e1c0b4000 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9e14000f80 tx=0x7f9e14007b10 comp rx=0 tx=0).stop 2024-09-13T17:58:07.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.131+0000 7f9e21c12640 1 -- 172.21.15.111:0/2021901373 shutdown_connections 2024-09-13T17:58:07.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.131+0000 7f9e21c12640 1 --2- 172.21.15.111:0/2021901373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e1c15f470 0x7f9e1c0b4000 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.131+0000 7f9e21c12640 1 --2- 172.21.15.111:0/2021901373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e1c15d4a0 0x7f9e1c1ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.131+0000 7f9e21c12640 1 --2- 172.21.15.111:0/2021901373 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9dfc06ac00 0x7f9dfc06d0c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.131+0000 7f9e21c12640 1 --2- 172.21.15.111:0/2021901373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e08004350 0x7f9e1c1f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.131+0000 7f9e21c12640 1 -- 172.21.15.111:0/2021901373 >> 172.21.15.111:0/2021901373 conn(0x7f9e1c089f60 msgr2=0x7f9e1c0b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:07.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.131+0000 7f9e21c12640 1 -- 172.21.15.111:0/2021901373 shutdown_connections 2024-09-13T17:58:07.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.131+0000 7f9e21c12640 1 -- 172.21.15.111:0/2021901373 wait complete. 2024-09-13T17:58:07.144 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:58:07.144 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:58:07.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.242+0000 7f21362d0640 1 Processor -- start 2024-09-13T17:58:07.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.242+0000 7f21362d0640 1 -- start start 2024-09-13T17:58:07.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.243+0000 7f21362d0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f21300cec00 0x7f2130164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.243+0000 7f21362d0640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f213015a790 con 0x7f213015a3b0 2024-09-13T17:58:07.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.243+0000 7f21362d0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21300b2680 con 0x7f21300b20a0 2024-09-13T17:58:07.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.243+0000 7f21362d0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21300b2960 con 0x7f21300cec00 2024-09-13T17:58:07.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.243+0000 7f213486d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f21300cec00 0x7f2130164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.243+0000 7f212ffff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f213015a3b0 0x7f21300b1720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58820/0 (socket says 172.21.15.111:58820) 2024-09-13T17:58:07.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.243+0000 7f212ffff640 1 -- 172.21.15.111:0/3846689403 learned_addr learned my addr 172.21.15.111:0/3846689403 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:07.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.243+0000 7f213486d640 1 -- 172.21.15.111:0/3846689403 >> v1:172.21.15.188:6789/0 conn(0x7f21300b20a0 legacy=0x7f21300ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:07.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.243+0000 7f213486d640 1 -- 172.21.15.111:0/3846689403 >> v1:172.21.15.111:6789/0 conn(0x7f213015a3b0 legacy=0x7f21300b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.243+0000 7f212f7fe640 1 -- 172.21.15.111:0/3846689403 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700766615 0 0) 0x7f213015a790 con 0x7f213015a3b0 2024-09-13T17:58:07.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.244+0000 7f213486d640 1 -- 172.21.15.111:0/3846689403 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21300b2c40 con 0x7f21300cec00 2024-09-13T17:58:07.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.244+0000 7f213486d640 1 --2- 172.21.15.111:0/3846689403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21300cec00 0x7f2130164ed0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2128009d90 tx=0x7f212802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3b1f69342c7dd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:07.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.244+0000 7f212f7fe640 1 -- 172.21.15.111:0/3846689403 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2128007740 con 0x7f21300cec00 2024-09-13T17:58:07.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.244+0000 7f212f7fe640 1 -- 172.21.15.111:0/3846689403 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212802d050 con 0x7f21300cec00 2024-09-13T17:58:07.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.244+0000 7f212f7fe640 1 -- 172.21.15.111:0/3846689403 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2128007ac0 con 0x7f21300cec00 2024-09-13T17:58:07.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.245+0000 7f21362d0640 1 -- 172.21.15.111:0/3846689403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21300cec00 msgr2=0x7f2130164ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.245+0000 7f21362d0640 1 --2- 172.21.15.111:0/3846689403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21300cec00 0x7f2130164ed0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2128009d90 tx=0x7f212802a6a0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.245+0000 7f21362d0640 1 -- 172.21.15.111:0/3846689403 shutdown_connections 2024-09-13T17:58:07.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.245+0000 7f21362d0640 1 --2- 172.21.15.111:0/3846689403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21300cec00 0x7f2130164ed0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.245+0000 7f21362d0640 1 -- 172.21.15.111:0/3846689403 >> 172.21.15.111:0/3846689403 conn(0x7f213006a0b0 msgr2=0x7f213006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:07.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.245+0000 7f21362d0640 1 -- 172.21.15.111:0/3846689403 shutdown_connections 2024-09-13T17:58:07.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.245+0000 7f21362d0640 1 -- 172.21.15.111:0/3846689403 wait complete. 2024-09-13T17:58:07.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.246+0000 7f21362d0640 1 Processor -- start 2024-09-13T17:58:07.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.246+0000 7f21362d0640 1 -- start start 2024-09-13T17:58:07.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.247+0000 7f21362d0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21300b20a0 0x7f213015d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.247+0000 7f21362d0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21300cec00 0x7f21301f1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.247+0000 7f21362d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f213015a3b0 0x7f21300b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.247+0000 7f21362d0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213015ae90 con 0x7f21300b20a0 2024-09-13T17:58:07.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.247+0000 7f21362d0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213015b170 con 0x7f21300cec00 2024-09-13T17:58:07.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.247+0000 7f21362d0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213015b450 con 0x7f213015a3b0 2024-09-13T17:58:07.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.247+0000 7f212ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21300b20a0 0x7f213015d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.247+0000 7f212ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21300b20a0 0x7f213015d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50958/0 (socket says 172.21.15.111:50958) 2024-09-13T17:58:07.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.247+0000 7f212ffff640 1 -- 172.21.15.111:0/390802933 learned_addr learned my addr 172.21.15.111:0/390802933 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:07.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.247+0000 7f212ffff640 1 -- 172.21.15.111:0/390802933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f213015a3b0 msgr2=0x7f21300b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.247+0000 7f212ffff640 1 --2- 172.21.15.111:0/390802933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f213015a3b0 0x7f21300b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.247+0000 7f212ffff640 1 -- 172.21.15.111:0/390802933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21300cec00 msgr2=0x7f21301f1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.247+0000 7f212ffff640 1 --2- 172.21.15.111:0/390802933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21300cec00 0x7f21301f1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.247+0000 7f212ffff640 1 -- 172.21.15.111:0/390802933 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2128009a70 con 0x7f21300b20a0 2024-09-13T17:58:07.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.247+0000 7f212ffff640 1 --2- 172.21.15.111:0/390802933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21300b20a0 0x7f213015d2a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f212000c4a0 tx=0x7f212000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:07.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.247+0000 7f212d7fa640 1 -- 172.21.15.111:0/390802933 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2120005ce0 con 0x7f21300b20a0 2024-09-13T17:58:07.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.248+0000 7f212d7fa640 1 -- 172.21.15.111:0/390802933 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2120005e40 con 0x7f21300b20a0 2024-09-13T17:58:07.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.248+0000 7f212d7fa640 1 -- 172.21.15.111:0/390802933 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21200072b0 con 0x7f21300b20a0 2024-09-13T17:58:07.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.248+0000 7f21362d0640 1 -- 172.21.15.111:0/390802933 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213015b730 con 0x7f21300b20a0 2024-09-13T17:58:07.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.248+0000 7f21362d0640 1 -- 172.21.15.111:0/390802933 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21301f1e40 con 0x7f21300b20a0 2024-09-13T17:58:07.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.248+0000 7f21362d0640 1 -- 172.21.15.111:0/390802933 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f213015bc50 con 0x7f21300b20a0 2024-09-13T17:58:07.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.249+0000 7f212d7fa640 1 -- 172.21.15.111:0/390802933 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2120007410 con 0x7f21300b20a0 2024-09-13T17:58:07.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.249+0000 7f212d7fa640 1 --2- 172.21.15.111:0/390802933 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f211006ac00 0x7f211006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.249+0000 7f212d7fa640 1 -- 172.21.15.111:0/390802933 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2120088cc0 con 0x7f21300b20a0 2024-09-13T17:58:07.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.249+0000 7f2127fff640 1 --2- 172.21.15.111:0/390802933 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f211006ac00 0x7f211006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.250+0000 7f2127fff640 1 --2- 172.21.15.111:0/390802933 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f211006ac00 0x7f211006d0c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f21300b18b0 tx=0x7f2118006ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:07.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.254+0000 7f212d7fa640 1 -- 172.21.15.111:0/390802933 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f213015bc50 con 0x7f21300b20a0 2024-09-13T17:58:07.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.414+0000 7f21362d0640 1 -- 172.21.15.111:0/390802933 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f213015bc50 con 0x7f21300b20a0 2024-09-13T17:58:07.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.414+0000 7f212d7fa640 1 -- 172.21.15.111:0/390802933 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f213015bc50 con 0x7f21300b20a0 2024-09-13T17:58:07.416 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:58:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.416+0000 7f21362d0640 1 -- 172.21.15.111:0/390802933 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f211006ac00 msgr2=0x7f211006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.416+0000 7f21362d0640 1 --2- 172.21.15.111:0/390802933 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f211006ac00 0x7f211006d0c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f21300b18b0 tx=0x7f2118006ba0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.416+0000 7f21362d0640 1 -- 172.21.15.111:0/390802933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21300b20a0 msgr2=0x7f213015d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.416+0000 7f21362d0640 1 --2- 172.21.15.111:0/390802933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21300b20a0 0x7f213015d2a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f212000c4a0 tx=0x7f212000c960 comp rx=0 tx=0).stop 2024-09-13T17:58:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.416+0000 7f21362d0640 1 -- 172.21.15.111:0/390802933 shutdown_connections 2024-09-13T17:58:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.416+0000 7f21362d0640 1 --2- 172.21.15.111:0/390802933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f213015a3b0 0x7f21300b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.416+0000 7f21362d0640 1 --2- 172.21.15.111:0/390802933 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f211006ac00 0x7f211006d0c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.416+0000 7f21362d0640 1 --2- 172.21.15.111:0/390802933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21300cec00 0x7f21301f1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.416+0000 7f21362d0640 1 --2- 172.21.15.111:0/390802933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21300b20a0 0x7f213015d2a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.416+0000 7f21362d0640 1 -- 172.21.15.111:0/390802933 >> 172.21.15.111:0/390802933 conn(0x7f213006a0b0 msgr2=0x7f21300ca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:07.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.416+0000 7f21362d0640 1 -- 172.21.15.111:0/390802933 shutdown_connections 2024-09-13T17:58:07.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.416+0000 7f21362d0640 1 -- 172.21.15.111:0/390802933 wait complete. 2024-09-13T17:58:07.428 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:58:07.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.527+0000 7f835bfff640 1 Processor -- start 2024-09-13T17:58:07.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.527+0000 7f835bfff640 1 -- start start 2024-09-13T17:58:07.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835bfff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f835c06eb50 0x7f835c0ac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835bfff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f835c0acd50 0x7f835c15fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f835c1602b0 0x7f835c1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835bfff640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f835c160740 con 0x7f835c06eb50 2024-09-13T17:58:07.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835bfff640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f835c0b36e0 con 0x7f835c0acd50 2024-09-13T17:58:07.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835bfff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835c0b2ad0 con 0x7f835c1602b0 2024-09-13T17:58:07.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f835c06eb50 0x7f835c0ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f835c06eb50 0x7f835c0ac810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50974/0 (socket says 172.21.15.111:50974) 2024-09-13T17:58:07.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835affd640 1 -- 172.21.15.111:0/3522139883 learned_addr learned my addr 172.21.15.111:0/3522139883 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:07.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835b7fe640 1 --2- 172.21.15.111:0/3522139883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f835c1602b0 0x7f835c1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835a7fc640 1 --2- 172.21.15.111:0/3522139883 >> v2:172.21.15.188:3300/0 conn(0x7f835c0acd50 0x7f835c15fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835affd640 1 -- 172.21.15.111:0/3522139883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f835c1602b0 msgr2=0x7f835c1529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:07.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835affd640 1 --2- 172.21.15.111:0/3522139883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f835c1602b0 0x7f835c1529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835affd640 1 -- 172.21.15.111:0/3522139883 >> v2:172.21.15.188:3300/0 conn(0x7f835c0acd50 msgr2=0x7f835c15fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:07.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835affd640 1 --2- 172.21.15.111:0/3522139883 >> v2:172.21.15.188:3300/0 conn(0x7f835c0acd50 0x7f835c15fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835affd640 1 -- 172.21.15.111:0/3522139883 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835c0b2db0 con 0x7f835c06eb50 2024-09-13T17:58:07.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835b7fe640 1 --2- 172.21.15.111:0/3522139883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f835c1602b0 0x7f835c1529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:07.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.528+0000 7f835a7fc640 1 --2- 172.21.15.111:0/3522139883 >> v2:172.21.15.188:3300/0 conn(0x7f835c0acd50 0x7f835c15fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:07.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.529+0000 7f835affd640 1 --2- 172.21.15.111:0/3522139883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f835c06eb50 0x7f835c0ac810 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f835c066430 tx=0x7f834c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e632ccde89e10a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:07.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.529+0000 7f8359ffb640 1 -- 172.21.15.111:0/3522139883 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c032440 con 0x7f835c06eb50 2024-09-13T17:58:07.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.529+0000 7f8359ffb640 1 -- 172.21.15.111:0/3522139883 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834c0325a0 con 0x7f835c06eb50 2024-09-13T17:58:07.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.529+0000 7f8359ffb640 1 -- 172.21.15.111:0/3522139883 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c032930 con 0x7f835c06eb50 2024-09-13T17:58:07.535 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.530+0000 7f835bfff640 1 -- 172.21.15.111:0/3522139883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f835c06eb50 msgr2=0x7f835c0ac810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.530+0000 7f835bfff640 1 --2- 172.21.15.111:0/3522139883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f835c06eb50 0x7f835c0ac810 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f835c066430 tx=0x7f834c02a1e0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.530+0000 7f835bfff640 1 -- 172.21.15.111:0/3522139883 shutdown_connections 2024-09-13T17:58:07.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.530+0000 7f835bfff640 1 --2- 172.21.15.111:0/3522139883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f835c1602b0 0x7f835c1529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.530+0000 7f835bfff640 1 --2- 172.21.15.111:0/3522139883 >> v2:172.21.15.188:3300/0 conn(0x7f835c0acd50 0x7f835c15fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.530+0000 7f835bfff640 1 --2- 172.21.15.111:0/3522139883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f835c06eb50 0x7f835c0ac810 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.530+0000 7f835bfff640 1 -- 172.21.15.111:0/3522139883 >> 172.21.15.111:0/3522139883 conn(0x7f835c0ceb30 msgr2=0x7f835c0cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:07.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.530+0000 7f835bfff640 1 -- 172.21.15.111:0/3522139883 shutdown_connections 2024-09-13T17:58:07.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.530+0000 7f835bfff640 1 -- 172.21.15.111:0/3522139883 wait complete. 2024-09-13T17:58:07.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.531+0000 7f835bfff640 1 Processor -- start 2024-09-13T17:58:07.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.531+0000 7f835bfff640 1 -- start start 2024-09-13T17:58:07.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.532+0000 7f835bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f835c06eb50 0x7f835c144350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.532+0000 7f835bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f835c0acd50 0x7f835c0c3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.532+0000 7f835bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f835c1602b0 0x7f835c0c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.532+0000 7f835bfff640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c1f2cd0 con 0x7f835c0acd50 2024-09-13T17:58:07.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.532+0000 7f835bfff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c1f3950 con 0x7f835c06eb50 2024-09-13T17:58:07.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.532+0000 7f835bfff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835c1f3020 con 0x7f835c1602b0 2024-09-13T17:58:07.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.532+0000 7f835a7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f835c0acd50 0x7f835c0c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.532+0000 7f835a7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f835c0acd50 0x7f835c0c3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50978/0 (socket says 172.21.15.111:50978) 2024-09-13T17:58:07.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.532+0000 7f835a7fc640 1 -- 172.21.15.111:0/1762986117 learned_addr learned my addr 172.21.15.111:0/1762986117 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:07.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.532+0000 7f835affd640 1 --2- 172.21.15.111:0/1762986117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f835c06eb50 0x7f835c144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.532+0000 7f835b7fe640 1 --2- 172.21.15.111:0/1762986117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f835c1602b0 0x7f835c0c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.533+0000 7f835a7fc640 1 -- 172.21.15.111:0/1762986117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f835c1602b0 msgr2=0x7f835c0c3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.533+0000 7f835a7fc640 1 --2- 172.21.15.111:0/1762986117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f835c1602b0 0x7f835c0c3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.533+0000 7f835a7fc640 1 -- 172.21.15.111:0/1762986117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f835c06eb50 msgr2=0x7f835c144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.533+0000 7f835a7fc640 1 --2- 172.21.15.111:0/1762986117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f835c06eb50 0x7f835c144350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.533+0000 7f835a7fc640 1 -- 172.21.15.111:0/1762986117 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c0094d0 con 0x7f835c0acd50 2024-09-13T17:58:07.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.533+0000 7f835a7fc640 1 --2- 172.21.15.111:0/1762986117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f835c0acd50 0x7f835c0c3760 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f835000c7f0 tx=0x7f835000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:07.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.533+0000 7f8343fff640 1 -- 172.21.15.111:0/1762986117 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8350007a30 con 0x7f835c0acd50 2024-09-13T17:58:07.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.533+0000 7f8343fff640 1 -- 172.21.15.111:0/1762986117 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8350007b90 con 0x7f835c0acd50 2024-09-13T17:58:07.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.533+0000 7f835bfff640 1 -- 172.21.15.111:0/1762986117 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835c0b3cf0 con 0x7f835c0acd50 2024-09-13T17:58:07.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.533+0000 7f8343fff640 1 -- 172.21.15.111:0/1762986117 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835000ce40 con 0x7f835c0acd50 2024-09-13T17:58:07.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.533+0000 7f835bfff640 1 -- 172.21.15.111:0/1762986117 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835c1f35e0 con 0x7f835c0acd50 2024-09-13T17:58:07.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.534+0000 7f835bfff640 1 -- 172.21.15.111:0/1762986117 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f835c0b3b50 con 0x7f835c0acd50 2024-09-13T17:58:07.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.534+0000 7f8343fff640 1 -- 172.21.15.111:0/1762986117 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f83500113f0 con 0x7f835c0acd50 2024-09-13T17:58:07.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.535+0000 7f8343fff640 1 --2- 172.21.15.111:0/1762986117 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f834806ac00 0x7f834806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.535+0000 7f8343fff640 1 -- 172.21.15.111:0/1762986117 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f83500897c0 con 0x7f835c0acd50 2024-09-13T17:58:07.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.535+0000 7f835affd640 1 --2- 172.21.15.111:0/1762986117 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f834806ac00 0x7f834806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.535+0000 7f835affd640 1 --2- 172.21.15.111:0/1762986117 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f834806ac00 0x7f834806d0c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f834c034000 tx=0x7f834c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:07.540 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.539+0000 7f8343fff640 1 -- 172.21.15.111:0/1762986117 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f835c0b3b50 con 0x7f835c0acd50 2024-09-13T17:58:07.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.702+0000 7f835bfff640 1 -- 172.21.15.111:0/1762986117 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f835c0b3b50 con 0x7f835c0acd50 2024-09-13T17:58:07.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.702+0000 7f8343fff640 1 -- 172.21.15.111:0/1762986117 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f835c0b3b50 con 0x7f835c0acd50 2024-09-13T17:58:07.705 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:07.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.704+0000 7f835bfff640 1 -- 172.21.15.111:0/1762986117 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f834806ac00 msgr2=0x7f834806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.704+0000 7f835bfff640 1 --2- 172.21.15.111:0/1762986117 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f834806ac00 0x7f834806d0c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f834c034000 tx=0x7f834c02e000 comp rx=0 tx=0).stop 2024-09-13T17:58:07.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.704+0000 7f835bfff640 1 -- 172.21.15.111:0/1762986117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f835c0acd50 msgr2=0x7f835c0c3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.704+0000 7f835bfff640 1 --2- 172.21.15.111:0/1762986117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f835c0acd50 0x7f835c0c3760 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f835000c7f0 tx=0x7f835000cbf0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.704+0000 7f835bfff640 1 -- 172.21.15.111:0/1762986117 shutdown_connections 2024-09-13T17:58:07.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.704+0000 7f835bfff640 1 --2- 172.21.15.111:0/1762986117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f835c1602b0 0x7f835c0c3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.704+0000 7f835bfff640 1 --2- 172.21.15.111:0/1762986117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f835c0acd50 0x7f835c0c3760 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.704+0000 7f835bfff640 1 --2- 172.21.15.111:0/1762986117 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f834806ac00 0x7f834806d0c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.704+0000 7f835bfff640 1 --2- 172.21.15.111:0/1762986117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f835c06eb50 0x7f835c144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.704+0000 7f835bfff640 1 -- 172.21.15.111:0/1762986117 >> 172.21.15.111:0/1762986117 conn(0x7f835c0ceb30 msgr2=0x7f835c06d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:07.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.704+0000 7f835bfff640 1 -- 172.21.15.111:0/1762986117 shutdown_connections 2024-09-13T17:58:07.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.704+0000 7f835bfff640 1 -- 172.21.15.111:0/1762986117 wait complete. 2024-09-13T17:58:07.717 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:58:07.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.822+0000 7f3452353640 1 Processor -- start 2024-09-13T17:58:07.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.822+0000 7f3452353640 1 -- start start 2024-09-13T17:58:07.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f3452353640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344c15a3b0 0x7f344c0b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f3452353640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f344c0cec00 0x7f344c0ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f3452353640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f344c15a790 con 0x7f344c0b1720 2024-09-13T17:58:07.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f3452353640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f344c0b2680 con 0x7f344c0cec00 2024-09-13T17:58:07.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f3452353640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0b2960 con 0x7f344c15a3b0 2024-09-13T17:58:07.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344b7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f344c0b1720 0x7f344c0ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58832/0 (socket says 172.21.15.111:58832) 2024-09-13T17:58:07.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344b7fe640 1 -- 172.21.15.111:0/656169061 learned_addr learned my addr 172.21.15.111:0/656169061 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:07.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f34508f0640 1 --2- 172.21.15.111:0/656169061 >> v2:172.21.15.188:3300/0 conn(0x7f344c0cec00 0x7f344c0ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344bfff640 1 --2- 172.21.15.111:0/656169061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344c15a3b0 0x7f344c0b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344affd640 1 -- 172.21.15.111:0/656169061 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591483708 0 0) 0x7f344c15a790 con 0x7f344c0b1720 2024-09-13T17:58:07.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344affd640 1 -- 172.21.15.111:0/656169061 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3434003520 con 0x7f344c0b1720 2024-09-13T17:58:07.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344affd640 1 -- 172.21.15.111:0/656169061 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1380810760 0 0) 0x7f3434003520 con 0x7f344c0b1720 2024-09-13T17:58:07.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344affd640 1 -- 172.21.15.111:0/656169061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344c15a3b0 msgr2=0x7f344c0b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:07.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344affd640 1 --2- 172.21.15.111:0/656169061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344c15a3b0 0x7f344c0b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344affd640 1 -- 172.21.15.111:0/656169061 >> v2:172.21.15.188:3300/0 conn(0x7f344c0cec00 msgr2=0x7f344c0ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:07.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344affd640 1 --2- 172.21.15.111:0/656169061 >> v2:172.21.15.188:3300/0 conn(0x7f344c0cec00 0x7f344c0ab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344affd640 1 -- 172.21.15.111:0/656169061 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c0b2c40 con 0x7f344c0b1720 2024-09-13T17:58:07.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344affd640 1 -- 172.21.15.111:0/656169061 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f3440001670 con 0x7f344c0b1720 2024-09-13T17:58:07.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f34508f0640 1 --2- 172.21.15.111:0/656169061 >> v2:172.21.15.188:3300/0 conn(0x7f344c0cec00 0x7f344c0ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:07.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344affd640 1 -- 172.21.15.111:0/656169061 >> v1:172.21.15.111:6789/0 conn(0x7f344c0b1720 legacy=0x7f344c0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344affd640 1 --2- 172.21.15.111:0/656169061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3434003970 0x7f3434003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344affd640 1 --2- 172.21.15.111:0/656169061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34340047c0 0x7f3434004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344affd640 1 --2- 172.21.15.111:0/656169061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344c15a3b0 0x7f3434005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344affd640 1 -- 172.21.15.111:0/656169061 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0b2680 con 0x7f34340047c0 2024-09-13T17:58:07.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344affd640 1 -- 172.21.15.111:0/656169061 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0b2960 con 0x7f3434003970 2024-09-13T17:58:07.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344affd640 1 -- 172.21.15.111:0/656169061 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343400b980 con 0x7f344c15a3b0 2024-09-13T17:58:07.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344b7fe640 1 --2- 172.21.15.111:0/656169061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34340047c0 0x7f3434004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.823+0000 7f344bfff640 1 --2- 172.21.15.111:0/656169061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3434003970 0x7f3434003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.824+0000 7f34508f0640 1 --2- 172.21.15.111:0/656169061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344c15a3b0 0x7f3434005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.824+0000 7f344b7fe640 1 -- 172.21.15.111:0/656169061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344c15a3b0 msgr2=0x7f3434005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:07.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.824+0000 7f344b7fe640 1 --2- 172.21.15.111:0/656169061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344c15a3b0 0x7f3434005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.824+0000 7f344b7fe640 1 -- 172.21.15.111:0/656169061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3434003970 msgr2=0x7f3434003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:07.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.824+0000 7f344b7fe640 1 --2- 172.21.15.111:0/656169061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3434003970 0x7f3434003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.824+0000 7f344b7fe640 1 -- 172.21.15.111:0/656169061 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3434003520 con 0x7f34340047c0 2024-09-13T17:58:07.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.824+0000 7f34508f0640 1 --2- 172.21.15.111:0/656169061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344c15a3b0 0x7f3434005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:07.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.824+0000 7f344b7fe640 1 --2- 172.21.15.111:0/656169061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34340047c0 0x7f3434004c40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f344000c210 tx=0x7f34340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27704343d917d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:07.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.825+0000 7f344affd640 1 -- 172.21.15.111:0/656169061 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344002e070 con 0x7f34340047c0 2024-09-13T17:58:07.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.825+0000 7f344affd640 1 -- 172.21.15.111:0/656169061 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34400058d0 con 0x7f34340047c0 2024-09-13T17:58:07.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.825+0000 7f344affd640 1 -- 172.21.15.111:0/656169061 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3440005ca0 con 0x7f34340047c0 2024-09-13T17:58:07.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.825+0000 7f3452353640 1 -- 172.21.15.111:0/656169061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34340047c0 msgr2=0x7f3434004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.825+0000 7f3452353640 1 --2- 172.21.15.111:0/656169061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34340047c0 0x7f3434004c40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f344000c210 tx=0x7f34340057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.825+0000 7f34508f0640 1 -- 172.21.15.111:0/656169061 reap_dead start 2024-09-13T17:58:07.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.825+0000 7f3452353640 1 -- 172.21.15.111:0/656169061 shutdown_connections 2024-09-13T17:58:07.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.825+0000 7f3452353640 1 -- 172.21.15.111:0/656169061 >> 172.21.15.111:0/656169061 conn(0x7f344c06a0b0 msgr2=0x7f344c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:07.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.825+0000 7f3452353640 1 -- 172.21.15.111:0/656169061 shutdown_connections 2024-09-13T17:58:07.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.825+0000 7f3452353640 1 -- 172.21.15.111:0/656169061 wait complete. 2024-09-13T17:58:07.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.826+0000 7f3452353640 1 Processor -- start 2024-09-13T17:58:07.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.826+0000 7f3452353640 1 -- start start 2024-09-13T17:58:07.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.827+0000 7f3452353640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34340047c0 0x7f344c0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.827+0000 7f3452353640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344c152cd0 0x7f344c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.827+0000 7f3452353640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f344c06d680 0x7f344c156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.827+0000 7f3452353640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c06db00 con 0x7f344c06d680 2024-09-13T17:58:07.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.827+0000 7f3452353640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0c7b60 con 0x7f34340047c0 2024-09-13T17:58:07.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.827+0000 7f3452353640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0c5c50 con 0x7f344c152cd0 2024-09-13T17:58:07.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.827+0000 7f34508f0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f344c06d680 0x7f344c156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.827+0000 7f34508f0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f344c06d680 0x7f344c156140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50986/0 (socket says 172.21.15.111:50986) 2024-09-13T17:58:07.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.827+0000 7f34508f0640 1 -- 172.21.15.111:0/419326656 learned_addr learned my addr 172.21.15.111:0/419326656 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:07.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.828+0000 7f34508f0640 1 -- 172.21.15.111:0/419326656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344c152cd0 msgr2=0x7f344c06d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.828+0000 7f344b7fe640 1 --2- 172.21.15.111:0/419326656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344c152cd0 0x7f344c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.828+0000 7f344bfff640 1 --2- 172.21.15.111:0/419326656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34340047c0 0x7f344c0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.828+0000 7f34508f0640 1 --2- 172.21.15.111:0/419326656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344c152cd0 0x7f344c06d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.828+0000 7f34508f0640 1 -- 172.21.15.111:0/419326656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34340047c0 msgr2=0x7f344c0c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.828+0000 7f34508f0640 1 --2- 172.21.15.111:0/419326656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34340047c0 0x7f344c0c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.828+0000 7f34508f0640 1 -- 172.21.15.111:0/419326656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c0af610 con 0x7f344c06d680 2024-09-13T17:58:07.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.828+0000 7f34508f0640 1 --2- 172.21.15.111:0/419326656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f344c06d680 0x7f344c156140 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f344c06c5d0 tx=0x7f3444011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:07.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.828+0000 7f3448ff9640 1 -- 172.21.15.111:0/419326656 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344400e030 con 0x7f344c06d680 2024-09-13T17:58:07.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.828+0000 7f3448ff9640 1 -- 172.21.15.111:0/419326656 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3444016470 con 0x7f344c06d680 2024-09-13T17:58:07.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.828+0000 7f3448ff9640 1 -- 172.21.15.111:0/419326656 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3444016820 con 0x7f344c06d680 2024-09-13T17:58:07.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.828+0000 7f3452353640 1 -- 172.21.15.111:0/419326656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344c0aea00 con 0x7f344c06d680 2024-09-13T17:58:07.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.828+0000 7f3452353640 1 -- 172.21.15.111:0/419326656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344c0aefc0 con 0x7f344c06d680 2024-09-13T17:58:07.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.829+0000 7f3452353640 1 -- 172.21.15.111:0/419326656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344c0ad1b0 con 0x7f344c06d680 2024-09-13T17:58:07.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.829+0000 7f3448ff9640 1 -- 172.21.15.111:0/419326656 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3444016980 con 0x7f344c06d680 2024-09-13T17:58:07.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.830+0000 7f3448ff9640 1 --2- 172.21.15.111:0/419326656 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f343806ac00 0x7f343806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:07.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.830+0000 7f344bfff640 1 --2- 172.21.15.111:0/419326656 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f343806ac00 0x7f343806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:07.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.830+0000 7f3448ff9640 1 -- 172.21.15.111:0/419326656 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f344400f070 con 0x7f344c06d680 2024-09-13T17:58:07.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.830+0000 7f344bfff640 1 --2- 172.21.15.111:0/419326656 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f343806ac00 0x7f343806d0c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f343c0098a0 tx=0x7f343c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:07.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.834+0000 7f3448ff9640 1 -- 172.21.15.111:0/419326656 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f3444055160 con 0x7f344c06d680 2024-09-13T17:58:07.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.993+0000 7f3452353640 1 -- 172.21.15.111:0/419326656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f344c066f30 con 0x7f344c06d680 2024-09-13T17:58:07.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.993+0000 7f3448ff9640 1 -- 172.21.15.111:0/419326656 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3444058780 con 0x7f344c06d680 2024-09-13T17:58:07.995 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:58:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.995+0000 7f3452353640 1 -- 172.21.15.111:0/419326656 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f343806ac00 msgr2=0x7f343806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.995+0000 7f3452353640 1 --2- 172.21.15.111:0/419326656 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f343806ac00 0x7f343806d0c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f343c0098a0 tx=0x7f343c008040 comp rx=0 tx=0).stop 2024-09-13T17:58:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.995+0000 7f3452353640 1 -- 172.21.15.111:0/419326656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f344c06d680 msgr2=0x7f344c156140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.995+0000 7f3452353640 1 --2- 172.21.15.111:0/419326656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f344c06d680 0x7f344c156140 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f344c06c5d0 tx=0x7f3444011890 comp rx=0 tx=0).stop 2024-09-13T17:58:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.995+0000 7f3452353640 1 -- 172.21.15.111:0/419326656 shutdown_connections 2024-09-13T17:58:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.995+0000 7f3452353640 1 --2- 172.21.15.111:0/419326656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f344c06d680 0x7f344c156140 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.996+0000 7f3452353640 1 --2- 172.21.15.111:0/419326656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344c152cd0 0x7f344c06d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.996+0000 7f3452353640 1 --2- 172.21.15.111:0/419326656 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f343806ac00 0x7f343806d0c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.996+0000 7f3452353640 1 --2- 172.21.15.111:0/419326656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34340047c0 0x7f344c0c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.996+0000 7f3452353640 1 -- 172.21.15.111:0/419326656 >> 172.21.15.111:0/419326656 conn(0x7f344c06a0b0 msgr2=0x7f344c0c9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.996+0000 7f3452353640 1 -- 172.21.15.111:0/419326656 shutdown_connections 2024-09-13T17:58:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:07.996+0000 7f3452353640 1 -- 172.21.15.111:0/419326656 wait complete. 2024-09-13T17:58:08.008 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:58:08.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.107+0000 7facd0701640 1 Processor -- start 2024-09-13T17:58:08.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.107+0000 7facd0701640 1 -- start start 2024-09-13T17:58:08.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.108+0000 7facd0701640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facc8158b00 0x7facc806c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.108+0000 7facd0701640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7facc806c6d0 0x7facc8070b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.108+0000 7facd0701640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc806cb30 con 0x7facc8071050 2024-09-13T17:58:08.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.108+0000 7facd0701640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc80b2d10 con 0x7facc806c6d0 2024-09-13T17:58:08.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.108+0000 7facd0701640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc80b2ff0 con 0x7facc8158b00 2024-09-13T17:58:08.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.108+0000 7faccdc9c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7facc806c6d0 0x7facc8070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.108+0000 7faccec9e640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7facc8071050 0x7facc8071500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58844/0 (socket says 172.21.15.111:58844) 2024-09-13T17:58:08.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.108+0000 7faccec9e640 1 -- 172.21.15.111:0/2873014523 learned_addr learned my addr 172.21.15.111:0/2873014523 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:08.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.108+0000 7facce49d640 1 --2- 172.21.15.111:0/2873014523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facc8158b00 0x7facc806c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.108+0000 7faccdc9c640 1 -- 172.21.15.111:0/2873014523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facc8158b00 msgr2=0x7facc806c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:08.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.108+0000 7faccdc9c640 1 --2- 172.21.15.111:0/2873014523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facc8158b00 0x7facc806c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.108+0000 7faccdc9c640 1 -- 172.21.15.111:0/2873014523 >> v1:172.21.15.111:6789/0 conn(0x7facc8071050 legacy=0x7facc8071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.108+0000 7faccdc9c640 1 -- 172.21.15.111:0/2873014523 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc8070310 con 0x7facc806c6d0 2024-09-13T17:58:08.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.108+0000 7faccd49b640 1 -- 172.21.15.111:0/2873014523 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1206031027 0 0) 0x7facc806cb30 con 0x7facc8071050 2024-09-13T17:58:08.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.108+0000 7facce49d640 1 --2- 172.21.15.111:0/2873014523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facc8158b00 0x7facc806c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:08.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.109+0000 7faccdc9c640 1 --2- 172.21.15.111:0/2873014523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facc806c6d0 0x7facc8070b10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7facb80097a0 tx=0x7facb802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2e9d800a14d63e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:08.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.109+0000 7faccd49b640 1 -- 172.21.15.111:0/2873014523 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8032440 con 0x7facc806c6d0 2024-09-13T17:58:08.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.109+0000 7faccd49b640 1 -- 172.21.15.111:0/2873014523 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb80325a0 con 0x7facc806c6d0 2024-09-13T17:58:08.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.109+0000 7faccd49b640 1 -- 172.21.15.111:0/2873014523 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8032930 con 0x7facc806c6d0 2024-09-13T17:58:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.109+0000 7facd0701640 1 -- 172.21.15.111:0/2873014523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facc806c6d0 msgr2=0x7facc8070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.109+0000 7facd0701640 1 --2- 172.21.15.111:0/2873014523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facc806c6d0 0x7facc8070b10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7facb80097a0 tx=0x7facb802a180 comp rx=0 tx=0).stop 2024-09-13T17:58:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.109+0000 7facd0701640 1 -- 172.21.15.111:0/2873014523 shutdown_connections 2024-09-13T17:58:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.109+0000 7facd0701640 1 --2- 172.21.15.111:0/2873014523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facc806c6d0 0x7facc8070b10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.109+0000 7facd0701640 1 --2- 172.21.15.111:0/2873014523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facc8158b00 0x7facc806c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.109+0000 7facd0701640 1 -- 172.21.15.111:0/2873014523 >> 172.21.15.111:0/2873014523 conn(0x7facc806a0b0 msgr2=0x7facc806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:08.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.110+0000 7facd0701640 1 -- 172.21.15.111:0/2873014523 shutdown_connections 2024-09-13T17:58:08.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.110+0000 7facd0701640 1 -- 172.21.15.111:0/2873014523 wait complete. 2024-09-13T17:58:08.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.111+0000 7facd0701640 1 Processor -- start 2024-09-13T17:58:08.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.111+0000 7facd0701640 1 -- start start 2024-09-13T17:58:08.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.111+0000 7facd0701640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facc806c6d0 0x7facc80c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.111+0000 7facd0701640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facc8071050 0x7facc8150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.111+0000 7facd0701640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facc8158b00 0x7facc8097030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.111+0000 7facd0701640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc80c32f0 con 0x7facc8071050 2024-09-13T17:58:08.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.111+0000 7facd0701640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc80c1dc0 con 0x7facc806c6d0 2024-09-13T17:58:08.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.111+0000 7facd0701640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc80c2670 con 0x7facc8158b00 2024-09-13T17:58:08.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.111+0000 7faccdc9c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facc8071050 0x7facc8150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.111+0000 7faccdc9c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facc8071050 0x7facc8150b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51006/0 (socket says 172.21.15.111:51006) 2024-09-13T17:58:08.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.111+0000 7faccdc9c640 1 -- 172.21.15.111:0/71269497 learned_addr learned my addr 172.21.15.111:0/71269497 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:08.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.112+0000 7faccec9e640 1 --2- 172.21.15.111:0/71269497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facc8158b00 0x7facc8097030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.112+0000 7facce49d640 1 --2- 172.21.15.111:0/71269497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facc806c6d0 0x7facc80c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.112+0000 7faccdc9c640 1 -- 172.21.15.111:0/71269497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facc8158b00 msgr2=0x7facc8097030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.112+0000 7faccdc9c640 1 --2- 172.21.15.111:0/71269497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facc8158b00 0x7facc8097030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.112+0000 7faccdc9c640 1 -- 172.21.15.111:0/71269497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facc806c6d0 msgr2=0x7facc80c1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.112+0000 7faccdc9c640 1 --2- 172.21.15.111:0/71269497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facc806c6d0 0x7facc80c1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.112+0000 7faccdc9c640 1 -- 172.21.15.111:0/71269497 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb8009400 con 0x7facc8071050 2024-09-13T17:58:08.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.112+0000 7faccdc9c640 1 --2- 172.21.15.111:0/71269497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facc8071050 0x7facc8150b80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7facb8035040 tx=0x7facb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:08.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.112+0000 7facbeffd640 1 -- 172.21.15.111:0/71269497 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8006e10 con 0x7facc8071050 2024-09-13T17:58:08.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.112+0000 7facbeffd640 1 -- 172.21.15.111:0/71269497 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb8030980 con 0x7facc8071050 2024-09-13T17:58:08.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.112+0000 7facd0701640 1 -- 172.21.15.111:0/71269497 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc80c2950 con 0x7facc8071050 2024-09-13T17:58:08.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.112+0000 7facbeffd640 1 -- 172.21.15.111:0/71269497 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8030d50 con 0x7facc8071050 2024-09-13T17:58:08.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.112+0000 7facd0701640 1 -- 172.21.15.111:0/71269497 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc80c2f10 con 0x7facc8071050 2024-09-13T17:58:08.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.113+0000 7facd0701640 1 -- 172.21.15.111:0/71269497 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc80c3430 con 0x7facc8071050 2024-09-13T17:58:08.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.113+0000 7facbeffd640 1 -- 172.21.15.111:0/71269497 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7facb8039040 con 0x7facc8071050 2024-09-13T17:58:08.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.114+0000 7facbeffd640 1 --2- 172.21.15.111:0/71269497 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7facac06ac00 0x7facac06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.114+0000 7facbeffd640 1 -- 172.21.15.111:0/71269497 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7facb80aa180 con 0x7facc8071050 2024-09-13T17:58:08.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.114+0000 7facce49d640 1 --2- 172.21.15.111:0/71269497 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7facac06ac00 0x7facac06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.114+0000 7facce49d640 1 --2- 172.21.15.111:0/71269497 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7facac06ac00 0x7facac06d0c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7facc80af510 tx=0x7facc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:08.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.118+0000 7facbeffd640 1 -- 172.21.15.111:0/71269497 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7facc80c3430 con 0x7facc8071050 2024-09-13T17:58:08.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.280+0000 7facd0701640 1 -- 172.21.15.111:0/71269497 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7facc80c3430 con 0x7facc8071050 2024-09-13T17:58:08.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.281+0000 7facbeffd640 1 -- 172.21.15.111:0/71269497 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7facc80c3430 con 0x7facc8071050 2024-09-13T17:58:08.283 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:08.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.283+0000 7facd0701640 1 -- 172.21.15.111:0/71269497 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7facac06ac00 msgr2=0x7facac06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.283+0000 7facd0701640 1 --2- 172.21.15.111:0/71269497 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7facac06ac00 0x7facac06d0c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7facc80af510 tx=0x7facc4008040 comp rx=0 tx=0).stop 2024-09-13T17:58:08.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.283+0000 7facd0701640 1 -- 172.21.15.111:0/71269497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facc8071050 msgr2=0x7facc8150b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.283+0000 7facd0701640 1 --2- 172.21.15.111:0/71269497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facc8071050 0x7facc8150b80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7facb8035040 tx=0x7facb80046c0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.283+0000 7facd0701640 1 -- 172.21.15.111:0/71269497 shutdown_connections 2024-09-13T17:58:08.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.283+0000 7facd0701640 1 --2- 172.21.15.111:0/71269497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facc8158b00 0x7facc8097030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.283+0000 7facd0701640 1 --2- 172.21.15.111:0/71269497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facc8071050 0x7facc8150b80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.283+0000 7facd0701640 1 --2- 172.21.15.111:0/71269497 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7facac06ac00 0x7facac06d0c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.283+0000 7facd0701640 1 --2- 172.21.15.111:0/71269497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facc806c6d0 0x7facc80c1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.283+0000 7facd0701640 1 -- 172.21.15.111:0/71269497 >> 172.21.15.111:0/71269497 conn(0x7facc806a0b0 msgr2=0x7facc8071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:08.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.283+0000 7facd0701640 1 -- 172.21.15.111:0/71269497 shutdown_connections 2024-09-13T17:58:08.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.283+0000 7facd0701640 1 -- 172.21.15.111:0/71269497 wait complete. 2024-09-13T17:58:08.296 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:08.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.392+0000 7fc1c98e9640 1 Processor -- start 2024-09-13T17:58:08.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.392+0000 7fc1c98e9640 1 -- start start 2024-09-13T17:58:08.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.393+0000 7fc1c98e9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc1c4072110 0x7fc1c4070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.393+0000 7fc1c98e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c40c8920 0x7fc1c40c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.393+0000 7fc1c98e9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c40abdf0 con 0x7fc1c4071d30 2024-09-13T17:58:08.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.393+0000 7fc1c98e9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c40c9310 con 0x7fc1c4072110 2024-09-13T17:58:08.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.393+0000 7fc1c98e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40b2280 con 0x7fc1c40c8920 2024-09-13T17:58:08.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.393+0000 7fc1c2ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc1c4071d30 0x7fc1c40b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58848/0 (socket says 172.21.15.111:58848) 2024-09-13T17:58:08.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.393+0000 7fc1c2ffd640 1 -- 172.21.15.111:0/1417017886 learned_addr learned my addr 172.21.15.111:0/1417017886 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:08.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c1ffb640 1 -- 172.21.15.111:0/1417017886 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257800670 0 0) 0x7fc1c40abdf0 con 0x7fc1c4071d30 2024-09-13T17:58:08.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c1ffb640 1 -- 172.21.15.111:0/1417017886 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1ac003520 con 0x7fc1c4071d30 2024-09-13T17:58:08.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c27fc640 1 --2- 172.21.15.111:0/1417017886 >> v2:172.21.15.188:3300/0 conn(0x7fc1c4072110 0x7fc1c4070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c37fe640 1 --2- 172.21.15.111:0/1417017886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c40c8920 0x7fc1c40c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c1ffb640 1 -- 172.21.15.111:0/1417017886 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 621583009 0 0) 0x7fc1ac003520 con 0x7fc1c4071d30 2024-09-13T17:58:08.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c1ffb640 1 -- 172.21.15.111:0/1417017886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c40c8920 msgr2=0x7fc1c40c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:08.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c1ffb640 1 --2- 172.21.15.111:0/1417017886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c40c8920 0x7fc1c40c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c1ffb640 1 -- 172.21.15.111:0/1417017886 >> v2:172.21.15.188:3300/0 conn(0x7fc1c4072110 msgr2=0x7fc1c4070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:08.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c1ffb640 1 --2- 172.21.15.111:0/1417017886 >> v2:172.21.15.188:3300/0 conn(0x7fc1c4072110 0x7fc1c4070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c1ffb640 1 -- 172.21.15.111:0/1417017886 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c40ab1e0 con 0x7fc1c4071d30 2024-09-13T17:58:08.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c1ffb640 1 -- 172.21.15.111:0/1417017886 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7fc1b4001670 con 0x7fc1c4071d30 2024-09-13T17:58:08.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c1ffb640 1 -- 172.21.15.111:0/1417017886 >> v1:172.21.15.111:6789/0 conn(0x7fc1c4071d30 legacy=0x7fc1c40b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c1ffb640 1 --2- 172.21.15.111:0/1417017886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ac003970 0x7fc1ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c1ffb640 1 --2- 172.21.15.111:0/1417017886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1ac0047c0 0x7fc1ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c1ffb640 1 --2- 172.21.15.111:0/1417017886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ac005180 0x7fc1ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c1ffb640 1 -- 172.21.15.111:0/1417017886 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40c9310 con 0x7fc1ac0047c0 2024-09-13T17:58:08.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c1ffb640 1 -- 172.21.15.111:0/1417017886 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40b2280 con 0x7fc1ac003970 2024-09-13T17:58:08.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c1ffb640 1 -- 172.21.15.111:0/1417017886 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac00bfc0 con 0x7fc1ac005180 2024-09-13T17:58:08.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c27fc640 1 --2- 172.21.15.111:0/1417017886 >> v2:172.21.15.188:3300/0 conn(0x7fc1c4072110 0x7fc1c4070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:08.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c37fe640 1 --2- 172.21.15.111:0/1417017886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c40c8920 0x7fc1c40c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:08.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c27fc640 1 --2- 172.21.15.111:0/1417017886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1ac0047c0 0x7fc1ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c2ffd640 1 --2- 172.21.15.111:0/1417017886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ac003970 0x7fc1ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c37fe640 1 --2- 172.21.15.111:0/1417017886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ac005180 0x7fc1ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c27fc640 1 -- 172.21.15.111:0/1417017886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ac005180 msgr2=0x7fc1ac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:08.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c27fc640 1 --2- 172.21.15.111:0/1417017886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ac005180 0x7fc1ac00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c27fc640 1 -- 172.21.15.111:0/1417017886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ac003970 msgr2=0x7fc1ac003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:08.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c27fc640 1 --2- 172.21.15.111:0/1417017886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ac003970 0x7fc1ac003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.394+0000 7fc1c27fc640 1 -- 172.21.15.111:0/1417017886 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac003520 con 0x7fc1ac0047c0 2024-09-13T17:58:08.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.395+0000 7fc1c27fc640 1 --2- 172.21.15.111:0/1417017886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1ac0047c0 0x7fc1ac004c40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009600 tx=0x7fc1ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d7e8cee48dfc73 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:08.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.395+0000 7fc1c1ffb640 1 -- 172.21.15.111:0/1417017886 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8033600 con 0x7fc1ac0047c0 2024-09-13T17:58:08.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.395+0000 7fc1c1ffb640 1 -- 172.21.15.111:0/1417017886 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b802f050 con 0x7fc1ac0047c0 2024-09-13T17:58:08.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.395+0000 7fc1c1ffb640 1 -- 172.21.15.111:0/1417017886 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8033960 con 0x7fc1ac0047c0 2024-09-13T17:58:08.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.396+0000 7fc1c98e9640 1 -- 172.21.15.111:0/1417017886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1ac0047c0 msgr2=0x7fc1ac004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.396+0000 7fc1c98e9640 1 --2- 172.21.15.111:0/1417017886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1ac0047c0 0x7fc1ac004c40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009600 tx=0x7fc1ac0057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.396+0000 7fc1c37fe640 1 -- 172.21.15.111:0/1417017886 reap_dead start 2024-09-13T17:58:08.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.396+0000 7fc1c98e9640 1 -- 172.21.15.111:0/1417017886 shutdown_connections 2024-09-13T17:58:08.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.396+0000 7fc1c98e9640 1 -- 172.21.15.111:0/1417017886 >> 172.21.15.111:0/1417017886 conn(0x7fc1c4069ed0 msgr2=0x7fc1c406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:08.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.396+0000 7fc1c98e9640 1 -- 172.21.15.111:0/1417017886 shutdown_connections 2024-09-13T17:58:08.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.396+0000 7fc1c98e9640 1 -- 172.21.15.111:0/1417017886 wait complete. 2024-09-13T17:58:08.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.397+0000 7fc1c98e9640 1 Processor -- start 2024-09-13T17:58:08.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.397+0000 7fc1c98e9640 1 -- start start 2024-09-13T17:58:08.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c98e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ac0047c0 0x7fc1c40c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c98e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c409b920 0x7fc1c40c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c98e9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c40c1fc0 0x7fc1c41eddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c98e9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40c2440 con 0x7fc1c40c1fc0 2024-09-13T17:58:08.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c98e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40c4cd0 con 0x7fc1ac0047c0 2024-09-13T17:58:08.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c98e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40c3a50 con 0x7fc1c409b920 2024-09-13T17:58:08.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c40c1fc0 0x7fc1c41eddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c40c1fc0 0x7fc1c41eddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51038/0 (socket says 172.21.15.111:51038) 2024-09-13T17:58:08.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c37fe640 1 -- 172.21.15.111:0/625451863 learned_addr learned my addr 172.21.15.111:0/625451863 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:08.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c2ffd640 1 --2- 172.21.15.111:0/625451863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ac0047c0 0x7fc1c40c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c27fc640 1 --2- 172.21.15.111:0/625451863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c409b920 0x7fc1c40c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c37fe640 1 -- 172.21.15.111:0/625451863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c409b920 msgr2=0x7fc1c40c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c37fe640 1 --2- 172.21.15.111:0/625451863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c409b920 0x7fc1c40c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c37fe640 1 -- 172.21.15.111:0/625451863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ac0047c0 msgr2=0x7fc1c40c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c37fe640 1 --2- 172.21.15.111:0/625451863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ac0047c0 0x7fc1c40c4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c37fe640 1 -- 172.21.15.111:0/625451863 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c40c3cd0 con 0x7fc1c40c1fc0 2024-09-13T17:58:08.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c27fc640 1 --2- 172.21.15.111:0/625451863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c409b920 0x7fc1c40c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:08.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1c37fe640 1 --2- 172.21.15.111:0/625451863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c40c1fc0 0x7fc1c41eddc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0045e0 tx=0x7fc1bc007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:08.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.398+0000 7fc1abfff640 1 -- 172.21.15.111:0/625451863 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc015480 con 0x7fc1c40c1fc0 2024-09-13T17:58:08.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.399+0000 7fc1abfff640 1 -- 172.21.15.111:0/625451863 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc00f040 con 0x7fc1c40c1fc0 2024-09-13T17:58:08.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.399+0000 7fc1c98e9640 1 -- 172.21.15.111:0/625451863 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c41eeb60 con 0x7fc1c40c1fc0 2024-09-13T17:58:08.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.399+0000 7fc1abfff640 1 -- 172.21.15.111:0/625451863 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc015820 con 0x7fc1c40c1fc0 2024-09-13T17:58:08.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.399+0000 7fc1c98e9640 1 -- 172.21.15.111:0/625451863 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c40c4270 con 0x7fc1c40c1fc0 2024-09-13T17:58:08.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.399+0000 7fc1c98e9640 1 -- 172.21.15.111:0/625451863 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c40c4460 con 0x7fc1c40c1fc0 2024-09-13T17:58:08.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.400+0000 7fc1abfff640 1 -- 172.21.15.111:0/625451863 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc1bc015980 con 0x7fc1c40c1fc0 2024-09-13T17:58:08.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.400+0000 7fc1abfff640 1 --2- 172.21.15.111:0/625451863 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc1b006ac30 0x7fc1b006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.400+0000 7fc1abfff640 1 -- 172.21.15.111:0/625451863 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc1bc08a210 con 0x7fc1c40c1fc0 2024-09-13T17:58:08.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.400+0000 7fc1c2ffd640 1 --2- 172.21.15.111:0/625451863 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc1b006ac30 0x7fc1b006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.401+0000 7fc1c2ffd640 1 --2- 172.21.15.111:0/625451863 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc1b006ac30 0x7fc1b006d0f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4000c00 tx=0x7fc1b40078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:08.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.405+0000 7fc1abfff640 1 -- 172.21.15.111:0/625451863 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc1bc055030 con 0x7fc1c40c1fc0 2024-09-13T17:58:08.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.567+0000 7fc1c98e9640 1 -- 172.21.15.111:0/625451863 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1c40c8dd0 con 0x7fc1c40c1fc0 2024-09-13T17:58:08.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.568+0000 7fc1abfff640 1 -- 172.21.15.111:0/625451863 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18794 (secure 0 0 0) 0x7fc1bc058650 con 0x7fc1c40c1fc0 2024-09-13T17:58:08.570 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:08.571 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":26,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:07.749113+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:08.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.570+0000 7fc1c98e9640 1 -- 172.21.15.111:0/625451863 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc1b006ac30 msgr2=0x7fc1b006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.570+0000 7fc1c98e9640 1 --2- 172.21.15.111:0/625451863 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc1b006ac30 0x7fc1b006d0f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4000c00 tx=0x7fc1b40078f0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.570+0000 7fc1c98e9640 1 -- 172.21.15.111:0/625451863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c40c1fc0 msgr2=0x7fc1c41eddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.570+0000 7fc1c98e9640 1 --2- 172.21.15.111:0/625451863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c40c1fc0 0x7fc1c41eddc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0045e0 tx=0x7fc1bc007700 comp rx=0 tx=0).stop 2024-09-13T17:58:08.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.570+0000 7fc1c98e9640 1 -- 172.21.15.111:0/625451863 shutdown_connections 2024-09-13T17:58:08.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.570+0000 7fc1c98e9640 1 --2- 172.21.15.111:0/625451863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c40c1fc0 0x7fc1c41eddc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.570+0000 7fc1c98e9640 1 --2- 172.21.15.111:0/625451863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c409b920 0x7fc1c40c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.570+0000 7fc1c98e9640 1 --2- 172.21.15.111:0/625451863 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc1b006ac30 0x7fc1b006d0f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.570+0000 7fc1c98e9640 1 --2- 172.21.15.111:0/625451863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ac0047c0 0x7fc1c40c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.570+0000 7fc1c98e9640 1 -- 172.21.15.111:0/625451863 >> 172.21.15.111:0/625451863 conn(0x7fc1c4069ed0 msgr2=0x7fc1c40ade10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:08.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.570+0000 7fc1c98e9640 1 -- 172.21.15.111:0/625451863 shutdown_connections 2024-09-13T17:58:08.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.570+0000 7fc1c98e9640 1 -- 172.21.15.111:0/625451863 wait complete. 2024-09-13T17:58:08.583 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:58:08.584 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:58:08.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.682+0000 7f9a86576640 1 Processor -- start 2024-09-13T17:58:08.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.682+0000 7f9a86576640 1 -- start start 2024-09-13T17:58:08.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.683+0000 7f9a86576640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9a8015c500 0x7f9a800d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.683+0000 7f9a86576640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9a800cdff0 0x7f9a800d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.683+0000 7f9a86576640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a800cd450 con 0x7f9a800d2590 2024-09-13T17:58:08.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.683+0000 7f9a86576640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a800d29f0 con 0x7f9a800cdff0 2024-09-13T17:58:08.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.683+0000 7f9a86576640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a800d0cf0 con 0x7f9a8015c500 2024-09-13T17:58:08.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.683+0000 7f9a84b13640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9a800cdff0 0x7f9a800d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.683+0000 7f9a84b13640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9a800cdff0 0x7f9a800d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51050/0 (socket says 172.21.15.111:51050) 2024-09-13T17:58:08.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.683+0000 7f9a84b13640 1 -- 172.21.15.111:0/1756039924 learned_addr learned my addr 172.21.15.111:0/1756039924 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:08.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.684+0000 7f9a7ffff640 1 --2- 172.21.15.111:0/1756039924 >> v2:172.21.15.188:3300/0 conn(0x7f9a8015c500 0x7f9a800d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.684+0000 7f9a84b13640 1 -- 172.21.15.111:0/1756039924 >> v2:172.21.15.188:3300/0 conn(0x7f9a8015c500 msgr2=0x7f9a800d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:08.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.684+0000 7f9a84b13640 1 --2- 172.21.15.111:0/1756039924 >> v2:172.21.15.188:3300/0 conn(0x7f9a8015c500 0x7f9a800d2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.684+0000 7f9a84b13640 1 -- 172.21.15.111:0/1756039924 >> v1:172.21.15.188:6789/0 conn(0x7f9a800d2590 legacy=0x7f9a800cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.684+0000 7f9a84b13640 1 -- 172.21.15.111:0/1756039924 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a800d0fd0 con 0x7f9a800cdff0 2024-09-13T17:58:08.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.684+0000 7f9a84b13640 1 --2- 172.21.15.111:0/1756039924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a800cdff0 0x7f9a800d0000 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9a800cfec0 tx=0x7f9a7802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3715bca70bd2b913 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:08.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.685+0000 7f9a7effd640 1 -- 172.21.15.111:0/1756039924 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78031600 con 0x7f9a800cdff0 2024-09-13T17:58:08.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.685+0000 7f9a7effd640 1 -- 172.21.15.111:0/1756039924 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7802d050 con 0x7f9a800cdff0 2024-09-13T17:58:08.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.685+0000 7f9a7effd640 1 -- 172.21.15.111:0/1756039924 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78031980 con 0x7f9a800cdff0 2024-09-13T17:58:08.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.685+0000 7f9a86576640 1 -- 172.21.15.111:0/1756039924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a800cdff0 msgr2=0x7f9a800d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.685+0000 7f9a86576640 1 --2- 172.21.15.111:0/1756039924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a800cdff0 0x7f9a800d0000 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9a800cfec0 tx=0x7f9a7802a730 comp rx=0 tx=0).stop 2024-09-13T17:58:08.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.685+0000 7f9a86576640 1 -- 172.21.15.111:0/1756039924 shutdown_connections 2024-09-13T17:58:08.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.685+0000 7f9a86576640 1 --2- 172.21.15.111:0/1756039924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a800cdff0 0x7f9a800d0000 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.685+0000 7f9a86576640 1 --2- 172.21.15.111:0/1756039924 >> v2:172.21.15.188:3300/0 conn(0x7f9a8015c500 0x7f9a800d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.685+0000 7f9a86576640 1 -- 172.21.15.111:0/1756039924 >> 172.21.15.111:0/1756039924 conn(0x7f9a80089f60 msgr2=0x7f9a8009b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:08.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.685+0000 7f9a86576640 1 -- 172.21.15.111:0/1756039924 shutdown_connections 2024-09-13T17:58:08.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.685+0000 7f9a86576640 1 -- 172.21.15.111:0/1756039924 wait complete. 2024-09-13T17:58:08.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.686+0000 7f9a86576640 1 Processor -- start 2024-09-13T17:58:08.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.687+0000 7f9a86576640 1 -- start start 2024-09-13T17:58:08.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.687+0000 7f9a86576640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a800cdff0 0x7f9a80156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.687+0000 7f9a86576640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a800d2590 0x7f9a8015d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.687+0000 7f9a86576640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a8015c500 0x7f9a80157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.687+0000 7f9a86576640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a800b5260 con 0x7f9a8015c500 2024-09-13T17:58:08.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.687+0000 7f9a86576640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a800b5540 con 0x7f9a800cdff0 2024-09-13T17:58:08.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.687+0000 7f9a86576640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a80157280 con 0x7f9a800d2590 2024-09-13T17:58:08.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.687+0000 7f9a84b13640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a8015c500 0x7f9a80157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.687+0000 7f9a7ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a800cdff0 0x7f9a80156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.687+0000 7f9a84b13640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a8015c500 0x7f9a80157790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51064/0 (socket says 172.21.15.111:51064) 2024-09-13T17:58:08.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.687+0000 7f9a84b13640 1 -- 172.21.15.111:0/536233375 learned_addr learned my addr 172.21.15.111:0/536233375 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:08.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.687+0000 7f9a7f7fe640 1 --2- 172.21.15.111:0/536233375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a800d2590 0x7f9a8015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.688+0000 7f9a84b13640 1 -- 172.21.15.111:0/536233375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a800d2590 msgr2=0x7f9a8015d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.688+0000 7f9a84b13640 1 --2- 172.21.15.111:0/536233375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a800d2590 0x7f9a8015d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.688+0000 7f9a84b13640 1 -- 172.21.15.111:0/536233375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a800cdff0 msgr2=0x7f9a80156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.688+0000 7f9a84b13640 1 --2- 172.21.15.111:0/536233375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a800cdff0 0x7f9a80156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.688+0000 7f9a84b13640 1 -- 172.21.15.111:0/536233375 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a78009b30 con 0x7f9a8015c500 2024-09-13T17:58:08.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.688+0000 7f9a7ffff640 1 --2- 172.21.15.111:0/536233375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a800cdff0 0x7f9a80156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:08.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.688+0000 7f9a84b13640 1 --2- 172.21.15.111:0/536233375 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a8015c500 0x7f9a80157790 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9a78034040 tx=0x7f9a78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:08.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.688+0000 7f9a7cff9640 1 -- 172.21.15.111:0/536233375 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7803d860 con 0x7f9a8015c500 2024-09-13T17:58:08.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.688+0000 7f9a7cff9640 1 -- 172.21.15.111:0/536233375 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7803d9c0 con 0x7f9a8015c500 2024-09-13T17:58:08.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.688+0000 7f9a86576640 1 -- 172.21.15.111:0/536233375 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a801581e0 con 0x7f9a8015c500 2024-09-13T17:58:08.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.688+0000 7f9a7cff9640 1 -- 172.21.15.111:0/536233375 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7803dd70 con 0x7f9a8015c500 2024-09-13T17:58:08.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.688+0000 7f9a86576640 1 -- 172.21.15.111:0/536233375 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a801587a0 con 0x7f9a8015c500 2024-09-13T17:58:08.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.689+0000 7f9a86576640 1 -- 172.21.15.111:0/536233375 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a800d0870 con 0x7f9a8015c500 2024-09-13T17:58:08.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.689+0000 7f9a7cff9640 1 -- 172.21.15.111:0/536233375 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9a7803b400 con 0x7f9a8015c500 2024-09-13T17:58:08.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.690+0000 7f9a7cff9640 1 --2- 172.21.15.111:0/536233375 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9a6c06ac00 0x7f9a6c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.690+0000 7f9a7cff9640 1 -- 172.21.15.111:0/536233375 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9a780aa3a0 con 0x7f9a8015c500 2024-09-13T17:58:08.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.690+0000 7f9a7ffff640 1 --2- 172.21.15.111:0/536233375 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9a6c06ac00 0x7f9a6c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.690+0000 7f9a7ffff640 1 --2- 172.21.15.111:0/536233375 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9a6c06ac00 0x7f9a6c06d0c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9a800b3d40 tx=0x7f9a70009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:08.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.694+0000 7f9a7cff9640 1 -- 172.21.15.111:0/536233375 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f9a78075270 con 0x7f9a8015c500 2024-09-13T17:58:08.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.855+0000 7f9a86576640 1 -- 172.21.15.111:0/536233375 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a800b5680 con 0x7f9a8015c500 2024-09-13T17:58:08.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.856+0000 7f9a7cff9640 1 -- 172.21.15.111:0/536233375 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a78009510 con 0x7f9a8015c500 2024-09-13T17:58:08.858 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:58:08.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.857+0000 7f9a86576640 1 -- 172.21.15.111:0/536233375 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9a6c06ac00 msgr2=0x7f9a6c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.857+0000 7f9a86576640 1 --2- 172.21.15.111:0/536233375 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9a6c06ac00 0x7f9a6c06d0c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9a800b3d40 tx=0x7f9a70009190 comp rx=0 tx=0).stop 2024-09-13T17:58:08.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.857+0000 7f9a86576640 1 -- 172.21.15.111:0/536233375 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a8015c500 msgr2=0x7f9a80157790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.857+0000 7f9a86576640 1 --2- 172.21.15.111:0/536233375 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a8015c500 0x7f9a80157790 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9a78034040 tx=0x7f9a78004a40 comp rx=0 tx=0).stop 2024-09-13T17:58:08.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.857+0000 7f9a86576640 1 -- 172.21.15.111:0/536233375 shutdown_connections 2024-09-13T17:58:08.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.857+0000 7f9a86576640 1 --2- 172.21.15.111:0/536233375 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a8015c500 0x7f9a80157790 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.857+0000 7f9a86576640 1 --2- 172.21.15.111:0/536233375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a800d2590 0x7f9a8015d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.857+0000 7f9a86576640 1 --2- 172.21.15.111:0/536233375 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9a6c06ac00 0x7f9a6c06d0c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.857+0000 7f9a86576640 1 --2- 172.21.15.111:0/536233375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a800cdff0 0x7f9a80156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.857+0000 7f9a86576640 1 -- 172.21.15.111:0/536233375 >> 172.21.15.111:0/536233375 conn(0x7f9a80089f60 msgr2=0x7f9a800b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:08.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.858+0000 7f9a86576640 1 -- 172.21.15.111:0/536233375 shutdown_connections 2024-09-13T17:58:08.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.858+0000 7f9a86576640 1 -- 172.21.15.111:0/536233375 wait complete. 2024-09-13T17:58:08.870 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:58:08.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.968+0000 7f6b25976640 1 Processor -- start 2024-09-13T17:58:08.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.968+0000 7f6b25976640 1 -- start start 2024-09-13T17:58:08.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.969+0000 7f6b25976640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6b200cdff0 0x7f6b2006c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.969+0000 7f6b25976640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b200cd450 con 0x7f6b2015c500 2024-09-13T17:58:08.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.969+0000 7f6b25976640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b200d0cf0 con 0x7f6b200d2760 2024-09-13T17:58:08.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.969+0000 7f6b25976640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b200d0fd0 con 0x7f6b200cdff0 2024-09-13T17:58:08.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.969+0000 7f6b1effd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6b2015c500 0x7f6b200d2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58852/0 (socket says 172.21.15.111:58852) 2024-09-13T17:58:08.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.969+0000 7f6b1effd640 1 -- 172.21.15.111:0/1737928502 learned_addr learned my addr 172.21.15.111:0/1737928502 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:08.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1f7fe640 1 --2- 172.21.15.111:0/1737928502 >> v2:172.21.15.188:3300/0 conn(0x7f6b200cdff0 0x7f6b2006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1e7fc640 1 -- 172.21.15.111:0/1737928502 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2363259906 0 0) 0x7f6b200cd450 con 0x7f6b2015c500 2024-09-13T17:58:08.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1e7fc640 1 -- 172.21.15.111:0/1737928502 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b0c003520 con 0x7f6b2015c500 2024-09-13T17:58:08.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1e7fc640 1 -- 172.21.15.111:0/1737928502 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1888371492 0 0) 0x7f6b0c003520 con 0x7f6b2015c500 2024-09-13T17:58:08.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1e7fc640 1 -- 172.21.15.111:0/1737928502 >> v2:172.21.15.188:3300/0 conn(0x7f6b200cdff0 msgr2=0x7f6b2006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:08.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1e7fc640 1 --2- 172.21.15.111:0/1737928502 >> v2:172.21.15.188:3300/0 conn(0x7f6b200cdff0 0x7f6b2006c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1e7fc640 1 -- 172.21.15.111:0/1737928502 >> v1:172.21.15.188:6789/0 conn(0x7f6b200d2760 legacy=0x7f6b200cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1e7fc640 1 -- 172.21.15.111:0/1737928502 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b200cd0e0 con 0x7f6b2015c500 2024-09-13T17:58:08.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1e7fc640 1 -- 172.21.15.111:0/1737928502 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f6b10001670 con 0x7f6b2015c500 2024-09-13T17:58:08.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1e7fc640 1 -- 172.21.15.111:0/1737928502 >> v1:172.21.15.111:6789/0 conn(0x7f6b2015c500 legacy=0x7f6b200d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1e7fc640 1 --2- 172.21.15.111:0/1737928502 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b0c003970 0x7f6b0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1e7fc640 1 --2- 172.21.15.111:0/1737928502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b0c0047c0 0x7f6b0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1e7fc640 1 --2- 172.21.15.111:0/1737928502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b0c005180 0x7f6b0c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1e7fc640 1 -- 172.21.15.111:0/1737928502 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b200d0fd0 con 0x7f6b0c0047c0 2024-09-13T17:58:08.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1e7fc640 1 -- 172.21.15.111:0/1737928502 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c00c020 con 0x7f6b0c005180 2024-09-13T17:58:08.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1e7fc640 1 -- 172.21.15.111:0/1737928502 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c00c1e0 con 0x7f6b0c003970 2024-09-13T17:58:08.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b167fc640 1 --2- 172.21.15.111:0/1737928502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b0c0047c0 0x7f6b0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1effd640 1 --2- 172.21.15.111:0/1737928502 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b0c003970 0x7f6b0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1f7fe640 1 --2- 172.21.15.111:0/1737928502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b0c005180 0x7f6b0c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b167fc640 1 -- 172.21.15.111:0/1737928502 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b0c003970 msgr2=0x7f6b0c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:08.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b167fc640 1 --2- 172.21.15.111:0/1737928502 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b0c003970 0x7f6b0c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b167fc640 1 -- 172.21.15.111:0/1737928502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b0c005180 msgr2=0x7f6b0c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:08.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b167fc640 1 --2- 172.21.15.111:0/1737928502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b0c005180 0x7f6b0c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b167fc640 1 -- 172.21.15.111:0/1737928502 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c003520 con 0x7f6b0c0047c0 2024-09-13T17:58:08.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1effd640 1 --2- 172.21.15.111:0/1737928502 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b0c003970 0x7f6b0c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:08.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1f7fe640 1 --2- 172.21.15.111:0/1737928502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b0c005180 0x7f6b0c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:08.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.970+0000 7f6b1f7fe640 1 -- 172.21.15.111:0/1737928502 reap_dead start 2024-09-13T17:58:08.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.971+0000 7f6b167fc640 1 --2- 172.21.15.111:0/1737928502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b0c0047c0 0x7f6b0c004c40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6b08001750 tx=0x7f6b0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de70291731765bf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:08.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.971+0000 7f6b1e7fc640 1 -- 172.21.15.111:0/1737928502 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b080052d0 con 0x7f6b0c0047c0 2024-09-13T17:58:08.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.971+0000 7f6b1e7fc640 1 -- 172.21.15.111:0/1737928502 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b08005430 con 0x7f6b0c0047c0 2024-09-13T17:58:08.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.971+0000 7f6b1e7fc640 1 -- 172.21.15.111:0/1737928502 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0802abc0 con 0x7f6b0c0047c0 2024-09-13T17:58:08.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.972+0000 7f6b25976640 1 -- 172.21.15.111:0/1737928502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b0c0047c0 msgr2=0x7f6b0c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.972+0000 7f6b25976640 1 --2- 172.21.15.111:0/1737928502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b0c0047c0 0x7f6b0c004c40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6b08001750 tx=0x7f6b0c0057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.972+0000 7f6b25976640 1 -- 172.21.15.111:0/1737928502 shutdown_connections 2024-09-13T17:58:08.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.972+0000 7f6b25976640 1 --2- 172.21.15.111:0/1737928502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b0c0047c0 0x7f6b0c004c40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.972+0000 7f6b25976640 1 -- 172.21.15.111:0/1737928502 >> 172.21.15.111:0/1737928502 conn(0x7f6b20089f60 msgr2=0x7f6b2009b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:08.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.972+0000 7f6b25976640 1 -- 172.21.15.111:0/1737928502 shutdown_connections 2024-09-13T17:58:08.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.972+0000 7f6b25976640 1 -- 172.21.15.111:0/1737928502 wait complete. 2024-09-13T17:58:08.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.973+0000 7f6b25976640 1 Processor -- start 2024-09-13T17:58:08.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.973+0000 7f6b25976640 1 -- start start 2024-09-13T17:58:08.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b25976640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b0c0047c0 0x7f6b200cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b25976640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b200c9770 0x7f6b201f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b25976640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b201f5870 0x7f6b2014a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b25976640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b201f62d0 con 0x7f6b0c0047c0 2024-09-13T17:58:08.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b25976640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b200cb410 con 0x7f6b200c9770 2024-09-13T17:58:08.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b25976640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b201f5d10 con 0x7f6b201f5870 2024-09-13T17:58:08.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b1effd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b0c0047c0 0x7f6b200cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b1effd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b0c0047c0 0x7f6b200cbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51080/0 (socket says 172.21.15.111:51080) 2024-09-13T17:58:08.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b1effd640 1 -- 172.21.15.111:0/2993555495 learned_addr learned my addr 172.21.15.111:0/2993555495 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:08.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b1f7fe640 1 --2- 172.21.15.111:0/2993555495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b201f5870 0x7f6b2014a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b1effd640 1 -- 172.21.15.111:0/2993555495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b201f5870 msgr2=0x7f6b2014a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b167fc640 1 --2- 172.21.15.111:0/2993555495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b200c9770 0x7f6b201f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b1effd640 1 --2- 172.21.15.111:0/2993555495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b201f5870 0x7f6b2014a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b1effd640 1 -- 172.21.15.111:0/2993555495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b200c9770 msgr2=0x7f6b201f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:08.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b1effd640 1 --2- 172.21.15.111:0/2993555495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b200c9770 0x7f6b201f5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b1effd640 1 -- 172.21.15.111:0/2993555495 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b201f5ff0 con 0x7f6b0c0047c0 2024-09-13T17:58:08.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b1f7fe640 1 --2- 172.21.15.111:0/2993555495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b201f5870 0x7f6b2014a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:08.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b167fc640 1 --2- 172.21.15.111:0/2993555495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b200c9770 0x7f6b201f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:08.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b1effd640 1 --2- 172.21.15.111:0/2993555495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b0c0047c0 0x7f6b200cbeb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6b10000fb0 tx=0x7f6b1000eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:08.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b1cff9640 1 -- 172.21.15.111:0/2993555495 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1000c660 con 0x7f6b0c0047c0 2024-09-13T17:58:08.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b1cff9640 1 -- 172.21.15.111:0/2993555495 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1000c7c0 con 0x7f6b0c0047c0 2024-09-13T17:58:08.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b1cff9640 1 -- 172.21.15.111:0/2993555495 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1000cb50 con 0x7f6b0c0047c0 2024-09-13T17:58:08.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.974+0000 7f6b25976640 1 -- 172.21.15.111:0/2993555495 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b200cb6b0 con 0x7f6b0c0047c0 2024-09-13T17:58:08.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.975+0000 7f6b25976640 1 -- 172.21.15.111:0/2993555495 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2014d180 con 0x7f6b0c0047c0 2024-09-13T17:58:08.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.975+0000 7f6b25976640 1 -- 172.21.15.111:0/2993555495 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2014d340 con 0x7f6b0c0047c0 2024-09-13T17:58:08.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.976+0000 7f6b1cff9640 1 -- 172.21.15.111:0/2993555495 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6b1000ccb0 con 0x7f6b0c0047c0 2024-09-13T17:58:08.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.976+0000 7f6b1cff9640 1 --2- 172.21.15.111:0/2993555495 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b0006ac00 0x7f6b0006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.976+0000 7f6b1cff9640 1 -- 172.21.15.111:0/2993555495 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6b100895e0 con 0x7f6b0c0047c0 2024-09-13T17:58:08.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.976+0000 7f6b167fc640 1 --2- 172.21.15.111:0/2993555495 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b0006ac00 0x7f6b0006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.977+0000 7f6b167fc640 1 --2- 172.21.15.111:0/2993555495 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b0006ac00 0x7f6b0006d0c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6b08032000 tx=0x7f6b0802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:08.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:08.980+0000 7f6b1cff9640 1 -- 172.21.15.111:0/2993555495 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f6b10054400 con 0x7f6b0c0047c0 2024-09-13T17:58:09.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.143+0000 7f6b25976640 1 -- 172.21.15.111:0/2993555495 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6b200cdd90 con 0x7f6b0c0047c0 2024-09-13T17:58:09.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.143+0000 7f6b1cff9640 1 -- 172.21.15.111:0/2993555495 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6b10057a20 con 0x7f6b0c0047c0 2024-09-13T17:58:09.146 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.145+0000 7f6b25976640 1 -- 172.21.15.111:0/2993555495 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b0006ac00 msgr2=0x7f6b0006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.145+0000 7f6b25976640 1 --2- 172.21.15.111:0/2993555495 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b0006ac00 0x7f6b0006d0c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6b08032000 tx=0x7f6b0802e000 comp rx=0 tx=0).stop 2024-09-13T17:58:09.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.145+0000 7f6b25976640 1 -- 172.21.15.111:0/2993555495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b0c0047c0 msgr2=0x7f6b200cbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.145+0000 7f6b25976640 1 --2- 172.21.15.111:0/2993555495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b0c0047c0 0x7f6b200cbeb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6b10000fb0 tx=0x7f6b1000eba0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.145+0000 7f6b25976640 1 -- 172.21.15.111:0/2993555495 shutdown_connections 2024-09-13T17:58:09.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.145+0000 7f6b25976640 1 --2- 172.21.15.111:0/2993555495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b201f5870 0x7f6b2014a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.145+0000 7f6b25976640 1 --2- 172.21.15.111:0/2993555495 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b0006ac00 0x7f6b0006d0c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.145+0000 7f6b25976640 1 --2- 172.21.15.111:0/2993555495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b200c9770 0x7f6b201f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.145+0000 7f6b25976640 1 --2- 172.21.15.111:0/2993555495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b0c0047c0 0x7f6b200cbeb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.145+0000 7f6b25976640 1 -- 172.21.15.111:0/2993555495 >> 172.21.15.111:0/2993555495 conn(0x7f6b20089f60 msgr2=0x7f6b200b1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:09.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.146+0000 7f6b25976640 1 -- 172.21.15.111:0/2993555495 shutdown_connections 2024-09-13T17:58:09.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.146+0000 7f6b25976640 1 -- 172.21.15.111:0/2993555495 wait complete. 2024-09-13T17:58:09.158 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:58:09.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.256+0000 7fac7e480640 1 Processor -- start 2024-09-13T17:58:09.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.256+0000 7fac7e480640 1 -- start start 2024-09-13T17:58:09.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.257+0000 7fac7e480640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fac7815a3b0 0x7fac780b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.257+0000 7fac7e480640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac780cec00 0x7fac780ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.257+0000 7fac7e480640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac7815a790 con 0x7fac780b1720 2024-09-13T17:58:09.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.257+0000 7fac7e480640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac780b2680 con 0x7fac7815a3b0 2024-09-13T17:58:09.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.257+0000 7fac7e480640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac780b2960 con 0x7fac780cec00 2024-09-13T17:58:09.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.257+0000 7fac77fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fac7815a3b0 0x7fac780b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.257+0000 7fac77fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fac7815a3b0 0x7fac780b20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51092/0 (socket says 172.21.15.111:51092) 2024-09-13T17:58:09.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.257+0000 7fac77fff640 1 -- 172.21.15.111:0/1727437864 learned_addr learned my addr 172.21.15.111:0/1727437864 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:09.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.258+0000 7fac7ca1d640 1 --2- 172.21.15.111:0/1727437864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac780cec00 0x7fac780ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.258+0000 7fac77fff640 1 -- 172.21.15.111:0/1727437864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac780cec00 msgr2=0x7fac780ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:09.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.258+0000 7fac77fff640 1 --2- 172.21.15.111:0/1727437864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac780cec00 0x7fac780ab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.258+0000 7fac77fff640 1 -- 172.21.15.111:0/1727437864 >> v1:172.21.15.188:6789/0 conn(0x7fac780b1720 legacy=0x7fac780ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.258+0000 7fac77fff640 1 -- 172.21.15.111:0/1727437864 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac780b2c40 con 0x7fac7815a3b0 2024-09-13T17:58:09.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.258+0000 7fac77fff640 1 --2- 172.21.15.111:0/1727437864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac7815a3b0 0x7fac780b20a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fac68009870 tx=0x7fac6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc53530c1398357b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:09.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.259+0000 7fac777fe640 1 -- 172.21.15.111:0/1727437864 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac68032440 con 0x7fac7815a3b0 2024-09-13T17:58:09.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.259+0000 7fac777fe640 1 -- 172.21.15.111:0/1727437864 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac680325a0 con 0x7fac7815a3b0 2024-09-13T17:58:09.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.259+0000 7fac777fe640 1 -- 172.21.15.111:0/1727437864 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac68032930 con 0x7fac7815a3b0 2024-09-13T17:58:09.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.259+0000 7fac7e480640 1 -- 172.21.15.111:0/1727437864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac7815a3b0 msgr2=0x7fac780b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.259+0000 7fac7e480640 1 --2- 172.21.15.111:0/1727437864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac7815a3b0 0x7fac780b20a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fac68009870 tx=0x7fac6802a270 comp rx=0 tx=0).stop 2024-09-13T17:58:09.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.259+0000 7fac7e480640 1 -- 172.21.15.111:0/1727437864 shutdown_connections 2024-09-13T17:58:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.259+0000 7fac7e480640 1 --2- 172.21.15.111:0/1727437864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac780cec00 0x7fac780ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.259+0000 7fac7e480640 1 --2- 172.21.15.111:0/1727437864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac7815a3b0 0x7fac780b20a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.259+0000 7fac7e480640 1 -- 172.21.15.111:0/1727437864 >> 172.21.15.111:0/1727437864 conn(0x7fac7806a0b0 msgr2=0x7fac7806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.259+0000 7fac7e480640 1 -- 172.21.15.111:0/1727437864 shutdown_connections 2024-09-13T17:58:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.259+0000 7fac7e480640 1 -- 172.21.15.111:0/1727437864 wait complete. 2024-09-13T17:58:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.260+0000 7fac7e480640 1 Processor -- start 2024-09-13T17:58:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.260+0000 7fac7e480640 1 -- start start 2024-09-13T17:58:09.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.261+0000 7fac7e480640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac780b1720 0x7fac781eff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.261+0000 7fac7e480640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac780cec00 0x7fac7815bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.261+0000 7fac7e480640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac7815a3b0 0x7fac781581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.261+0000 7fac7e480640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac781f04b0 con 0x7fac780b1720 2024-09-13T17:58:09.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.261+0000 7fac7e480640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac780c8620 con 0x7fac780cec00 2024-09-13T17:58:09.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.261+0000 7fac7e480640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac780af380 con 0x7fac7815a3b0 2024-09-13T17:58:09.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.261+0000 7fac77fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac780b1720 0x7fac781eff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.261+0000 7fac77fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac780b1720 0x7fac781eff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51094/0 (socket says 172.21.15.111:51094) 2024-09-13T17:58:09.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.261+0000 7fac77fff640 1 -- 172.21.15.111:0/523055655 learned_addr learned my addr 172.21.15.111:0/523055655 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:09.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.261+0000 7fac7ca1d640 1 --2- 172.21.15.111:0/523055655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac7815a3b0 0x7fac781581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.261+0000 7fac6ffff640 1 --2- 172.21.15.111:0/523055655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac780cec00 0x7fac7815bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.261+0000 7fac77fff640 1 -- 172.21.15.111:0/523055655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac7815a3b0 msgr2=0x7fac781581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.262+0000 7fac77fff640 1 --2- 172.21.15.111:0/523055655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac7815a3b0 0x7fac781581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.262+0000 7fac77fff640 1 -- 172.21.15.111:0/523055655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac780cec00 msgr2=0x7fac7815bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.262+0000 7fac77fff640 1 --2- 172.21.15.111:0/523055655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac780cec00 0x7fac7815bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.262+0000 7fac77fff640 1 -- 172.21.15.111:0/523055655 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac680094d0 con 0x7fac780b1720 2024-09-13T17:58:09.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.262+0000 7fac7ca1d640 1 --2- 172.21.15.111:0/523055655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac7815a3b0 0x7fac781581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:09.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.262+0000 7fac77fff640 1 --2- 172.21.15.111:0/523055655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac780b1720 0x7fac781eff70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fac68000c00 tx=0x7fac680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:09.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.262+0000 7fac757fa640 1 -- 172.21.15.111:0/523055655 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac68032440 con 0x7fac780b1720 2024-09-13T17:58:09.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.262+0000 7fac757fa640 1 -- 172.21.15.111:0/523055655 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac68030b80 con 0x7fac780b1720 2024-09-13T17:58:09.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.262+0000 7fac7e480640 1 -- 172.21.15.111:0/523055655 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac780af660 con 0x7fac780b1720 2024-09-13T17:58:09.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.262+0000 7fac757fa640 1 -- 172.21.15.111:0/523055655 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6803d440 con 0x7fac780b1720 2024-09-13T17:58:09.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.262+0000 7fac7e480640 1 -- 172.21.15.111:0/523055655 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7815c190 con 0x7fac780b1720 2024-09-13T17:58:09.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.263+0000 7fac7e480640 1 -- 172.21.15.111:0/523055655 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac7815c3e0 con 0x7fac780b1720 2024-09-13T17:58:09.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.263+0000 7fac757fa640 1 -- 172.21.15.111:0/523055655 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fac68039040 con 0x7fac780b1720 2024-09-13T17:58:09.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.264+0000 7fac757fa640 1 --2- 172.21.15.111:0/523055655 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fac5806ac00 0x7fac5806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.264+0000 7fac757fa640 1 -- 172.21.15.111:0/523055655 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fac680aa1f0 con 0x7fac780b1720 2024-09-13T17:58:09.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.264+0000 7fac6ffff640 1 --2- 172.21.15.111:0/523055655 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fac5806ac00 0x7fac5806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.264+0000 7fac6ffff640 1 --2- 172.21.15.111:0/523055655 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fac5806ac00 0x7fac5806d0c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fac60007fd0 tx=0x7fac60009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:09.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.268+0000 7fac757fa640 1 -- 172.21.15.111:0/523055655 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fac680750a0 con 0x7fac780b1720 2024-09-13T17:58:09.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.429+0000 7fac7e480640 1 -- 172.21.15.111:0/523055655 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac780af1e0 con 0x7fac780b1720 2024-09-13T17:58:09.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.430+0000 7fac757fa640 1 -- 172.21.15.111:0/523055655 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fac780af1e0 con 0x7fac780b1720 2024-09-13T17:58:09.432 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:58:09.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.431+0000 7fac7e480640 1 -- 172.21.15.111:0/523055655 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fac5806ac00 msgr2=0x7fac5806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.431+0000 7fac7e480640 1 --2- 172.21.15.111:0/523055655 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fac5806ac00 0x7fac5806d0c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fac60007fd0 tx=0x7fac60009040 comp rx=0 tx=0).stop 2024-09-13T17:58:09.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.431+0000 7fac7e480640 1 -- 172.21.15.111:0/523055655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac780b1720 msgr2=0x7fac781eff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.431+0000 7fac7e480640 1 --2- 172.21.15.111:0/523055655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac780b1720 0x7fac781eff70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fac68000c00 tx=0x7fac680046c0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.431+0000 7fac7e480640 1 -- 172.21.15.111:0/523055655 shutdown_connections 2024-09-13T17:58:09.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.431+0000 7fac7e480640 1 --2- 172.21.15.111:0/523055655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac7815a3b0 0x7fac781581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.431+0000 7fac7e480640 1 --2- 172.21.15.111:0/523055655 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fac5806ac00 0x7fac5806d0c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.431+0000 7fac7e480640 1 --2- 172.21.15.111:0/523055655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac780cec00 0x7fac7815bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.431+0000 7fac7e480640 1 --2- 172.21.15.111:0/523055655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac780b1720 0x7fac781eff70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.431+0000 7fac7e480640 1 -- 172.21.15.111:0/523055655 >> 172.21.15.111:0/523055655 conn(0x7fac7806a0b0 msgr2=0x7fac780ac5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:09.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.432+0000 7fac7e480640 1 -- 172.21.15.111:0/523055655 shutdown_connections 2024-09-13T17:58:09.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.432+0000 7fac7e480640 1 -- 172.21.15.111:0/523055655 wait complete. 2024-09-13T17:58:09.444 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:58:09.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.543+0000 7fc3ef59d640 1 Processor -- start 2024-09-13T17:58:09.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.543+0000 7fc3ef59d640 1 -- start start 2024-09-13T17:58:09.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.544+0000 7fc3ef59d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc3e815d1e0 0x7fc3e80cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.544+0000 7fc3ef59d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc3e80ccf20 0x7fc3e8070600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.544+0000 7fc3ef59d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e80cd3b0 con 0x7fc3e813be90 2024-09-13T17:58:09.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.544+0000 7fc3ef59d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3e813c5b0 con 0x7fc3e815d1e0 2024-09-13T17:58:09.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.544+0000 7fc3ef59d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3e806fe20 con 0x7fc3e80ccf20 2024-09-13T17:58:09.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.544+0000 7fc3edd9a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc3e815d1e0 0x7fc3e80cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.544+0000 7fc3edd9a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc3e815d1e0 0x7fc3e80cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51098/0 (socket says 172.21.15.111:51098) 2024-09-13T17:58:09.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.544+0000 7fc3edd9a640 1 -- 172.21.15.111:0/614373455 learned_addr learned my addr 172.21.15.111:0/614373455 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:09.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.544+0000 7fc3eed9c640 1 --2- 172.21.15.111:0/614373455 >> v2:172.21.15.188:3300/0 conn(0x7fc3e80ccf20 0x7fc3e8070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.544+0000 7fc3edd9a640 1 -- 172.21.15.111:0/614373455 >> v2:172.21.15.188:3300/0 conn(0x7fc3e80ccf20 msgr2=0x7fc3e8070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:09.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.544+0000 7fc3edd9a640 1 --2- 172.21.15.111:0/614373455 >> v2:172.21.15.188:3300/0 conn(0x7fc3e80ccf20 0x7fc3e8070600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.544+0000 7fc3edd9a640 1 -- 172.21.15.111:0/614373455 >> v1:172.21.15.188:6789/0 conn(0x7fc3e813be90 legacy=0x7fc3e815cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.544+0000 7fc3edd9a640 1 -- 172.21.15.111:0/614373455 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e813c270 con 0x7fc3e815d1e0 2024-09-13T17:58:09.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.545+0000 7fc3edd9a640 1 --2- 172.21.15.111:0/614373455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3e815d1e0 0x7fc3e80cc9e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8071ca0 tx=0x7fc3d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cda2e303cb5f3d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:09.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.545+0000 7fc3ed599640 1 -- 172.21.15.111:0/614373455 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8032440 con 0x7fc3e815d1e0 2024-09-13T17:58:09.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.545+0000 7fc3ed599640 1 -- 172.21.15.111:0/614373455 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d80325a0 con 0x7fc3e815d1e0 2024-09-13T17:58:09.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.545+0000 7fc3ed599640 1 -- 172.21.15.111:0/614373455 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8032930 con 0x7fc3e815d1e0 2024-09-13T17:58:09.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.546+0000 7fc3ef59d640 1 -- 172.21.15.111:0/614373455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3e815d1e0 msgr2=0x7fc3e80cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.546+0000 7fc3ef59d640 1 --2- 172.21.15.111:0/614373455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3e815d1e0 0x7fc3e80cc9e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8071ca0 tx=0x7fc3d802a270 comp rx=0 tx=0).stop 2024-09-13T17:58:09.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.546+0000 7fc3ef59d640 1 -- 172.21.15.111:0/614373455 shutdown_connections 2024-09-13T17:58:09.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.546+0000 7fc3ef59d640 1 --2- 172.21.15.111:0/614373455 >> v2:172.21.15.188:3300/0 conn(0x7fc3e80ccf20 0x7fc3e8070600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.546+0000 7fc3ef59d640 1 --2- 172.21.15.111:0/614373455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3e815d1e0 0x7fc3e80cc9e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.546+0000 7fc3ef59d640 1 -- 172.21.15.111:0/614373455 >> 172.21.15.111:0/614373455 conn(0x7fc3e806d650 msgr2=0x7fc3e8069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:09.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.546+0000 7fc3ef59d640 1 -- 172.21.15.111:0/614373455 shutdown_connections 2024-09-13T17:58:09.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.546+0000 7fc3ef59d640 1 -- 172.21.15.111:0/614373455 wait complete. 2024-09-13T17:58:09.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.547+0000 7fc3ef59d640 1 Processor -- start 2024-09-13T17:58:09.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.547+0000 7fc3ef59d640 1 -- start start 2024-09-13T17:58:09.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3ef59d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3e80ccf20 0x7fc3e814f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3ef59d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e813be90 0x7fc3e80af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3ef59d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3e815d1e0 0x7fc3e80af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3ef59d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e814f7e0 con 0x7fc3e815d1e0 2024-09-13T17:58:09.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3ef59d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e81f2ab0 con 0x7fc3e813be90 2024-09-13T17:58:09.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3ef59d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e80b1590 con 0x7fc3e80ccf20 2024-09-13T17:58:09.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3eed9c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3e815d1e0 0x7fc3e80af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3eed9c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3e815d1e0 0x7fc3e80af920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51100/0 (socket says 172.21.15.111:51100) 2024-09-13T17:58:09.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3eed9c640 1 -- 172.21.15.111:0/3988516080 learned_addr learned my addr 172.21.15.111:0/3988516080 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:09.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3edd9a640 1 --2- 172.21.15.111:0/3988516080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e813be90 0x7fc3e80af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3ee59b640 1 --2- 172.21.15.111:0/3988516080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3e80ccf20 0x7fc3e814f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3eed9c640 1 -- 172.21.15.111:0/3988516080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3e80ccf20 msgr2=0x7fc3e814f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3eed9c640 1 --2- 172.21.15.111:0/3988516080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3e80ccf20 0x7fc3e814f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3eed9c640 1 -- 172.21.15.111:0/3988516080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e813be90 msgr2=0x7fc3e80af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3eed9c640 1 --2- 172.21.15.111:0/3988516080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e813be90 0x7fc3e80af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3eed9c640 1 -- 172.21.15.111:0/3988516080 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d80094d0 con 0x7fc3e815d1e0 2024-09-13T17:58:09.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3ee59b640 1 --2- 172.21.15.111:0/3988516080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3e80ccf20 0x7fc3e814f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:09.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3eed9c640 1 --2- 172.21.15.111:0/3988516080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3e815d1e0 0x7fc3e80af920 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000d800 tx=0x7fc3e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:09.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3deffd640 1 -- 172.21.15.111:0/3988516080 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e000b6c0 con 0x7fc3e815d1e0 2024-09-13T17:58:09.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3deffd640 1 -- 172.21.15.111:0/3988516080 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e000b840 con 0x7fc3e815d1e0 2024-09-13T17:58:09.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3deffd640 1 -- 172.21.15.111:0/3988516080 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e000bbf0 con 0x7fc3e815d1e0 2024-09-13T17:58:09.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3ef59d640 1 -- 172.21.15.111:0/3988516080 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e80affa0 con 0x7fc3e815d1e0 2024-09-13T17:58:09.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.548+0000 7fc3ef59d640 1 -- 172.21.15.111:0/3988516080 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e80b1b10 con 0x7fc3e815d1e0 2024-09-13T17:58:09.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.549+0000 7fc3ef59d640 1 -- 172.21.15.111:0/3988516080 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e80700e0 con 0x7fc3e815d1e0 2024-09-13T17:58:09.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.550+0000 7fc3deffd640 1 -- 172.21.15.111:0/3988516080 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc3e000bd50 con 0x7fc3e815d1e0 2024-09-13T17:58:09.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.550+0000 7fc3deffd640 1 --2- 172.21.15.111:0/3988516080 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc3cc06ac00 0x7fc3cc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.551+0000 7fc3deffd640 1 -- 172.21.15.111:0/3988516080 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc3e0089b90 con 0x7fc3e815d1e0 2024-09-13T17:58:09.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.551+0000 7fc3ee59b640 1 --2- 172.21.15.111:0/3988516080 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc3cc06ac00 0x7fc3cc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.551+0000 7fc3ee59b640 1 --2- 172.21.15.111:0/3988516080 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc3cc06ac00 0x7fc3cc06d0c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40012e0 tx=0x7fc3e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:09.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.555+0000 7fc3deffd640 1 -- 172.21.15.111:0/3988516080 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc3e00549b0 con 0x7fc3e815d1e0 2024-09-13T17:58:09.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.715+0000 7fc3ef59d640 1 -- 172.21.15.111:0/3988516080 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc3e8071120 con 0x7fc3e815d1e0 2024-09-13T17:58:09.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.716+0000 7fc3deffd640 1 -- 172.21.15.111:0/3988516080 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc3e0057fd0 con 0x7fc3e815d1e0 2024-09-13T17:58:09.718 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:09.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.718+0000 7fc3ef59d640 1 -- 172.21.15.111:0/3988516080 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc3cc06ac00 msgr2=0x7fc3cc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.718+0000 7fc3ef59d640 1 --2- 172.21.15.111:0/3988516080 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc3cc06ac00 0x7fc3cc06d0c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40012e0 tx=0x7fc3e4009040 comp rx=0 tx=0).stop 2024-09-13T17:58:09.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.718+0000 7fc3ef59d640 1 -- 172.21.15.111:0/3988516080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3e815d1e0 msgr2=0x7fc3e80af920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.718+0000 7fc3ef59d640 1 --2- 172.21.15.111:0/3988516080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3e815d1e0 0x7fc3e80af920 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000d800 tx=0x7fc3e000dcc0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.718+0000 7fc3ef59d640 1 -- 172.21.15.111:0/3988516080 shutdown_connections 2024-09-13T17:58:09.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.718+0000 7fc3ef59d640 1 --2- 172.21.15.111:0/3988516080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3e815d1e0 0x7fc3e80af920 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.718+0000 7fc3ef59d640 1 --2- 172.21.15.111:0/3988516080 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc3cc06ac00 0x7fc3cc06d0c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.718+0000 7fc3ef59d640 1 --2- 172.21.15.111:0/3988516080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e813be90 0x7fc3e80af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.718+0000 7fc3ef59d640 1 --2- 172.21.15.111:0/3988516080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3e80ccf20 0x7fc3e814f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.718+0000 7fc3ef59d640 1 -- 172.21.15.111:0/3988516080 >> 172.21.15.111:0/3988516080 conn(0x7fc3e806d650 msgr2=0x7fc3e815c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:09.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.718+0000 7fc3ef59d640 1 -- 172.21.15.111:0/3988516080 shutdown_connections 2024-09-13T17:58:09.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.718+0000 7fc3ef59d640 1 -- 172.21.15.111:0/3988516080 wait complete. 2024-09-13T17:58:09.731 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:09.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.828+0000 7f5c3ebae640 1 Processor -- start 2024-09-13T17:58:09.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.828+0000 7f5c3ebae640 1 -- start start 2024-09-13T17:58:09.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.828+0000 7f5c3ebae640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5c380b2590 0x7f5c380ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.828+0000 7f5c3ebae640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c3806d3a0 0x7f5c3806c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.828+0000 7f5c3ebae640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c380b2170 con 0x7f5c3806ca00 2024-09-13T17:58:09.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.828+0000 7f5c3ebae640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c3806deb0 con 0x7f5c380b2590 2024-09-13T17:58:09.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.828+0000 7f5c3ebae640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3806e0c0 con 0x7f5c3806d3a0 2024-09-13T17:58:09.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.829+0000 7f5c3c94a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5c380b2590 0x7f5c380ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.829+0000 7f5c37fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c3806d3a0 0x7f5c3806c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.829+0000 7f5c3d14b640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f5c3806ca00 0x7f5c380cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:37648/0 (socket says 172.21.15.111:37648) 2024-09-13T17:58:09.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.829+0000 7f5c3c94a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5c380b2590 0x7f5c380ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:46540/0 (socket says 172.21.15.111:46540) 2024-09-13T17:58:09.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.829+0000 7f5c3d14b640 1 -- 172.21.15.111:0/1802863506 learned_addr learned my addr 172.21.15.111:0/1802863506 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:09.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.829+0000 7f5c37fff640 1 --2- 172.21.15.111:0/1802863506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c3806d3a0 0x7f5c3806c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:35272/0 (socket says 172.21.15.111:35272) 2024-09-13T17:58:09.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.829+0000 7f5c3c94a640 1 -- 172.21.15.111:0/1802863506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c3806d3a0 msgr2=0x7f5c3806c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:09.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.829+0000 7f5c3c94a640 1 --2- 172.21.15.111:0/1802863506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c3806d3a0 0x7f5c3806c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.829+0000 7f5c3c94a640 1 -- 172.21.15.111:0/1802863506 >> v1:172.21.15.188:6789/0 conn(0x7f5c3806ca00 legacy=0x7f5c380cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.829+0000 7f5c377fe640 1 -- 172.21.15.111:0/1802863506 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1709750165 0 0) 0x7f5c380b2170 con 0x7f5c3806ca00 2024-09-13T17:58:09.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.829+0000 7f5c3c94a640 1 -- 172.21.15.111:0/1802863506 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3806e3a0 con 0x7f5c380b2590 2024-09-13T17:58:09.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.830+0000 7f5c3c94a640 1 --2- 172.21.15.111:0/1802863506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c380b2590 0x7f5c380ce2d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5c200097a0 tx=0x7f5c2002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4343395cd7f7165 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:09.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.830+0000 7f5c377fe640 1 -- 172.21.15.111:0/1802863506 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2002ae50 con 0x7f5c380b2590 2024-09-13T17:58:09.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.830+0000 7f5c377fe640 1 -- 172.21.15.111:0/1802863506 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c20032440 con 0x7f5c380b2590 2024-09-13T17:58:09.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.830+0000 7f5c377fe640 1 -- 172.21.15.111:0/1802863506 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c200327d0 con 0x7f5c380b2590 2024-09-13T17:58:09.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.831+0000 7f5c3ebae640 1 -- 172.21.15.111:0/1802863506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c380b2590 msgr2=0x7f5c380ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.831+0000 7f5c3ebae640 1 --2- 172.21.15.111:0/1802863506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c380b2590 0x7f5c380ce2d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5c200097a0 tx=0x7f5c2002a100 comp rx=0 tx=0).stop 2024-09-13T17:58:09.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.831+0000 7f5c3ebae640 1 -- 172.21.15.111:0/1802863506 shutdown_connections 2024-09-13T17:58:09.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.831+0000 7f5c3ebae640 1 --2- 172.21.15.111:0/1802863506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c3806d3a0 0x7f5c3806c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.831+0000 7f5c3ebae640 1 --2- 172.21.15.111:0/1802863506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c380b2590 0x7f5c380ce2d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.831+0000 7f5c3ebae640 1 -- 172.21.15.111:0/1802863506 >> 172.21.15.111:0/1802863506 conn(0x7f5c38069ed0 msgr2=0x7f5c3806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:09.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.831+0000 7f5c3ebae640 1 -- 172.21.15.111:0/1802863506 shutdown_connections 2024-09-13T17:58:09.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.831+0000 7f5c3ebae640 1 -- 172.21.15.111:0/1802863506 wait complete. 2024-09-13T17:58:09.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.832+0000 7f5c3ebae640 1 Processor -- start 2024-09-13T17:58:09.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.832+0000 7f5c3ebae640 1 -- start start 2024-09-13T17:58:09.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c3ebae640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c3806ca00 0x7f5c381573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c3ebae640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c3806d3a0 0x7f5c380c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c3ebae640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c380b2590 0x7f5c380b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c3ebae640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c381eee10 con 0x7f5c380b2590 2024-09-13T17:58:09.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c3ebae640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380c6dd0 con 0x7f5c3806ca00 2024-09-13T17:58:09.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c3ebae640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380c70b0 con 0x7f5c3806d3a0 2024-09-13T17:58:09.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c3d14b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c380b2590 0x7f5c380b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c3d14b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c380b2590 0x7f5c380b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51104/0 (socket says 172.21.15.111:51104) 2024-09-13T17:58:09.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c3d14b640 1 -- 172.21.15.111:0/76364448 learned_addr learned my addr 172.21.15.111:0/76364448 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:09.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c37fff640 1 --2- 172.21.15.111:0/76364448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c3806d3a0 0x7f5c380c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c3c94a640 1 --2- 172.21.15.111:0/76364448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c3806ca00 0x7f5c381573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c3d14b640 1 -- 172.21.15.111:0/76364448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c3806d3a0 msgr2=0x7f5c380c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c3d14b640 1 --2- 172.21.15.111:0/76364448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c3806d3a0 0x7f5c380c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c3d14b640 1 -- 172.21.15.111:0/76364448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c3806ca00 msgr2=0x7f5c381573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:09.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c3d14b640 1 --2- 172.21.15.111:0/76364448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c3806ca00 0x7f5c381573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:09.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c3d14b640 1 -- 172.21.15.111:0/76364448 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c20009400 con 0x7f5c380b2590 2024-09-13T17:58:09.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c3d14b640 1 --2- 172.21.15.111:0/76364448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c380b2590 0x7f5c380b1160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000c3e0 tx=0x7f5c3000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:09.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c357fa640 1 -- 172.21.15.111:0/76364448 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c300074e0 con 0x7f5c380b2590 2024-09-13T17:58:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c357fa640 1 -- 172.21.15.111:0/76364448 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c30007640 con 0x7f5c380b2590 2024-09-13T17:58:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c357fa640 1 -- 172.21.15.111:0/76364448 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30004a20 con 0x7f5c380b2590 2024-09-13T17:58:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.833+0000 7f5c3ebae640 1 -- 172.21.15.111:0/76364448 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c380c7390 con 0x7f5c380b2590 2024-09-13T17:58:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.834+0000 7f5c3ebae640 1 -- 172.21.15.111:0/76364448 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c380c78d0 con 0x7f5c380b2590 2024-09-13T17:58:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.834+0000 7f5c3ebae640 1 -- 172.21.15.111:0/76364448 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c381eec60 con 0x7f5c380b2590 2024-09-13T17:58:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.835+0000 7f5c357fa640 1 -- 172.21.15.111:0/76364448 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5c30004b80 con 0x7f5c380b2590 2024-09-13T17:58:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.835+0000 7f5c357fa640 1 --2- 172.21.15.111:0/76364448 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5c1806ac30 0x7f5c1806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.835+0000 7f5c357fa640 1 -- 172.21.15.111:0/76364448 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5c30089230 con 0x7f5c380b2590 2024-09-13T17:58:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.835+0000 7f5c3c94a640 1 --2- 172.21.15.111:0/76364448 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5c1806ac30 0x7f5c1806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.836+0000 7f5c3c94a640 1 --2- 172.21.15.111:0/76364448 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5c1806ac30 0x7f5c1806d0f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5c20034000 tx=0x7f5c2002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:09.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:09.840+0000 7f5c357fa640 1 -- 172.21.15.111:0/76364448 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f5c30054190 con 0x7f5c380b2590 2024-09-13T17:58:10.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.000+0000 7f5c3ebae640 1 -- 172.21.15.111:0/76364448 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c38066f30 con 0x7f5c380b2590 2024-09-13T17:58:10.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.001+0000 7f5c357fa640 1 -- 172.21.15.111:0/76364448 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18794 (secure 0 0 0) 0x7f5c300577b0 con 0x7f5c380b2590 2024-09-13T17:58:10.004 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:10.004 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":26,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:07.749113+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:10.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.003+0000 7f5c3ebae640 1 -- 172.21.15.111:0/76364448 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5c1806ac30 msgr2=0x7f5c1806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.003+0000 7f5c3ebae640 1 --2- 172.21.15.111:0/76364448 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5c1806ac30 0x7f5c1806d0f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5c20034000 tx=0x7f5c2002e000 comp rx=0 tx=0).stop 2024-09-13T17:58:10.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.003+0000 7f5c3ebae640 1 -- 172.21.15.111:0/76364448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c380b2590 msgr2=0x7f5c380b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.003+0000 7f5c3ebae640 1 --2- 172.21.15.111:0/76364448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c380b2590 0x7f5c380b1160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000c3e0 tx=0x7f5c3000c8a0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.004+0000 7f5c3ebae640 1 -- 172.21.15.111:0/76364448 shutdown_connections 2024-09-13T17:58:10.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.004+0000 7f5c3ebae640 1 --2- 172.21.15.111:0/76364448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c380b2590 0x7f5c380b1160 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.004+0000 7f5c3ebae640 1 --2- 172.21.15.111:0/76364448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c3806d3a0 0x7f5c380c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.004+0000 7f5c3ebae640 1 --2- 172.21.15.111:0/76364448 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5c1806ac30 0x7f5c1806d0f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.004+0000 7f5c3ebae640 1 --2- 172.21.15.111:0/76364448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c3806ca00 0x7f5c381573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.004+0000 7f5c3ebae640 1 -- 172.21.15.111:0/76364448 >> 172.21.15.111:0/76364448 conn(0x7f5c38069ed0 msgr2=0x7f5c380ad150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:10.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.004+0000 7f5c3ebae640 1 -- 172.21.15.111:0/76364448 shutdown_connections 2024-09-13T17:58:10.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.004+0000 7f5c3ebae640 1 -- 172.21.15.111:0/76364448 wait complete. 2024-09-13T17:58:10.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:58:10.017 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:58:10.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.114+0000 7f325f183640 1 Processor -- start 2024-09-13T17:58:10.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.115+0000 7f325f183640 1 -- start start 2024-09-13T17:58:10.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.115+0000 7f325f183640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32580d2760 0x7f32580cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.115+0000 7f325f183640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f32580cde20 0x7f32580d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.115+0000 7f325f183640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32580cd450 con 0x7f325815c500 2024-09-13T17:58:10.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.115+0000 7f325f183640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32580d0cf0 con 0x7f32580cde20 2024-09-13T17:58:10.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.115+0000 7f325f183640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32580d0fd0 con 0x7f32580d2760 2024-09-13T17:58:10.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f325cf1f640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f325815c500 0x7f32580d2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58866/0 (socket says 172.21.15.111:58866) 2024-09-13T17:58:10.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f325cf1f640 1 -- 172.21.15.111:0/466812685 learned_addr learned my addr 172.21.15.111:0/466812685 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:10.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324ffff640 1 --2- 172.21.15.111:0/466812685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32580d2760 0x7f32580cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f325d720640 1 --2- 172.21.15.111:0/466812685 >> v2:172.21.15.188:3300/0 conn(0x7f32580cde20 0x7f32580d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324f7fe640 1 -- 172.21.15.111:0/466812685 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3677780156 0 0) 0x7f32580cd450 con 0x7f325815c500 2024-09-13T17:58:10.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324f7fe640 1 -- 172.21.15.111:0/466812685 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3240003520 con 0x7f325815c500 2024-09-13T17:58:10.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324f7fe640 1 -- 172.21.15.111:0/466812685 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1000865842 0 0) 0x7f3240003520 con 0x7f325815c500 2024-09-13T17:58:10.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324f7fe640 1 -- 172.21.15.111:0/466812685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32580d2760 msgr2=0x7f32580cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:10.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324f7fe640 1 --2- 172.21.15.111:0/466812685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32580d2760 0x7f32580cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324f7fe640 1 -- 172.21.15.111:0/466812685 >> v2:172.21.15.188:3300/0 conn(0x7f32580cde20 msgr2=0x7f32580d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:10.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324f7fe640 1 --2- 172.21.15.111:0/466812685 >> v2:172.21.15.188:3300/0 conn(0x7f32580cde20 0x7f32580d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324f7fe640 1 -- 172.21.15.111:0/466812685 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32580cd0e0 con 0x7f325815c500 2024-09-13T17:58:10.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324f7fe640 1 -- 172.21.15.111:0/466812685 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f3254001670 con 0x7f325815c500 2024-09-13T17:58:10.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324f7fe640 1 -- 172.21.15.111:0/466812685 >> v1:172.21.15.111:6789/0 conn(0x7f325815c500 legacy=0x7f32580d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324f7fe640 1 --2- 172.21.15.111:0/466812685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3240003970 0x7f3240003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324f7fe640 1 --2- 172.21.15.111:0/466812685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32400047c0 0x7f3240004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324f7fe640 1 --2- 172.21.15.111:0/466812685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32580d2760 0x7f3240005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324f7fe640 1 -- 172.21.15.111:0/466812685 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32580d0cf0 con 0x7f3240003970 2024-09-13T17:58:10.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324f7fe640 1 -- 172.21.15.111:0/466812685 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32580d0fd0 con 0x7f32400047c0 2024-09-13T17:58:10.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324f7fe640 1 -- 172.21.15.111:0/466812685 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324000b980 con 0x7f32580d2760 2024-09-13T17:58:10.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f325cf1f640 1 --2- 172.21.15.111:0/466812685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3240003970 0x7f3240003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f324ffff640 1 --2- 172.21.15.111:0/466812685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32400047c0 0x7f3240004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f325d720640 1 --2- 172.21.15.111:0/466812685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32580d2760 0x7f3240005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f325cf1f640 1 -- 172.21.15.111:0/466812685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32580d2760 msgr2=0x7f3240005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:10.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f325cf1f640 1 --2- 172.21.15.111:0/466812685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32580d2760 0x7f3240005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f325cf1f640 1 -- 172.21.15.111:0/466812685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32400047c0 msgr2=0x7f3240004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:10.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f325cf1f640 1 --2- 172.21.15.111:0/466812685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32400047c0 0x7f3240004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.116+0000 7f325cf1f640 1 -- 172.21.15.111:0/466812685 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3240003520 con 0x7f3240003970 2024-09-13T17:58:10.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.117+0000 7f325cf1f640 1 --2- 172.21.15.111:0/466812685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3240003970 0x7f3240003d90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f325400c210 tx=0x7f32400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5216a7b153e9dc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:10.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.117+0000 7f324f7fe640 1 -- 172.21.15.111:0/466812685 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254005ad0 con 0x7f3240003970 2024-09-13T17:58:10.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.117+0000 7f324f7fe640 1 -- 172.21.15.111:0/466812685 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3254005c30 con 0x7f3240003970 2024-09-13T17:58:10.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.117+0000 7f324f7fe640 1 -- 172.21.15.111:0/466812685 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254007070 con 0x7f3240003970 2024-09-13T17:58:10.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.118+0000 7f325f183640 1 -- 172.21.15.111:0/466812685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3240003970 msgr2=0x7f3240003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.118+0000 7f325f183640 1 --2- 172.21.15.111:0/466812685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3240003970 0x7f3240003d90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f325400c210 tx=0x7f32400057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.118+0000 7f325d720640 1 -- 172.21.15.111:0/466812685 reap_dead start 2024-09-13T17:58:10.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.118+0000 7f325f183640 1 -- 172.21.15.111:0/466812685 shutdown_connections 2024-09-13T17:58:10.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.118+0000 7f325f183640 1 -- 172.21.15.111:0/466812685 >> 172.21.15.111:0/466812685 conn(0x7f3258089f60 msgr2=0x7f325809b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:10.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.118+0000 7f325f183640 1 -- 172.21.15.111:0/466812685 shutdown_connections 2024-09-13T17:58:10.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.118+0000 7f325f183640 1 -- 172.21.15.111:0/466812685 wait complete. 2024-09-13T17:58:10.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.119+0000 7f325f183640 1 Processor -- start 2024-09-13T17:58:10.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.119+0000 7f325f183640 1 -- start start 2024-09-13T17:58:10.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.120+0000 7f325f183640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3240003970 0x7f325815f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.120+0000 7f325f183640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3258158860 0x7f32581f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.120+0000 7f325f183640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32580caf20 0x7f32580cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.120+0000 7f325f183640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3258158ce0 con 0x7f3240003970 2024-09-13T17:58:10.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.120+0000 7f325f183640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32580cb8e0 con 0x7f3258158860 2024-09-13T17:58:10.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.120+0000 7f325f183640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325815ca60 con 0x7f32580caf20 2024-09-13T17:58:10.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.120+0000 7f325cf1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3240003970 0x7f325815f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.120+0000 7f325cf1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3240003970 0x7f325815f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51122/0 (socket says 172.21.15.111:51122) 2024-09-13T17:58:10.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.120+0000 7f325cf1f640 1 -- 172.21.15.111:0/3073659785 learned_addr learned my addr 172.21.15.111:0/3073659785 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:10.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.120+0000 7f325d720640 1 --2- 172.21.15.111:0/3073659785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32580caf20 0x7f32580cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.120+0000 7f324ffff640 1 --2- 172.21.15.111:0/3073659785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3258158860 0x7f32581f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.120+0000 7f325cf1f640 1 -- 172.21.15.111:0/3073659785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32580caf20 msgr2=0x7f32580cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.120+0000 7f325cf1f640 1 --2- 172.21.15.111:0/3073659785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32580caf20 0x7f32580cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.120+0000 7f325cf1f640 1 -- 172.21.15.111:0/3073659785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3258158860 msgr2=0x7f32581f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.120+0000 7f325cf1f640 1 --2- 172.21.15.111:0/3073659785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3258158860 0x7f32581f2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.120+0000 7f325cf1f640 1 -- 172.21.15.111:0/3073659785 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32581f2b60 con 0x7f3240003970 2024-09-13T17:58:10.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.121+0000 7f325cf1f640 1 --2- 172.21.15.111:0/3073659785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3240003970 0x7f325815f750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f325402ad50 tx=0x7f3254001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:10.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.121+0000 7f324d7fa640 1 -- 172.21.15.111:0/3073659785 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254004cf0 con 0x7f3240003970 2024-09-13T17:58:10.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.121+0000 7f324d7fa640 1 -- 172.21.15.111:0/3073659785 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3254004e50 con 0x7f3240003970 2024-09-13T17:58:10.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.121+0000 7f324d7fa640 1 -- 172.21.15.111:0/3073659785 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254007290 con 0x7f3240003970 2024-09-13T17:58:10.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.121+0000 7f325f183640 1 -- 172.21.15.111:0/3073659785 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32581f2e40 con 0x7f3240003970 2024-09-13T17:58:10.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.121+0000 7f325f183640 1 -- 172.21.15.111:0/3073659785 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325815cd20 con 0x7f3240003970 2024-09-13T17:58:10.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.122+0000 7f325f183640 1 -- 172.21.15.111:0/3073659785 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325815ceb0 con 0x7f3240003970 2024-09-13T17:58:10.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.122+0000 7f324d7fa640 1 -- 172.21.15.111:0/3073659785 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f32540073f0 con 0x7f3240003970 2024-09-13T17:58:10.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.123+0000 7f324d7fa640 1 --2- 172.21.15.111:0/3073659785 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f324406ac30 0x7f324406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.123+0000 7f324d7fa640 1 -- 172.21.15.111:0/3073659785 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f32540a9c30 con 0x7f3240003970 2024-09-13T17:58:10.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.123+0000 7f324ffff640 1 --2- 172.21.15.111:0/3073659785 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f324406ac30 0x7f324406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.123+0000 7f324ffff640 1 --2- 172.21.15.111:0/3073659785 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f324406ac30 0x7f324406d0f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f32580ca750 tx=0x7f3248009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:10.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.127+0000 7f324d7fa640 1 -- 172.21.15.111:0/3073659785 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f32540784e0 con 0x7f3240003970 2024-09-13T17:58:10.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.288+0000 7f325f183640 1 -- 172.21.15.111:0/3073659785 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f325815ceb0 con 0x7f3240003970 2024-09-13T17:58:10.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.289+0000 7f324d7fa640 1 -- 172.21.15.111:0/3073659785 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3254042020 con 0x7f3240003970 2024-09-13T17:58:10.291 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:58:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.291+0000 7f325f183640 1 -- 172.21.15.111:0/3073659785 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f324406ac30 msgr2=0x7f324406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.291+0000 7f325f183640 1 --2- 172.21.15.111:0/3073659785 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f324406ac30 0x7f324406d0f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f32580ca750 tx=0x7f3248009190 comp rx=0 tx=0).stop 2024-09-13T17:58:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.291+0000 7f325f183640 1 -- 172.21.15.111:0/3073659785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3240003970 msgr2=0x7f325815f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.291+0000 7f325f183640 1 --2- 172.21.15.111:0/3073659785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3240003970 0x7f325815f750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f325402ad50 tx=0x7f3254001a00 comp rx=0 tx=0).stop 2024-09-13T17:58:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.291+0000 7f325f183640 1 -- 172.21.15.111:0/3073659785 shutdown_connections 2024-09-13T17:58:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.291+0000 7f325f183640 1 --2- 172.21.15.111:0/3073659785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32580caf20 0x7f32580cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.291+0000 7f325f183640 1 --2- 172.21.15.111:0/3073659785 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f324406ac30 0x7f324406d0f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.291+0000 7f325f183640 1 --2- 172.21.15.111:0/3073659785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3258158860 0x7f32581f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.291+0000 7f325f183640 1 --2- 172.21.15.111:0/3073659785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3240003970 0x7f325815f750 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.291+0000 7f325f183640 1 -- 172.21.15.111:0/3073659785 >> 172.21.15.111:0/3073659785 conn(0x7f3258089f60 msgr2=0x7f32580cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.291+0000 7f325f183640 1 -- 172.21.15.111:0/3073659785 shutdown_connections 2024-09-13T17:58:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.291+0000 7f325f183640 1 -- 172.21.15.111:0/3073659785 wait complete. 2024-09-13T17:58:10.303 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:58:10.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.402+0000 7f6f79e4f640 1 Processor -- start 2024-09-13T17:58:10.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.402+0000 7f6f79e4f640 1 -- start start 2024-09-13T17:58:10.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.402+0000 7f6f79e4f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6f7415a3b0 0x7f6f740b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.402+0000 7f6f79e4f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f740cec00 0x7f6f740ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.402+0000 7f6f79e4f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f7415a790 con 0x7f6f740b1720 2024-09-13T17:58:10.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.402+0000 7f6f79e4f640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f740b2680 con 0x7f6f7415a3b0 2024-09-13T17:58:10.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.402+0000 7f6f79e4f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740b2960 con 0x7f6f740cec00 2024-09-13T17:58:10.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.403+0000 7f6f737fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6f7415a3b0 0x7f6f740b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.403+0000 7f6f737fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6f7415a3b0 0x7f6f740b20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51124/0 (socket says 172.21.15.111:51124) 2024-09-13T17:58:10.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.403+0000 7f6f737fe640 1 -- 172.21.15.111:0/2713568292 learned_addr learned my addr 172.21.15.111:0/2713568292 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:10.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.403+0000 7f6f73fff640 1 --2- 172.21.15.111:0/2713568292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f740cec00 0x7f6f740ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.403+0000 7f6f737fe640 1 -- 172.21.15.111:0/2713568292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f740cec00 msgr2=0x7f6f740ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:10.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.403+0000 7f6f737fe640 1 --2- 172.21.15.111:0/2713568292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f740cec00 0x7f6f740ab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.403+0000 7f6f737fe640 1 -- 172.21.15.111:0/2713568292 >> v1:172.21.15.188:6789/0 conn(0x7f6f740b1720 legacy=0x7f6f740ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:10.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.403+0000 7f6f737fe640 1 -- 172.21.15.111:0/2713568292 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f740b2c40 con 0x7f6f7415a3b0 2024-09-13T17:58:10.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.403+0000 7f6f737fe640 1 --2- 172.21.15.111:0/2713568292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f7415a3b0 0x7f6f740b20a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009870 tx=0x7f6f6402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d504d9f112b2e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:10.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.404+0000 7f6f727fc640 1 -- 172.21.15.111:0/2713568292 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64032440 con 0x7f6f7415a3b0 2024-09-13T17:58:10.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.404+0000 7f6f727fc640 1 -- 172.21.15.111:0/2713568292 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f640325a0 con 0x7f6f7415a3b0 2024-09-13T17:58:10.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.404+0000 7f6f727fc640 1 -- 172.21.15.111:0/2713568292 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64032930 con 0x7f6f7415a3b0 2024-09-13T17:58:10.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.404+0000 7f6f79e4f640 1 -- 172.21.15.111:0/2713568292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f7415a3b0 msgr2=0x7f6f740b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.404+0000 7f6f79e4f640 1 --2- 172.21.15.111:0/2713568292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f7415a3b0 0x7f6f740b20a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009870 tx=0x7f6f6402a210 comp rx=0 tx=0).stop 2024-09-13T17:58:10.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.404+0000 7f6f79e4f640 1 -- 172.21.15.111:0/2713568292 shutdown_connections 2024-09-13T17:58:10.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.404+0000 7f6f79e4f640 1 --2- 172.21.15.111:0/2713568292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f740cec00 0x7f6f740ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.404+0000 7f6f79e4f640 1 --2- 172.21.15.111:0/2713568292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f7415a3b0 0x7f6f740b20a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.404+0000 7f6f79e4f640 1 -- 172.21.15.111:0/2713568292 >> 172.21.15.111:0/2713568292 conn(0x7f6f7406a0b0 msgr2=0x7f6f7406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:10.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.404+0000 7f6f79e4f640 1 -- 172.21.15.111:0/2713568292 shutdown_connections 2024-09-13T17:58:10.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.404+0000 7f6f79e4f640 1 -- 172.21.15.111:0/2713568292 wait complete. 2024-09-13T17:58:10.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.405+0000 7f6f79e4f640 1 Processor -- start 2024-09-13T17:58:10.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.406+0000 7f6f79e4f640 1 -- start start 2024-09-13T17:58:10.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.406+0000 7f6f79e4f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f740b1720 0x7f6f7415bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.406+0000 7f6f79e4f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f740cec00 0x7f6f74071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.406+0000 7f6f79e4f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f7415a3b0 0x7f6f740974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.406+0000 7f6f79e4f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7415d880 con 0x7f6f7415a3b0 2024-09-13T17:58:10.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.406+0000 7f6f79e4f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7415c3b0 con 0x7f6f740cec00 2024-09-13T17:58:10.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.406+0000 7f6f79e4f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7415c690 con 0x7f6f740b1720 2024-09-13T17:58:10.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.406+0000 7f6f73fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f7415a3b0 0x7f6f740974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.406+0000 7f6f73fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f7415a3b0 0x7f6f740974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51134/0 (socket says 172.21.15.111:51134) 2024-09-13T17:58:10.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.406+0000 7f6f73fff640 1 -- 172.21.15.111:0/2146103273 learned_addr learned my addr 172.21.15.111:0/2146103273 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:10.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.406+0000 7f6f737fe640 1 --2- 172.21.15.111:0/2146103273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f740b1720 0x7f6f7415bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.406+0000 7f6f72ffd640 1 --2- 172.21.15.111:0/2146103273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f740cec00 0x7f6f74071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.407+0000 7f6f73fff640 1 -- 172.21.15.111:0/2146103273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f740b1720 msgr2=0x7f6f7415bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.407+0000 7f6f73fff640 1 --2- 172.21.15.111:0/2146103273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f740b1720 0x7f6f7415bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.407+0000 7f6f73fff640 1 -- 172.21.15.111:0/2146103273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f740cec00 msgr2=0x7f6f74071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.407+0000 7f6f73fff640 1 --2- 172.21.15.111:0/2146103273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f740cec00 0x7f6f74071ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.407+0000 7f6f73fff640 1 -- 172.21.15.111:0/2146103273 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f640094d0 con 0x7f6f7415a3b0 2024-09-13T17:58:10.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.407+0000 7f6f73fff640 1 --2- 172.21.15.111:0/2146103273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f7415a3b0 0x7f6f740974f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c007d40 tx=0x7f6f6c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:10.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.407+0000 7f6f5bfff640 1 -- 172.21.15.111:0/2146103273 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c014420 con 0x7f6f7415a3b0 2024-09-13T17:58:10.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.407+0000 7f6f5bfff640 1 -- 172.21.15.111:0/2146103273 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c0145a0 con 0x7f6f7415a3b0 2024-09-13T17:58:10.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.407+0000 7f6f79e4f640 1 -- 172.21.15.111:0/2146103273 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7415c970 con 0x7f6f7415a3b0 2024-09-13T17:58:10.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.407+0000 7f6f5bfff640 1 -- 172.21.15.111:0/2146103273 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c014950 con 0x7f6f7415a3b0 2024-09-13T17:58:10.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.407+0000 7f6f79e4f640 1 -- 172.21.15.111:0/2146103273 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f7415cd00 con 0x7f6f7415a3b0 2024-09-13T17:58:10.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.408+0000 7f6f79e4f640 1 -- 172.21.15.111:0/2146103273 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7406a6e0 con 0x7f6f7415a3b0 2024-09-13T17:58:10.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.409+0000 7f6f5bfff640 1 -- 172.21.15.111:0/2146103273 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6f6c014ab0 con 0x7f6f7415a3b0 2024-09-13T17:58:10.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.409+0000 7f6f5bfff640 1 --2- 172.21.15.111:0/2146103273 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6f6006ac00 0x7f6f6006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.409+0000 7f6f5bfff640 1 -- 172.21.15.111:0/2146103273 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6f6c089e10 con 0x7f6f7415a3b0 2024-09-13T17:58:10.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.409+0000 7f6f737fe640 1 --2- 172.21.15.111:0/2146103273 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6f6006ac00 0x7f6f6006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.409+0000 7f6f737fe640 1 --2- 172.21.15.111:0/2146103273 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6f6006ac00 0x7f6f6006d0c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6f64034000 tx=0x7f6f6402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:10.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.413+0000 7f6f5bfff640 1 -- 172.21.15.111:0/2146103273 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f6f6c054d70 con 0x7f6f7415a3b0 2024-09-13T17:58:10.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.574+0000 7f6f79e4f640 1 -- 172.21.15.111:0/2146103273 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f7415cb00 con 0x7f6f7415a3b0 2024-09-13T17:58:10.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.574+0000 7f6f5bfff640 1 -- 172.21.15.111:0/2146103273 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f6c058390 con 0x7f6f7415a3b0 2024-09-13T17:58:10.577 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.576+0000 7f6f79e4f640 1 -- 172.21.15.111:0/2146103273 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6f6006ac00 msgr2=0x7f6f6006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.576+0000 7f6f79e4f640 1 --2- 172.21.15.111:0/2146103273 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6f6006ac00 0x7f6f6006d0c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6f64034000 tx=0x7f6f6402e000 comp rx=0 tx=0).stop 2024-09-13T17:58:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.576+0000 7f6f79e4f640 1 -- 172.21.15.111:0/2146103273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f7415a3b0 msgr2=0x7f6f740974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.576+0000 7f6f79e4f640 1 --2- 172.21.15.111:0/2146103273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f7415a3b0 0x7f6f740974f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c007d40 tx=0x7f6f6c00e4b0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.576+0000 7f6f79e4f640 1 -- 172.21.15.111:0/2146103273 shutdown_connections 2024-09-13T17:58:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.576+0000 7f6f79e4f640 1 --2- 172.21.15.111:0/2146103273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f7415a3b0 0x7f6f740974f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.576+0000 7f6f79e4f640 1 --2- 172.21.15.111:0/2146103273 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6f6006ac00 0x7f6f6006d0c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.576+0000 7f6f79e4f640 1 --2- 172.21.15.111:0/2146103273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f740cec00 0x7f6f74071ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.576+0000 7f6f79e4f640 1 --2- 172.21.15.111:0/2146103273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f740b1720 0x7f6f7415bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.576+0000 7f6f79e4f640 1 -- 172.21.15.111:0/2146103273 >> 172.21.15.111:0/2146103273 conn(0x7f6f7406a0b0 msgr2=0x7f6f74070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.576+0000 7f6f79e4f640 1 -- 172.21.15.111:0/2146103273 shutdown_connections 2024-09-13T17:58:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.576+0000 7f6f79e4f640 1 -- 172.21.15.111:0/2146103273 wait complete. 2024-09-13T17:58:10.589 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:58:10.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.687+0000 7f284a99d640 1 Processor -- start 2024-09-13T17:58:10.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.687+0000 7f284a99d640 1 -- start start 2024-09-13T17:58:10.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.688+0000 7f284a99d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f284415fa40 0x7f28440ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.688+0000 7f284a99d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f28440b1ac0 0x7f28440b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.688+0000 7f284a99d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f28440d2100 0x7f28440d25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.688+0000 7f284a99d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28440d2af0 con 0x7f28440d2100 2024-09-13T17:58:10.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.688+0000 7f284a99d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f284415fe20 con 0x7f28440b1ac0 2024-09-13T17:58:10.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.688+0000 7f284a99d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28440cd0d0 con 0x7f284415fa40 2024-09-13T17:58:10.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.688+0000 7f284a19c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f28440d2100 0x7f28440d25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.688+0000 7f284a19c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f28440d2100 0x7f28440d25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51140/0 (socket says 172.21.15.111:51140) 2024-09-13T17:58:10.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.688+0000 7f284a19c640 1 -- 172.21.15.111:0/4062296161 learned_addr learned my addr 172.21.15.111:0/4062296161 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:10.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.688+0000 7f284919a640 1 --2- 172.21.15.111:0/4062296161 >> v2:172.21.15.188:3300/0 conn(0x7f28440b1ac0 0x7f28440b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.688+0000 7f284999b640 1 --2- 172.21.15.111:0/4062296161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f284415fa40 0x7f28440ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.688+0000 7f284a19c640 1 -- 172.21.15.111:0/4062296161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f284415fa40 msgr2=0x7f28440ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:10.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.688+0000 7f284a19c640 1 --2- 172.21.15.111:0/4062296161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f284415fa40 0x7f28440ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.688+0000 7f284a19c640 1 -- 172.21.15.111:0/4062296161 >> v2:172.21.15.188:3300/0 conn(0x7f28440b1ac0 msgr2=0x7f28440b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:10.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.688+0000 7f284a19c640 1 --2- 172.21.15.111:0/4062296161 >> v2:172.21.15.188:3300/0 conn(0x7f28440b1ac0 0x7f28440b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.689+0000 7f284a19c640 1 -- 172.21.15.111:0/4062296161 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28440cd3b0 con 0x7f28440d2100 2024-09-13T17:58:10.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.689+0000 7f284a19c640 1 --2- 172.21.15.111:0/4062296161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28440d2100 0x7f28440d25b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f284408a960 tx=0x7f283c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b875551b5ca0ac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:10.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.689+0000 7f2848999640 1 -- 172.21.15.111:0/4062296161 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c031600 con 0x7f28440d2100 2024-09-13T17:58:10.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.689+0000 7f2848999640 1 -- 172.21.15.111:0/4062296161 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283c02d050 con 0x7f28440d2100 2024-09-13T17:58:10.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.689+0000 7f2848999640 1 -- 172.21.15.111:0/4062296161 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c031980 con 0x7f28440d2100 2024-09-13T17:58:10.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.690+0000 7f284a99d640 1 -- 172.21.15.111:0/4062296161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28440d2100 msgr2=0x7f28440d25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.690+0000 7f284a99d640 1 --2- 172.21.15.111:0/4062296161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28440d2100 0x7f28440d25b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f284408a960 tx=0x7f283c02a6a0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.690+0000 7f284a99d640 1 -- 172.21.15.111:0/4062296161 shutdown_connections 2024-09-13T17:58:10.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.690+0000 7f284a99d640 1 --2- 172.21.15.111:0/4062296161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28440d2100 0x7f28440d25b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.690+0000 7f284a99d640 1 --2- 172.21.15.111:0/4062296161 >> v2:172.21.15.188:3300/0 conn(0x7f28440b1ac0 0x7f28440b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.690+0000 7f284a99d640 1 --2- 172.21.15.111:0/4062296161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f284415fa40 0x7f28440ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.690+0000 7f284a99d640 1 -- 172.21.15.111:0/4062296161 >> 172.21.15.111:0/4062296161 conn(0x7f284408bcf0 msgr2=0x7f284409b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:10.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.690+0000 7f284a99d640 1 -- 172.21.15.111:0/4062296161 shutdown_connections 2024-09-13T17:58:10.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.690+0000 7f284a99d640 1 -- 172.21.15.111:0/4062296161 wait complete. 2024-09-13T17:58:10.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.691+0000 7f284a99d640 1 Processor -- start 2024-09-13T17:58:10.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.691+0000 7f284a99d640 1 -- start start 2024-09-13T17:58:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284a99d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28440b1ac0 0x7f284415d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284a99d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28440d2100 0x7f28440c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284a99d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f284415fa40 0x7f28440ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284a99d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284415dba0 con 0x7f28440d2100 2024-09-13T17:58:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284a99d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28440ca800 con 0x7f28440b1ac0 2024-09-13T17:58:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284a99d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28440caae0 con 0x7f284415fa40 2024-09-13T17:58:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284919a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28440d2100 0x7f28440c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284919a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28440d2100 0x7f28440c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51156/0 (socket says 172.21.15.111:51156) 2024-09-13T17:58:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284919a640 1 -- 172.21.15.111:0/1206091851 learned_addr learned my addr 172.21.15.111:0/1206091851 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284999b640 1 --2- 172.21.15.111:0/1206091851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28440b1ac0 0x7f284415d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284a19c640 1 --2- 172.21.15.111:0/1206091851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f284415fa40 0x7f28440ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284919a640 1 -- 172.21.15.111:0/1206091851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f284415fa40 msgr2=0x7f28440ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284919a640 1 --2- 172.21.15.111:0/1206091851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f284415fa40 0x7f28440ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284919a640 1 -- 172.21.15.111:0/1206091851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28440b1ac0 msgr2=0x7f284415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284919a640 1 --2- 172.21.15.111:0/1206091851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28440b1ac0 0x7f284415d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284919a640 1 -- 172.21.15.111:0/1206091851 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283c009b30 con 0x7f28440d2100 2024-09-13T17:58:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284999b640 1 --2- 172.21.15.111:0/1206091851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28440b1ac0 0x7f284415d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.692+0000 7f284919a640 1 --2- 172.21.15.111:0/1206091851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28440d2100 0x7f28440c9df0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f283400c7f0 tx=0x7f283400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.693+0000 7f283a7fc640 1 -- 172.21.15.111:0/1206091851 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2834004220 con 0x7f28440d2100 2024-09-13T17:58:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.693+0000 7f283a7fc640 1 -- 172.21.15.111:0/1206091851 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2834004380 con 0x7f28440d2100 2024-09-13T17:58:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.693+0000 7f284a99d640 1 -- 172.21.15.111:0/1206091851 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28440cadc0 con 0x7f28440d2100 2024-09-13T17:58:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.693+0000 7f283a7fc640 1 -- 172.21.15.111:0/1206091851 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2834007860 con 0x7f28440d2100 2024-09-13T17:58:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.693+0000 7f284a99d640 1 -- 172.21.15.111:0/1206091851 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28440cb380 con 0x7f28440d2100 2024-09-13T17:58:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.694+0000 7f284a99d640 1 -- 172.21.15.111:0/1206091851 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2844073270 con 0x7f28440d2100 2024-09-13T17:58:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.694+0000 7f283a7fc640 1 -- 172.21.15.111:0/1206091851 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2834007bd0 con 0x7f28440d2100 2024-09-13T17:58:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.695+0000 7f283a7fc640 1 --2- 172.21.15.111:0/1206091851 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f282806ac00 0x7f282806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.695+0000 7f284999b640 1 --2- 172.21.15.111:0/1206091851 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f282806ac00 0x7f282806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.695+0000 7f283a7fc640 1 -- 172.21.15.111:0/1206091851 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2834089990 con 0x7f28440d2100 2024-09-13T17:58:10.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.695+0000 7f284999b640 1 --2- 172.21.15.111:0/1206091851 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f282806ac00 0x7f282806d0c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2840006fd0 tx=0x7f2840008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:10.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.699+0000 7f283a7fc640 1 -- 172.21.15.111:0/1206091851 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f2834054840 con 0x7f28440d2100 2024-09-13T17:58:10.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.860+0000 7f284a99d640 1 -- 172.21.15.111:0/1206091851 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f28440b1f00 con 0x7f28440d2100 2024-09-13T17:58:10.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.860+0000 7f283a7fc640 1 -- 172.21.15.111:0/1206091851 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f283401e0a0 con 0x7f28440d2100 2024-09-13T17:58:10.862 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:58:10.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.862+0000 7f284a99d640 1 -- 172.21.15.111:0/1206091851 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f282806ac00 msgr2=0x7f282806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.862+0000 7f284a99d640 1 --2- 172.21.15.111:0/1206091851 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f282806ac00 0x7f282806d0c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2840006fd0 tx=0x7f2840008040 comp rx=0 tx=0).stop 2024-09-13T17:58:10.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.862+0000 7f284a99d640 1 -- 172.21.15.111:0/1206091851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28440d2100 msgr2=0x7f28440c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.862+0000 7f284a99d640 1 --2- 172.21.15.111:0/1206091851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28440d2100 0x7f28440c9df0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f283400c7f0 tx=0x7f283400ccb0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.862+0000 7f284a99d640 1 -- 172.21.15.111:0/1206091851 shutdown_connections 2024-09-13T17:58:10.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.862+0000 7f284a99d640 1 --2- 172.21.15.111:0/1206091851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f284415fa40 0x7f28440ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.862+0000 7f284a99d640 1 --2- 172.21.15.111:0/1206091851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28440d2100 0x7f28440c9df0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.862+0000 7f284a99d640 1 --2- 172.21.15.111:0/1206091851 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f282806ac00 0x7f282806d0c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.862+0000 7f284a99d640 1 --2- 172.21.15.111:0/1206091851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28440b1ac0 0x7f284415d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.862+0000 7f284a99d640 1 -- 172.21.15.111:0/1206091851 >> 172.21.15.111:0/1206091851 conn(0x7f284408bcf0 msgr2=0x7f28440b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:10.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.862+0000 7f284a99d640 1 -- 172.21.15.111:0/1206091851 shutdown_connections 2024-09-13T17:58:10.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.862+0000 7f284a99d640 1 -- 172.21.15.111:0/1206091851 wait complete. 2024-09-13T17:58:10.875 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:58:10.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.973+0000 7f733b8c5640 1 Processor -- start 2024-09-13T17:58:10.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.974+0000 7f733b8c5640 1 -- start start 2024-09-13T17:58:10.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.974+0000 7f733b8c5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340cdff0 0x7f733406c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.974+0000 7f733b8c5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73340cd450 con 0x7f73340d2760 2024-09-13T17:58:10.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.974+0000 7f733b8c5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73340d0cf0 con 0x7f733415c500 2024-09-13T17:58:10.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.974+0000 7f733b8c5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73340d0fd0 con 0x7f73340cdff0 2024-09-13T17:58:10.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.974+0000 7f7338e60640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f73340d2760 0x7f73340cd8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58882/0 (socket says 172.21.15.111:58882) 2024-09-13T17:58:10.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f7338e60640 1 -- 172.21.15.111:0/81160153 learned_addr learned my addr 172.21.15.111:0/81160153 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:10.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f7339e62640 1 --2- 172.21.15.111:0/81160153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340cdff0 0x7f733406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f732bfff640 1 -- 172.21.15.111:0/81160153 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 613920568 0 0) 0x7f73340cd450 con 0x7f73340d2760 2024-09-13T17:58:10.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f732bfff640 1 -- 172.21.15.111:0/81160153 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7320003520 con 0x7f73340d2760 2024-09-13T17:58:10.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f732bfff640 1 -- 172.21.15.111:0/81160153 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1099160497 0 0) 0x7f7320003520 con 0x7f73340d2760 2024-09-13T17:58:10.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f732bfff640 1 -- 172.21.15.111:0/81160153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340cdff0 msgr2=0x7f733406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:10.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f732bfff640 1 --2- 172.21.15.111:0/81160153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340cdff0 0x7f733406c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f732bfff640 1 -- 172.21.15.111:0/81160153 >> v1:172.21.15.188:6789/0 conn(0x7f733415c500 legacy=0x7f73340d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f732bfff640 1 -- 172.21.15.111:0/81160153 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73340cd0e0 con 0x7f73340d2760 2024-09-13T17:58:10.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f732bfff640 1 -- 172.21.15.111:0/81160153 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f7324001670 con 0x7f73340d2760 2024-09-13T17:58:10.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f732bfff640 1 -- 172.21.15.111:0/81160153 >> v1:172.21.15.111:6789/0 conn(0x7f73340d2760 legacy=0x7f73340cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f732bfff640 1 --2- 172.21.15.111:0/81160153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73200039a0 0x7f7320003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f732bfff640 1 --2- 172.21.15.111:0/81160153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7320004350 0x7f7320004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f732bfff640 1 --2- 172.21.15.111:0/81160153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340cdff0 0x7f7320005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f732bfff640 1 -- 172.21.15.111:0/81160153 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73340d0fd0 con 0x7f7320004350 2024-09-13T17:58:10.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f732bfff640 1 -- 172.21.15.111:0/81160153 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732000b980 con 0x7f73200039a0 2024-09-13T17:58:10.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f732bfff640 1 -- 172.21.15.111:0/81160153 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732000bb40 con 0x7f73340cdff0 2024-09-13T17:58:10.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f7338e60640 1 --2- 172.21.15.111:0/81160153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7320004350 0x7f7320004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f7339661640 1 --2- 172.21.15.111:0/81160153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73200039a0 0x7f7320003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f7339e62640 1 --2- 172.21.15.111:0/81160153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340cdff0 0x7f7320005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f7338e60640 1 -- 172.21.15.111:0/81160153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340cdff0 msgr2=0x7f7320005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:10.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f7338e60640 1 --2- 172.21.15.111:0/81160153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340cdff0 0x7f7320005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f7338e60640 1 -- 172.21.15.111:0/81160153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73200039a0 msgr2=0x7f7320003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:10.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f7338e60640 1 --2- 172.21.15.111:0/81160153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73200039a0 0x7f7320003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.975+0000 7f7338e60640 1 -- 172.21.15.111:0/81160153 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7320003520 con 0x7f7320004350 2024-09-13T17:58:10.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.976+0000 7f7338e60640 1 --2- 172.21.15.111:0/81160153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7320004350 0x7f7320004820 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f732400c160 tx=0x7f73200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e24057e8566411 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:10.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.976+0000 7f732bfff640 1 -- 172.21.15.111:0/81160153 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7324005ad0 con 0x7f7320004350 2024-09-13T17:58:10.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.976+0000 7f732bfff640 1 -- 172.21.15.111:0/81160153 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732402d070 con 0x7f7320004350 2024-09-13T17:58:10.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.976+0000 7f732bfff640 1 -- 172.21.15.111:0/81160153 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7324005e90 con 0x7f7320004350 2024-09-13T17:58:10.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.977+0000 7f733b8c5640 1 -- 172.21.15.111:0/81160153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7320004350 msgr2=0x7f7320004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.977+0000 7f733b8c5640 1 --2- 172.21.15.111:0/81160153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7320004350 0x7f7320004820 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f732400c160 tx=0x7f73200057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.977+0000 7f7339e62640 1 -- 172.21.15.111:0/81160153 reap_dead start 2024-09-13T17:58:10.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.977+0000 7f733b8c5640 1 -- 172.21.15.111:0/81160153 shutdown_connections 2024-09-13T17:58:10.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.977+0000 7f733b8c5640 1 -- 172.21.15.111:0/81160153 >> 172.21.15.111:0/81160153 conn(0x7f7334089f60 msgr2=0x7f733409b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:10.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.977+0000 7f733b8c5640 1 -- 172.21.15.111:0/81160153 shutdown_connections 2024-09-13T17:58:10.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.977+0000 7f733b8c5640 1 -- 172.21.15.111:0/81160153 wait complete. 2024-09-13T17:58:10.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.978+0000 7f733b8c5640 1 Processor -- start 2024-09-13T17:58:10.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.978+0000 7f733b8c5640 1 -- start start 2024-09-13T17:58:10.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.979+0000 7f733b8c5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7320004350 0x7f73340b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.979+0000 7f733b8c5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73341f6250 0x7f733415d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.979+0000 7f733b8c5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f733415caf0 0x7f733415d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.979+0000 7f733b8c5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73341f67c0 con 0x7f733415caf0 2024-09-13T17:58:10.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.979+0000 7f733b8c5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73340c9630 con 0x7f7320004350 2024-09-13T17:58:10.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.979+0000 7f733b8c5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73340c9910 con 0x7f73341f6250 2024-09-13T17:58:10.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.979+0000 7f7339e62640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f733415caf0 0x7f733415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.979+0000 7f7339e62640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f733415caf0 0x7f733415d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51172/0 (socket says 172.21.15.111:51172) 2024-09-13T17:58:10.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.979+0000 7f7339e62640 1 -- 172.21.15.111:0/98252862 learned_addr learned my addr 172.21.15.111:0/98252862 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:10.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.979+0000 7f7338e60640 1 --2- 172.21.15.111:0/98252862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73341f6250 0x7f733415d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.979+0000 7f7339661640 1 --2- 172.21.15.111:0/98252862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7320004350 0x7f73340b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.979+0000 7f7339e62640 1 -- 172.21.15.111:0/98252862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73341f6250 msgr2=0x7f733415d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.979+0000 7f7339e62640 1 --2- 172.21.15.111:0/98252862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73341f6250 0x7f733415d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.979+0000 7f7339e62640 1 -- 172.21.15.111:0/98252862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7320004350 msgr2=0x7f73340b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:10.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.979+0000 7f7339e62640 1 --2- 172.21.15.111:0/98252862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7320004350 0x7f73340b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:10.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.979+0000 7f7339e62640 1 -- 172.21.15.111:0/98252862 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73340c9bf0 con 0x7f733415caf0 2024-09-13T17:58:10.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.979+0000 7f7339e62640 1 --2- 172.21.15.111:0/98252862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f733415caf0 0x7f733415d880 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f73340c94b0 tx=0x7f732c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:10.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.980+0000 7f7329ffb640 1 -- 172.21.15.111:0/98252862 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c00e040 con 0x7f733415caf0 2024-09-13T17:58:10.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.980+0000 7f7329ffb640 1 -- 172.21.15.111:0/98252862 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c00f040 con 0x7f733415caf0 2024-09-13T17:58:10.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.980+0000 7f7329ffb640 1 -- 172.21.15.111:0/98252862 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c015670 con 0x7f733415caf0 2024-09-13T17:58:10.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.980+0000 7f733b8c5640 1 -- 172.21.15.111:0/98252862 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73340c9ed0 con 0x7f733415caf0 2024-09-13T17:58:10.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.980+0000 7f733b8c5640 1 -- 172.21.15.111:0/98252862 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73340b3240 con 0x7f733415caf0 2024-09-13T17:58:10.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.981+0000 7f733b8c5640 1 -- 172.21.15.111:0/98252862 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f733406c960 con 0x7f733415caf0 2024-09-13T17:58:10.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.981+0000 7f7329ffb640 1 -- 172.21.15.111:0/98252862 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f732c0157d0 con 0x7f733415caf0 2024-09-13T17:58:10.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.982+0000 7f7329ffb640 1 --2- 172.21.15.111:0/98252862 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f731806ac00 0x7f731806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:10.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.982+0000 7f7329ffb640 1 -- 172.21.15.111:0/98252862 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f732c08a050 con 0x7f733415caf0 2024-09-13T17:58:10.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.982+0000 7f7339661640 1 --2- 172.21.15.111:0/98252862 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f731806ac00 0x7f731806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:10.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.982+0000 7f7339661640 1 --2- 172.21.15.111:0/98252862 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f731806ac00 0x7f731806d0c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f73300018c0 tx=0x7f7330009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:10.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:10.986+0000 7f7329ffb640 1 -- 172.21.15.111:0/98252862 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f732c054fb0 con 0x7f733415caf0 2024-09-13T17:58:11.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.148+0000 7f733b8c5640 1 -- 172.21.15.111:0/98252862 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f733406b160 con 0x7f733415caf0 2024-09-13T17:58:11.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.148+0000 7f7329ffb640 1 -- 172.21.15.111:0/98252862 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f732c0585d0 con 0x7f733415caf0 2024-09-13T17:58:11.150 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:11.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.150+0000 7f733b8c5640 1 -- 172.21.15.111:0/98252862 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f731806ac00 msgr2=0x7f731806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:11.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.150+0000 7f733b8c5640 1 --2- 172.21.15.111:0/98252862 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f731806ac00 0x7f731806d0c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f73300018c0 tx=0x7f7330009040 comp rx=0 tx=0).stop 2024-09-13T17:58:11.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.150+0000 7f733b8c5640 1 -- 172.21.15.111:0/98252862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f733415caf0 msgr2=0x7f733415d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:11.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.150+0000 7f733b8c5640 1 --2- 172.21.15.111:0/98252862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f733415caf0 0x7f733415d880 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f73340c94b0 tx=0x7f732c00f580 comp rx=0 tx=0).stop 2024-09-13T17:58:11.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.150+0000 7f733b8c5640 1 -- 172.21.15.111:0/98252862 shutdown_connections 2024-09-13T17:58:11.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.150+0000 7f733b8c5640 1 --2- 172.21.15.111:0/98252862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f733415caf0 0x7f733415d880 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.150+0000 7f733b8c5640 1 --2- 172.21.15.111:0/98252862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73341f6250 0x7f733415d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.150+0000 7f733b8c5640 1 --2- 172.21.15.111:0/98252862 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f731806ac00 0x7f731806d0c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.150+0000 7f733b8c5640 1 --2- 172.21.15.111:0/98252862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7320004350 0x7f73340b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.150+0000 7f733b8c5640 1 -- 172.21.15.111:0/98252862 >> 172.21.15.111:0/98252862 conn(0x7f7334089f60 msgr2=0x7f733406b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:11.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.150+0000 7f733b8c5640 1 -- 172.21.15.111:0/98252862 shutdown_connections 2024-09-13T17:58:11.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.150+0000 7f733b8c5640 1 -- 172.21.15.111:0/98252862 wait complete. 2024-09-13T17:58:11.170 INFO:tasks.cephfs_test_runner:Starting test: test_replay_create (tasks.cephfs.test_cap_flush.TestCapFlush) MDS starts to handle client caps when it enters clientreplay stage. 2024-09-13T17:58:11.170 INFO:tasks.cephfs_test_runner:test_replay_create (tasks.cephfs.test_cap_flush.TestCapFlush) 2024-09-13T17:58:11.171 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:11.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.266+0000 7f4d443ce640 1 Processor -- start 2024-09-13T17:58:11.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.266+0000 7f4d443ce640 1 -- start start 2024-09-13T17:58:11.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.267+0000 7f4d443ce640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4d3c0ae670 0x7f4d3c0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.267+0000 7f4d443ce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4d3c0cb640 0x7f4d3c0989d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.267+0000 7f4d443ce640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3c0ad1d0 con 0x7f4d3c15a7a0 2024-09-13T17:58:11.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.267+0000 7f4d443ce640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3c0af8e0 con 0x7f4d3c0ae670 2024-09-13T17:58:11.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.267+0000 7f4d443ce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3c0ce6f0 con 0x7f4d3c0cb640 2024-09-13T17:58:11.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.268+0000 7f4d41969640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4d3c0ae670 0x7f4d3c0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.268+0000 7f4d41969640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4d3c0ae670 0x7f4d3c0ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51176/0 (socket says 172.21.15.111:51176) 2024-09-13T17:58:11.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.268+0000 7f4d41969640 1 -- 172.21.15.111:0/3449545246 learned_addr learned my addr 172.21.15.111:0/3449545246 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:11.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.268+0000 7f4d4296b640 1 --2- 172.21.15.111:0/3449545246 >> v2:172.21.15.188:3300/0 conn(0x7f4d3c0cb640 0x7f4d3c0989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.268+0000 7f4d41969640 1 -- 172.21.15.111:0/3449545246 >> v2:172.21.15.188:3300/0 conn(0x7f4d3c0cb640 msgr2=0x7f4d3c0989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:11.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.268+0000 7f4d41969640 1 --2- 172.21.15.111:0/3449545246 >> v2:172.21.15.188:3300/0 conn(0x7f4d3c0cb640 0x7f4d3c0989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.268+0000 7f4d41969640 1 -- 172.21.15.111:0/3449545246 >> v1:172.21.15.188:6789/0 conn(0x7f4d3c15a7a0 legacy=0x7f4d3c0caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:11.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.268+0000 7f4d41969640 1 -- 172.21.15.111:0/3449545246 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c0cbc00 con 0x7f4d3c0ae670 2024-09-13T17:58:11.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.269+0000 7f4d41969640 1 --2- 172.21.15.111:0/3449545246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d3c0ae670 0x7f4d3c0ce060 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c067910 tx=0x7f4d3002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec074720f38436c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:11.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.269+0000 7f4d41168640 1 -- 172.21.15.111:0/3449545246 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30031600 con 0x7f4d3c0ae670 2024-09-13T17:58:11.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.269+0000 7f4d41168640 1 -- 172.21.15.111:0/3449545246 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3002d050 con 0x7f4d3c0ae670 2024-09-13T17:58:11.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.269+0000 7f4d41168640 1 -- 172.21.15.111:0/3449545246 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30031980 con 0x7f4d3c0ae670 2024-09-13T17:58:11.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.269+0000 7f4d443ce640 1 -- 172.21.15.111:0/3449545246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d3c0ae670 msgr2=0x7f4d3c0ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:11.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.269+0000 7f4d443ce640 1 --2- 172.21.15.111:0/3449545246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d3c0ae670 0x7f4d3c0ce060 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c067910 tx=0x7f4d3002a6f0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.269+0000 7f4d443ce640 1 -- 172.21.15.111:0/3449545246 shutdown_connections 2024-09-13T17:58:11.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.269+0000 7f4d443ce640 1 --2- 172.21.15.111:0/3449545246 >> v2:172.21.15.188:3300/0 conn(0x7f4d3c0cb640 0x7f4d3c0989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.269+0000 7f4d443ce640 1 --2- 172.21.15.111:0/3449545246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d3c0ae670 0x7f4d3c0ce060 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.269+0000 7f4d443ce640 1 -- 172.21.15.111:0/3449545246 >> 172.21.15.111:0/3449545246 conn(0x7f4d3c0b1f80 msgr2=0x7f4d3c0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:11.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.269+0000 7f4d443ce640 1 -- 172.21.15.111:0/3449545246 shutdown_connections 2024-09-13T17:58:11.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.269+0000 7f4d443ce640 1 -- 172.21.15.111:0/3449545246 wait complete. 2024-09-13T17:58:11.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.271+0000 7f4d443ce640 1 Processor -- start 2024-09-13T17:58:11.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.271+0000 7f4d443ce640 1 -- start start 2024-09-13T17:58:11.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.271+0000 7f4d443ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3c0ae670 0x7f4d3c151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.271+0000 7f4d443ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d3c0cb640 0x7f4d3c0c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.271+0000 7f4d443ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d3c15a7a0 0x7f4d3c0b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.271+0000 7f4d443ce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0b4ca0 con 0x7f4d3c15a7a0 2024-09-13T17:58:11.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.271+0000 7f4d443ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0b5130 con 0x7f4d3c0cb640 2024-09-13T17:58:11.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.271+0000 7f4d443ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0b4090 con 0x7f4d3c0ae670 2024-09-13T17:58:11.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.271+0000 7f4d4296b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d3c15a7a0 0x7f4d3c0b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.271+0000 7f4d4296b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d3c15a7a0 0x7f4d3c0b0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51180/0 (socket says 172.21.15.111:51180) 2024-09-13T17:58:11.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.271+0000 7f4d4296b640 1 -- 172.21.15.111:0/1114467804 learned_addr learned my addr 172.21.15.111:0/1114467804 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:11.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.272+0000 7f4d41969640 1 --2- 172.21.15.111:0/1114467804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d3c0cb640 0x7f4d3c0c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.272+0000 7f4d4216a640 1 --2- 172.21.15.111:0/1114467804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3c0ae670 0x7f4d3c151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.272+0000 7f4d4296b640 1 -- 172.21.15.111:0/1114467804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3c0ae670 msgr2=0x7f4d3c151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:11.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.272+0000 7f4d4296b640 1 --2- 172.21.15.111:0/1114467804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3c0ae670 0x7f4d3c151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.272+0000 7f4d4296b640 1 -- 172.21.15.111:0/1114467804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d3c0cb640 msgr2=0x7f4d3c0c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:11.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.272+0000 7f4d4296b640 1 --2- 172.21.15.111:0/1114467804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d3c0cb640 0x7f4d3c0c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.272+0000 7f4d4296b640 1 -- 172.21.15.111:0/1114467804 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d30009b30 con 0x7f4d3c15a7a0 2024-09-13T17:58:11.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.272+0000 7f4d41969640 1 --2- 172.21.15.111:0/1114467804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d3c0cb640 0x7f4d3c0c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:11.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.272+0000 7f4d4296b640 1 --2- 172.21.15.111:0/1114467804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d3c15a7a0 0x7f4d3c0b0440 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00d800 tx=0x7f4d2c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:11.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.272+0000 7f4d3affd640 1 -- 172.21.15.111:0/1114467804 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c00b6c0 con 0x7f4d3c15a7a0 2024-09-13T17:58:11.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.272+0000 7f4d443ce640 1 -- 172.21.15.111:0/1114467804 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3c0b4370 con 0x7f4d3c15a7a0 2024-09-13T17:58:11.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.272+0000 7f4d3affd640 1 -- 172.21.15.111:0/1114467804 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c00b840 con 0x7f4d3c15a7a0 2024-09-13T17:58:11.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.272+0000 7f4d3affd640 1 -- 172.21.15.111:0/1114467804 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c00bbf0 con 0x7f4d3c15a7a0 2024-09-13T17:58:11.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.272+0000 7f4d443ce640 1 -- 172.21.15.111:0/1114467804 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3c0b4930 con 0x7f4d3c15a7a0 2024-09-13T17:58:11.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.273+0000 7f4d443ce640 1 -- 172.21.15.111:0/1114467804 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3c0cccf0 con 0x7f4d3c15a7a0 2024-09-13T17:58:11.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.274+0000 7f4d3affd640 1 -- 172.21.15.111:0/1114467804 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4d2c00bd50 con 0x7f4d3c15a7a0 2024-09-13T17:58:11.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.274+0000 7f4d3affd640 1 --2- 172.21.15.111:0/1114467804 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d2806ac00 0x7f4d2806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.274+0000 7f4d3affd640 1 -- 172.21.15.111:0/1114467804 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4d2c089e10 con 0x7f4d3c15a7a0 2024-09-13T17:58:11.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.274+0000 7f4d4216a640 1 --2- 172.21.15.111:0/1114467804 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d2806ac00 0x7f4d2806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.274+0000 7f4d4216a640 1 --2- 172.21.15.111:0/1114467804 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d2806ac00 0x7f4d2806d0c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4d34006fd0 tx=0x7f4d34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:11.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.278+0000 7f4d3affd640 1 -- 172.21.15.111:0/1114467804 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4d2c054b20 con 0x7f4d3c15a7a0 2024-09-13T17:58:11.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.440+0000 7f4d443ce640 1 -- 172.21.15.111:0/1114467804 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d3c0989d0 con 0x7f4d3c15a7a0 2024-09-13T17:58:11.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.441+0000 7f4d3affd640 1 -- 172.21.15.111:0/1114467804 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18794 (secure 0 0 0) 0x7f4d2c058140 con 0x7f4d3c15a7a0 2024-09-13T17:58:11.443 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:11.443 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":26,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:07.749113+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:11.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.443+0000 7f4d443ce640 1 -- 172.21.15.111:0/1114467804 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d2806ac00 msgr2=0x7f4d2806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:11.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.443+0000 7f4d443ce640 1 --2- 172.21.15.111:0/1114467804 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d2806ac00 0x7f4d2806d0c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4d34006fd0 tx=0x7f4d34008040 comp rx=0 tx=0).stop 2024-09-13T17:58:11.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.443+0000 7f4d443ce640 1 -- 172.21.15.111:0/1114467804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d3c15a7a0 msgr2=0x7f4d3c0b0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:11.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.443+0000 7f4d443ce640 1 --2- 172.21.15.111:0/1114467804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d3c15a7a0 0x7f4d3c0b0440 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00d800 tx=0x7f4d2c00dcc0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.443+0000 7f4d443ce640 1 -- 172.21.15.111:0/1114467804 shutdown_connections 2024-09-13T17:58:11.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.444+0000 7f4d443ce640 1 --2- 172.21.15.111:0/1114467804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d3c15a7a0 0x7f4d3c0b0440 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.444+0000 7f4d443ce640 1 --2- 172.21.15.111:0/1114467804 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d2806ac00 0x7f4d2806d0c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.444+0000 7f4d443ce640 1 --2- 172.21.15.111:0/1114467804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d3c0cb640 0x7f4d3c0c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.444+0000 7f4d443ce640 1 --2- 172.21.15.111:0/1114467804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3c0ae670 0x7f4d3c151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.444+0000 7f4d443ce640 1 -- 172.21.15.111:0/1114467804 >> 172.21.15.111:0/1114467804 conn(0x7f4d3c0b1f80 msgr2=0x7f4d3c073260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:11.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.444+0000 7f4d443ce640 1 -- 172.21.15.111:0/1114467804 shutdown_connections 2024-09-13T17:58:11.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.444+0000 7f4d443ce640 1 -- 172.21.15.111:0/1114467804 wait complete. 2024-09-13T17:58:11.457 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:58:11.457 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:58:11.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.555+0000 7fcebc983640 1 Processor -- start 2024-09-13T17:58:11.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.556+0000 7fcebc983640 1 -- start start 2024-09-13T17:58:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.556+0000 7fcebc983640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fceb815c500 0x7fceb80d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.556+0000 7fcebc983640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fceb80d2590 0x7fceb80cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.556+0000 7fcebc983640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceb80cd450 con 0x7fceb80cde20 2024-09-13T17:58:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.556+0000 7fcebc983640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceb80d29f0 con 0x7fceb80d2590 2024-09-13T17:58:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.556+0000 7fcebc983640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb80d0cf0 con 0x7fceb815c500 2024-09-13T17:58:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.557+0000 7fceb5d9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fceb80d2590 0x7fceb80cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.557+0000 7fceb5d9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fceb80d2590 0x7fceb80cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51188/0 (socket says 172.21.15.111:51188) 2024-09-13T17:58:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.557+0000 7fceb5d9b640 1 -- 172.21.15.111:0/1540686468 learned_addr learned my addr 172.21.15.111:0/1540686468 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.557+0000 7fceb659c640 1 --2- 172.21.15.111:0/1540686468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fceb815c500 0x7fceb80d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.557+0000 7fceb5d9b640 1 -- 172.21.15.111:0/1540686468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fceb815c500 msgr2=0x7fceb80d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:11.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.557+0000 7fceb5d9b640 1 --2- 172.21.15.111:0/1540686468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fceb815c500 0x7fceb80d2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.557+0000 7fceb5d9b640 1 -- 172.21.15.111:0/1540686468 >> v1:172.21.15.188:6789/0 conn(0x7fceb80cde20 legacy=0x7fceb80d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:11.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.557+0000 7fceb5d9b640 1 -- 172.21.15.111:0/1540686468 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb80d0fd0 con 0x7fceb80d2590 2024-09-13T17:58:11.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.558+0000 7fceb5d9b640 1 --2- 172.21.15.111:0/1540686468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fceb80d2590 0x7fceb80cd8e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcea0009870 tx=0x7fcea002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abe7d4f353044a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:11.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.558+0000 7fceb559a640 1 -- 172.21.15.111:0/1540686468 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea0032440 con 0x7fceb80d2590 2024-09-13T17:58:11.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.558+0000 7fceb559a640 1 -- 172.21.15.111:0/1540686468 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea00325a0 con 0x7fceb80d2590 2024-09-13T17:58:11.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.558+0000 7fceb559a640 1 -- 172.21.15.111:0/1540686468 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea0032930 con 0x7fceb80d2590 2024-09-13T17:58:11.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.558+0000 7fcebc983640 1 -- 172.21.15.111:0/1540686468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fceb80d2590 msgr2=0x7fceb80cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:11.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.558+0000 7fcebc983640 1 --2- 172.21.15.111:0/1540686468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fceb80d2590 0x7fceb80cd8e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcea0009870 tx=0x7fcea002a270 comp rx=0 tx=0).stop 2024-09-13T17:58:11.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.558+0000 7fcebc983640 1 -- 172.21.15.111:0/1540686468 shutdown_connections 2024-09-13T17:58:11.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.558+0000 7fcebc983640 1 --2- 172.21.15.111:0/1540686468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fceb80d2590 0x7fceb80cd8e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.558+0000 7fcebc983640 1 --2- 172.21.15.111:0/1540686468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fceb815c500 0x7fceb80d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.558+0000 7fcebc983640 1 -- 172.21.15.111:0/1540686468 >> 172.21.15.111:0/1540686468 conn(0x7fceb8089f60 msgr2=0x7fceb809b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:11.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.558+0000 7fcebc983640 1 -- 172.21.15.111:0/1540686468 shutdown_connections 2024-09-13T17:58:11.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.558+0000 7fcebc983640 1 -- 172.21.15.111:0/1540686468 wait complete. 2024-09-13T17:58:11.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.560+0000 7fcebc983640 1 Processor -- start 2024-09-13T17:58:11.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.560+0000 7fcebc983640 1 -- start start 2024-09-13T17:58:11.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.560+0000 7fcebc983640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fceb80cde20 0x7fceb80c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.560+0000 7fcebc983640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fceb80d2590 0x7fceb81f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.560+0000 7fcebc983640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fceb815c500 0x7fceb81f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.560+0000 7fcebc983640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb80cc7c0 con 0x7fceb80cde20 2024-09-13T17:58:11.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.560+0000 7fcebc983640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb815dda0 con 0x7fceb815c500 2024-09-13T17:58:11.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.560+0000 7fcebc983640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb814a3e0 con 0x7fceb80d2590 2024-09-13T17:58:11.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.561+0000 7fceb659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fceb80cde20 0x7fceb80c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.561+0000 7fceb659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fceb80cde20 0x7fceb80c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51202/0 (socket says 172.21.15.111:51202) 2024-09-13T17:58:11.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.561+0000 7fceb659c640 1 -- 172.21.15.111:0/3743083781 learned_addr learned my addr 172.21.15.111:0/3743083781 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:11.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.561+0000 7fceb6d9d640 1 --2- 172.21.15.111:0/3743083781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fceb815c500 0x7fceb81f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.561+0000 7fceb5d9b640 1 --2- 172.21.15.111:0/3743083781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fceb80d2590 0x7fceb81f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.561+0000 7fceb659c640 1 -- 172.21.15.111:0/3743083781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fceb80d2590 msgr2=0x7fceb81f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:11.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.561+0000 7fceb659c640 1 --2- 172.21.15.111:0/3743083781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fceb80d2590 0x7fceb81f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.561+0000 7fceb659c640 1 -- 172.21.15.111:0/3743083781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fceb815c500 msgr2=0x7fceb81f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:11.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.561+0000 7fceb659c640 1 --2- 172.21.15.111:0/3743083781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fceb815c500 0x7fceb81f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.561+0000 7fceb659c640 1 -- 172.21.15.111:0/3743083781 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea00094d0 con 0x7fceb80cde20 2024-09-13T17:58:11.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.561+0000 7fceb659c640 1 --2- 172.21.15.111:0/3743083781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fceb80cde20 0x7fceb80c9500 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009db0 tx=0x7fcea800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:11.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.561+0000 7fceaeffd640 1 -- 172.21.15.111:0/3743083781 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8014480 con 0x7fceb80cde20 2024-09-13T17:58:11.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.561+0000 7fceaeffd640 1 -- 172.21.15.111:0/3743083781 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea800f040 con 0x7fceb80cde20 2024-09-13T17:58:11.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.561+0000 7fcebc983640 1 -- 172.21.15.111:0/3743083781 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb80cbf20 con 0x7fceb80cde20 2024-09-13T17:58:11.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.561+0000 7fceaeffd640 1 -- 172.21.15.111:0/3743083781 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8014800 con 0x7fceb80cde20 2024-09-13T17:58:11.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.561+0000 7fcebc983640 1 -- 172.21.15.111:0/3743083781 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb80cc4e0 con 0x7fceb80cde20 2024-09-13T17:58:11.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.562+0000 7fcebc983640 1 -- 172.21.15.111:0/3743083781 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb80cd8e0 con 0x7fceb80cde20 2024-09-13T17:58:11.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.563+0000 7fceaeffd640 1 -- 172.21.15.111:0/3743083781 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcea8014960 con 0x7fceb80cde20 2024-09-13T17:58:11.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.563+0000 7fceaeffd640 1 --2- 172.21.15.111:0/3743083781 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fce9806ac30 0x7fce9806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.563+0000 7fceb5d9b640 1 --2- 172.21.15.111:0/3743083781 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fce9806ac30 0x7fce9806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.563+0000 7fceaeffd640 1 -- 172.21.15.111:0/3743083781 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcea808a0a0 con 0x7fceb80cde20 2024-09-13T17:58:11.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.564+0000 7fceb5d9b640 1 --2- 172.21.15.111:0/3743083781 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fce9806ac30 0x7fce9806d0f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcea0034000 tx=0x7fcea002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:11.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.568+0000 7fceaeffd640 1 -- 172.21.15.111:0/3743083781 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fcea8055000 con 0x7fceb80cde20 2024-09-13T17:58:11.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.727+0000 7fcebc983640 1 -- 172.21.15.111:0/3743083781 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fceb8073380 con 0x7fceb80cde20 2024-09-13T17:58:11.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.727+0000 7fceaeffd640 1 -- 172.21.15.111:0/3743083781 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcea8058620 con 0x7fceb80cde20 2024-09-13T17:58:11.730 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:58:11.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.729+0000 7fcebc983640 1 -- 172.21.15.111:0/3743083781 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fce9806ac30 msgr2=0x7fce9806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:11.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.729+0000 7fcebc983640 1 --2- 172.21.15.111:0/3743083781 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fce9806ac30 0x7fce9806d0f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcea0034000 tx=0x7fcea002e000 comp rx=0 tx=0).stop 2024-09-13T17:58:11.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.729+0000 7fcebc983640 1 -- 172.21.15.111:0/3743083781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fceb80cde20 msgr2=0x7fceb80c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:11.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.729+0000 7fcebc983640 1 --2- 172.21.15.111:0/3743083781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fceb80cde20 0x7fceb80c9500 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009db0 tx=0x7fcea800c490 comp rx=0 tx=0).stop 2024-09-13T17:58:11.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.729+0000 7fcebc983640 1 -- 172.21.15.111:0/3743083781 shutdown_connections 2024-09-13T17:58:11.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.729+0000 7fcebc983640 1 --2- 172.21.15.111:0/3743083781 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fce9806ac30 0x7fce9806d0f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.729+0000 7fcebc983640 1 --2- 172.21.15.111:0/3743083781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fceb815c500 0x7fceb81f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.729+0000 7fcebc983640 1 --2- 172.21.15.111:0/3743083781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fceb80d2590 0x7fceb81f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.729+0000 7fcebc983640 1 --2- 172.21.15.111:0/3743083781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fceb80cde20 0x7fceb80c9500 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.729+0000 7fcebc983640 1 -- 172.21.15.111:0/3743083781 >> 172.21.15.111:0/3743083781 conn(0x7fceb8089f60 msgr2=0x7fceb806c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:11.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.729+0000 7fcebc983640 1 -- 172.21.15.111:0/3743083781 shutdown_connections 2024-09-13T17:58:11.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.729+0000 7fcebc983640 1 -- 172.21.15.111:0/3743083781 wait complete. 2024-09-13T17:58:11.742 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:58:11.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.842+0000 7f4d9ecd7640 1 Processor -- start 2024-09-13T17:58:11.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.842+0000 7f4d9ecd7640 1 -- start start 2024-09-13T17:58:11.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.842+0000 7f4d9ecd7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4d98158b00 0x7f4d980ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.842+0000 7f4d9ecd7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4d980b2880 0x7f4d9806c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.843+0000 7f4d9ecd7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d980b2d10 con 0x7f4d9806c190 2024-09-13T17:58:11.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.843+0000 7f4d9ecd7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d9806cb60 con 0x7f4d980b2880 2024-09-13T17:58:11.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.843+0000 7f4d9ecd7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d980b2ff0 con 0x7f4d98158b00 2024-09-13T17:58:11.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.843+0000 7f4d9d274640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4d980b2880 0x7f4d9806c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.843+0000 7f4d9d274640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4d980b2880 0x7f4d9806c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51216/0 (socket says 172.21.15.111:51216) 2024-09-13T17:58:11.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.843+0000 7f4d9d274640 1 -- 172.21.15.111:0/3139585067 learned_addr learned my addr 172.21.15.111:0/3139585067 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:11.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.843+0000 7f4d9ca73640 1 --2- 172.21.15.111:0/3139585067 >> v2:172.21.15.188:3300/0 conn(0x7f4d98158b00 0x7f4d980ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.843+0000 7f4d9d274640 1 -- 172.21.15.111:0/3139585067 >> v2:172.21.15.188:3300/0 conn(0x7f4d98158b00 msgr2=0x7f4d980ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:11.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.843+0000 7f4d9d274640 1 --2- 172.21.15.111:0/3139585067 >> v2:172.21.15.188:3300/0 conn(0x7f4d98158b00 0x7f4d980ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.843+0000 7f4d9d274640 1 -- 172.21.15.111:0/3139585067 >> v1:172.21.15.188:6789/0 conn(0x7f4d9806c190 legacy=0x7f4d980b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:11.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.843+0000 7f4d9d274640 1 -- 172.21.15.111:0/3139585067 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d98070310 con 0x7f4d980b2880 2024-09-13T17:58:11.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.844+0000 7f4d9d274640 1 --2- 172.21.15.111:0/3139585067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d980b2880 0x7f4d9806c620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4d9806ade0 tx=0x7f4d9002c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03eea0f83f94f23 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:11.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.844+0000 7f4d977fe640 1 -- 172.21.15.111:0/3139585067 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90009600 con 0x7f4d980b2880 2024-09-13T17:58:11.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.844+0000 7f4d977fe640 1 -- 172.21.15.111:0/3139585067 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d90009760 con 0x7f4d980b2880 2024-09-13T17:58:11.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.844+0000 7f4d977fe640 1 -- 172.21.15.111:0/3139585067 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90009af0 con 0x7f4d980b2880 2024-09-13T17:58:11.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.844+0000 7f4d9ecd7640 1 -- 172.21.15.111:0/3139585067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d980b2880 msgr2=0x7f4d9806c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:11.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.844+0000 7f4d9ecd7640 1 --2- 172.21.15.111:0/3139585067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d980b2880 0x7f4d9806c620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4d9806ade0 tx=0x7f4d9002c900 comp rx=0 tx=0).stop 2024-09-13T17:58:11.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.844+0000 7f4d9ecd7640 1 -- 172.21.15.111:0/3139585067 shutdown_connections 2024-09-13T17:58:11.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.844+0000 7f4d9ecd7640 1 --2- 172.21.15.111:0/3139585067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d980b2880 0x7f4d9806c620 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.844+0000 7f4d9ecd7640 1 --2- 172.21.15.111:0/3139585067 >> v2:172.21.15.188:3300/0 conn(0x7f4d98158b00 0x7f4d980ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.844+0000 7f4d9ecd7640 1 -- 172.21.15.111:0/3139585067 >> 172.21.15.111:0/3139585067 conn(0x7f4d9806a0b0 msgr2=0x7f4d9806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:11.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.844+0000 7f4d9ecd7640 1 -- 172.21.15.111:0/3139585067 shutdown_connections 2024-09-13T17:58:11.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.844+0000 7f4d9ecd7640 1 -- 172.21.15.111:0/3139585067 wait complete. 2024-09-13T17:58:11.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.845+0000 7f4d9ecd7640 1 Processor -- start 2024-09-13T17:58:11.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.846+0000 7f4d9ecd7640 1 -- start start 2024-09-13T17:58:11.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.846+0000 7f4d9ecd7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d9806c190 0x7f4d98097890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.846+0000 7f4d9ecd7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d98158b00 0x7f4d981e12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.846+0000 7f4d9ecd7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d980cb3b0 0x7f4d980b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.846+0000 7f4d9ecd7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d981e2830 con 0x7f4d98158b00 2024-09-13T17:58:11.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.846+0000 7f4d9ecd7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d981e1c20 con 0x7f4d9806c190 2024-09-13T17:58:11.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.846+0000 7f4d9ecd7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d981e1f00 con 0x7f4d980cb3b0 2024-09-13T17:58:11.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.847+0000 7f4d97fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d98158b00 0x7f4d981e12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.847+0000 7f4d97fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d98158b00 0x7f4d981e12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51226/0 (socket says 172.21.15.111:51226) 2024-09-13T17:58:11.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.847+0000 7f4d97fff640 1 -- 172.21.15.111:0/4092353506 learned_addr learned my addr 172.21.15.111:0/4092353506 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:11.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.847+0000 7f4d9ca73640 1 --2- 172.21.15.111:0/4092353506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d9806c190 0x7f4d98097890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.847+0000 7f4d9d274640 1 --2- 172.21.15.111:0/4092353506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d980cb3b0 0x7f4d980b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.847+0000 7f4d97fff640 1 -- 172.21.15.111:0/4092353506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d980cb3b0 msgr2=0x7f4d980b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:11.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.847+0000 7f4d97fff640 1 --2- 172.21.15.111:0/4092353506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d980cb3b0 0x7f4d980b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.847+0000 7f4d97fff640 1 -- 172.21.15.111:0/4092353506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d9806c190 msgr2=0x7f4d98097890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:11.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.847+0000 7f4d97fff640 1 --2- 172.21.15.111:0/4092353506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d9806c190 0x7f4d98097890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.847+0000 7f4d97fff640 1 -- 172.21.15.111:0/4092353506 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9000bd40 con 0x7f4d98158b00 2024-09-13T17:58:11.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.847+0000 7f4d97fff640 1 --2- 172.21.15.111:0/4092353506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d98158b00 0x7f4d981e12f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400c430 tx=0x7f4d8400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:11.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.847+0000 7f4d957fa640 1 -- 172.21.15.111:0/4092353506 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d84011790 con 0x7f4d98158b00 2024-09-13T17:58:11.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.847+0000 7f4d957fa640 1 -- 172.21.15.111:0/4092353506 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d840118f0 con 0x7f4d98158b00 2024-09-13T17:58:11.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.847+0000 7f4d9ecd7640 1 -- 172.21.15.111:0/4092353506 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d981e21e0 con 0x7f4d98158b00 2024-09-13T17:58:11.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.847+0000 7f4d957fa640 1 -- 172.21.15.111:0/4092353506 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d84011ce0 con 0x7f4d98158b00 2024-09-13T17:58:11.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.847+0000 7f4d9ecd7640 1 -- 172.21.15.111:0/4092353506 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d98097dd0 con 0x7f4d98158b00 2024-09-13T17:58:11.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.848+0000 7f4d9ecd7640 1 -- 172.21.15.111:0/4092353506 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d981e29e0 con 0x7f4d98158b00 2024-09-13T17:58:11.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.849+0000 7f4d957fa640 1 -- 172.21.15.111:0/4092353506 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4d8401c460 con 0x7f4d98158b00 2024-09-13T17:58:11.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.849+0000 7f4d957fa640 1 --2- 172.21.15.111:0/4092353506 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d7c06ac30 0x7f4d7c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.849+0000 7f4d957fa640 1 -- 172.21.15.111:0/4092353506 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4d84089500 con 0x7f4d98158b00 2024-09-13T17:58:11.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.849+0000 7f4d9ca73640 1 --2- 172.21.15.111:0/4092353506 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d7c06ac30 0x7f4d7c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.850+0000 7f4d9ca73640 1 --2- 172.21.15.111:0/4092353506 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d7c06ac30 0x7f4d7c06d0f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0079a0 tx=0x7f4d8c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:11.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:11.853+0000 7f4d957fa640 1 -- 172.21.15.111:0/4092353506 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4d981e29e0 con 0x7f4d98158b00 2024-09-13T17:58:12.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.015+0000 7f4d9ecd7640 1 -- 172.21.15.111:0/4092353506 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d981e2970 con 0x7f4d98158b00 2024-09-13T17:58:12.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.015+0000 7f4d957fa640 1 -- 172.21.15.111:0/4092353506 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4d840543d0 con 0x7f4d98158b00 2024-09-13T17:58:12.019 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:12.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.017+0000 7f4d9ecd7640 1 -- 172.21.15.111:0/4092353506 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d7c06ac30 msgr2=0x7f4d7c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.017+0000 7f4d9ecd7640 1 --2- 172.21.15.111:0/4092353506 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d7c06ac30 0x7f4d7c06d0f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0079a0 tx=0x7f4d8c008040 comp rx=0 tx=0).stop 2024-09-13T17:58:12.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.017+0000 7f4d9ecd7640 1 -- 172.21.15.111:0/4092353506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d98158b00 msgr2=0x7f4d981e12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.017+0000 7f4d9ecd7640 1 --2- 172.21.15.111:0/4092353506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d98158b00 0x7f4d981e12f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400c430 tx=0x7f4d8400c8f0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.017+0000 7f4d9ecd7640 1 -- 172.21.15.111:0/4092353506 shutdown_connections 2024-09-13T17:58:12.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.017+0000 7f4d9ecd7640 1 --2- 172.21.15.111:0/4092353506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d980cb3b0 0x7f4d980b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.017+0000 7f4d9ecd7640 1 --2- 172.21.15.111:0/4092353506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d98158b00 0x7f4d981e12f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.017+0000 7f4d9ecd7640 1 --2- 172.21.15.111:0/4092353506 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d7c06ac30 0x7f4d7c06d0f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.017+0000 7f4d9ecd7640 1 --2- 172.21.15.111:0/4092353506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d9806c190 0x7f4d98097890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.017+0000 7f4d9ecd7640 1 -- 172.21.15.111:0/4092353506 >> 172.21.15.111:0/4092353506 conn(0x7f4d9806a0b0 msgr2=0x7f4d980730d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:12.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.017+0000 7f4d9ecd7640 1 -- 172.21.15.111:0/4092353506 shutdown_connections 2024-09-13T17:58:12.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.017+0000 7f4d9ecd7640 1 -- 172.21.15.111:0/4092353506 wait complete. 2024-09-13T17:58:12.030 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:58:12.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.129+0000 7f31b98c9640 1 Processor -- start 2024-09-13T17:58:12.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.129+0000 7f31b98c9640 1 -- start start 2024-09-13T17:58:12.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.130+0000 7f31b98c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31b40b1370 0x7f31b40b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.130+0000 7f31b98c9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b40d1f40 con 0x7f31b40d1650 2024-09-13T17:58:12.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.130+0000 7f31b98c9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b40d2190 con 0x7f31b40cd500 2024-09-13T17:58:12.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.130+0000 7f31b98c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b40ccb20 con 0x7f31b40b1370 2024-09-13T17:58:12.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.130+0000 7f31b37fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f31b40d1650 0x7f31b40b1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58884/0 (socket says 172.21.15.111:58884) 2024-09-13T17:58:12.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.130+0000 7f31b37fe640 1 -- 172.21.15.111:0/1271329424 learned_addr learned my addr 172.21.15.111:0/1271329424 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:12.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.130+0000 7f31abfff640 1 --2- 172.21.15.111:0/1271329424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31b40b1370 0x7f31b40b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.130+0000 7f31b27fc640 1 -- 172.21.15.111:0/1271329424 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190879074 0 0) 0x7f31b40d1f40 con 0x7f31b40d1650 2024-09-13T17:58:12.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.130+0000 7f31b27fc640 1 -- 172.21.15.111:0/1271329424 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31a0003520 con 0x7f31b40d1650 2024-09-13T17:58:12.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.130+0000 7f31b27fc640 1 -- 172.21.15.111:0/1271329424 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764505471 0 0) 0x7f31b40d2190 con 0x7f31b40cd500 2024-09-13T17:58:12.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.130+0000 7f31b27fc640 1 -- 172.21.15.111:0/1271329424 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31b40d1f40 con 0x7f31b40cd500 2024-09-13T17:58:12.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.130+0000 7f31b27fc640 1 -- 172.21.15.111:0/1271329424 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1684471088 0 0) 0x7f31a0003520 con 0x7f31b40d1650 2024-09-13T17:58:12.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.130+0000 7f31abfff640 1 -- 172.21.15.111:0/1271329424 >> v1:172.21.15.188:6789/0 conn(0x7f31b40cd500 legacy=0x7f31b40b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.130+0000 7f31abfff640 1 -- 172.21.15.111:0/1271329424 >> v1:172.21.15.111:6789/0 conn(0x7f31b40d1650 legacy=0x7f31b40b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.130+0000 7f31abfff640 1 -- 172.21.15.111:0/1271329424 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b40cce00 con 0x7f31b40b1370 2024-09-13T17:58:12.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.131+0000 7f31abfff640 1 --2- 172.21.15.111:0/1271329424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31b40b1370 0x7f31b40b3850 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f319c0097d0 tx=0x7f319c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f86088003183628e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:12.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.131+0000 7f31b27fc640 1 -- 172.21.15.111:0/1271329424 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac0019f0 con 0x7f31b40b1370 2024-09-13T17:58:12.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.131+0000 7f31b27fc640 1 -- 172.21.15.111:0/1271329424 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319c02ac30 con 0x7f31b40b1370 2024-09-13T17:58:12.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.131+0000 7f31b27fc640 1 -- 172.21.15.111:0/1271329424 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c032440 con 0x7f31b40b1370 2024-09-13T17:58:12.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.132+0000 7f31b98c9640 1 -- 172.21.15.111:0/1271329424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31b40b1370 msgr2=0x7f31b40b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.132+0000 7f31b98c9640 1 --2- 172.21.15.111:0/1271329424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31b40b1370 0x7f31b40b3850 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f319c0097d0 tx=0x7f319c02a1a0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.132+0000 7f31b98c9640 1 -- 172.21.15.111:0/1271329424 shutdown_connections 2024-09-13T17:58:12.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.132+0000 7f31b98c9640 1 --2- 172.21.15.111:0/1271329424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31b40b1370 0x7f31b40b3850 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.132+0000 7f31b98c9640 1 -- 172.21.15.111:0/1271329424 >> 172.21.15.111:0/1271329424 conn(0x7f31b4089f60 msgr2=0x7f31b40d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:12.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.132+0000 7f31b98c9640 1 -- 172.21.15.111:0/1271329424 shutdown_connections 2024-09-13T17:58:12.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.132+0000 7f31b98c9640 1 -- 172.21.15.111:0/1271329424 wait complete. 2024-09-13T17:58:12.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.133+0000 7f31b98c9640 1 Processor -- start 2024-09-13T17:58:12.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.133+0000 7f31b98c9640 1 -- start start 2024-09-13T17:58:12.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31b98c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31b40cd500 0x7f31b415b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31b98c9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31b40d1650 0x7f31b4150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31b98c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31b415a3b0 0x7f31b415d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31b98c9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b415be70 con 0x7f31b40d1650 2024-09-13T17:58:12.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31b98c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b41510a0 con 0x7f31b415a3b0 2024-09-13T17:58:12.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31b98c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b415a890 con 0x7f31b40cd500 2024-09-13T17:58:12.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31abfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31b40d1650 0x7f31b4150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31abfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31b40d1650 0x7f31b4150b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51240/0 (socket says 172.21.15.111:51240) 2024-09-13T17:58:12.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31abfff640 1 -- 172.21.15.111:0/3277780245 learned_addr learned my addr 172.21.15.111:0/3277780245 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:12.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31b37fe640 1 --2- 172.21.15.111:0/3277780245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31b415a3b0 0x7f31b415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31b2ffd640 1 --2- 172.21.15.111:0/3277780245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31b40cd500 0x7f31b415b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31abfff640 1 -- 172.21.15.111:0/3277780245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31b40cd500 msgr2=0x7f31b415b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31abfff640 1 --2- 172.21.15.111:0/3277780245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31b40cd500 0x7f31b415b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31abfff640 1 -- 172.21.15.111:0/3277780245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31b415a3b0 msgr2=0x7f31b415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31abfff640 1 --2- 172.21.15.111:0/3277780245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31b415a3b0 0x7f31b415d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31abfff640 1 -- 172.21.15.111:0/3277780245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c009400 con 0x7f31b40d1650 2024-09-13T17:58:12.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31b2ffd640 1 --2- 172.21.15.111:0/3277780245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31b40cd500 0x7f31b415b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:12.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.134+0000 7f31abfff640 1 --2- 172.21.15.111:0/3277780245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31b40d1650 0x7f31b4150b60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f319c0097d0 tx=0x7f319c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:12.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.135+0000 7f31b0ff9640 1 -- 172.21.15.111:0/3277780245 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c03c820 con 0x7f31b40d1650 2024-09-13T17:58:12.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.135+0000 7f31b0ff9640 1 -- 172.21.15.111:0/3277780245 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319c03c980 con 0x7f31b40d1650 2024-09-13T17:58:12.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.135+0000 7f31b98c9640 1 -- 172.21.15.111:0/3277780245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b415ab70 con 0x7f31b40d1650 2024-09-13T17:58:12.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.135+0000 7f31b0ff9640 1 -- 172.21.15.111:0/3277780245 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c03ccf0 con 0x7f31b40d1650 2024-09-13T17:58:12.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.135+0000 7f31b98c9640 1 -- 172.21.15.111:0/3277780245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b415b130 con 0x7f31b40d1650 2024-09-13T17:58:12.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.136+0000 7f31b98c9640 1 -- 172.21.15.111:0/3277780245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b40d2190 con 0x7f31b40d1650 2024-09-13T17:58:12.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.136+0000 7f31b0ff9640 1 -- 172.21.15.111:0/3277780245 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f319c039050 con 0x7f31b40d1650 2024-09-13T17:58:12.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.137+0000 7f31b0ff9640 1 --2- 172.21.15.111:0/3277780245 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f319406ac30 0x7f319406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.137+0000 7f31b0ff9640 1 -- 172.21.15.111:0/3277780245 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f319c0a9fe0 con 0x7f31b40d1650 2024-09-13T17:58:12.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.137+0000 7f31b2ffd640 1 --2- 172.21.15.111:0/3277780245 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f319406ac30 0x7f319406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.137+0000 7f31b2ffd640 1 --2- 172.21.15.111:0/3277780245 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f319406ac30 0x7f319406d0f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f31a40011e0 tx=0x7f31a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:12.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.141+0000 7f31b0ff9640 1 -- 172.21.15.111:0/3277780245 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f319c074fb0 con 0x7f31b40d1650 2024-09-13T17:58:12.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.302+0000 7f31b98c9640 1 -- 172.21.15.111:0/3277780245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f31b40b4ed0 con 0x7f31b40d1650 2024-09-13T17:58:12.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.302+0000 7f31b0ff9640 1 -- 172.21.15.111:0/3277780245 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f319c0370a0 con 0x7f31b40d1650 2024-09-13T17:58:12.304 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:58:12.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.304+0000 7f31b98c9640 1 -- 172.21.15.111:0/3277780245 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f319406ac30 msgr2=0x7f319406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.304+0000 7f31b98c9640 1 --2- 172.21.15.111:0/3277780245 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f319406ac30 0x7f319406d0f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f31a40011e0 tx=0x7f31a4009040 comp rx=0 tx=0).stop 2024-09-13T17:58:12.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.304+0000 7f31b98c9640 1 -- 172.21.15.111:0/3277780245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31b40d1650 msgr2=0x7f31b4150b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.304+0000 7f31b98c9640 1 --2- 172.21.15.111:0/3277780245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31b40d1650 0x7f31b4150b60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f319c0097d0 tx=0x7f319c0046c0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.304+0000 7f31b98c9640 1 -- 172.21.15.111:0/3277780245 shutdown_connections 2024-09-13T17:58:12.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.304+0000 7f31b98c9640 1 --2- 172.21.15.111:0/3277780245 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f319406ac30 0x7f319406d0f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.304+0000 7f31b98c9640 1 --2- 172.21.15.111:0/3277780245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31b415a3b0 0x7f31b415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.304+0000 7f31b98c9640 1 --2- 172.21.15.111:0/3277780245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31b40d1650 0x7f31b4150b60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.304+0000 7f31b98c9640 1 --2- 172.21.15.111:0/3277780245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31b40cd500 0x7f31b415b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.304+0000 7f31b98c9640 1 -- 172.21.15.111:0/3277780245 >> 172.21.15.111:0/3277780245 conn(0x7f31b4089f60 msgr2=0x7f31b406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:12.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.304+0000 7f31b98c9640 1 -- 172.21.15.111:0/3277780245 shutdown_connections 2024-09-13T17:58:12.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.304+0000 7f31b98c9640 1 -- 172.21.15.111:0/3277780245 wait complete. 2024-09-13T17:58:12.317 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:58:12.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.416+0000 7f08a73a0640 1 Processor -- start 2024-09-13T17:58:12.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.416+0000 7f08a73a0640 1 -- start start 2024-09-13T17:58:12.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.417+0000 7f08a73a0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f08a006eb50 0x7f08a00ac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.417+0000 7f08a73a0640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a00b36e0 con 0x7f08a0160480 2024-09-13T17:58:12.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.417+0000 7f08a73a0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a006aee0 con 0x7f08a00acd50 2024-09-13T17:58:12.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.417+0000 7f08a73a0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a00b2ad0 con 0x7f08a006eb50 2024-09-13T17:58:12.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.417+0000 7f08a639e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f08a006eb50 0x7f08a00ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.417+0000 7f08a6b9f640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f08a0160480 0x7f08a01529c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33486/0 (socket says 172.21.15.111:33486) 2024-09-13T17:58:12.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.417+0000 7f08a6b9f640 1 -- 172.21.15.111:0/1373304824 learned_addr learned my addr 172.21.15.111:0/1373304824 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:12.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.417+0000 7f08a639e640 1 -- 172.21.15.111:0/1373304824 >> v1:172.21.15.188:6789/0 conn(0x7f08a00acd50 legacy=0x7f08a015fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:12.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.417+0000 7f08a639e640 1 -- 172.21.15.111:0/1373304824 >> v1:172.21.15.111:6789/0 conn(0x7f08a0160480 legacy=0x7f08a01529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.417+0000 7f08a639e640 1 -- 172.21.15.111:0/1373304824 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a00b2db0 con 0x7f08a006eb50 2024-09-13T17:58:12.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.418+0000 7f08a639e640 1 --2- 172.21.15.111:0/1373304824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08a006eb50 0x7f08a00ac810 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f089c0097a0 tx=0x7f089c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93cc04b69f652366 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:12.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.418+0000 7f08a539c640 1 -- 172.21.15.111:0/1373304824 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c032440 con 0x7f08a006eb50 2024-09-13T17:58:12.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.418+0000 7f08a539c640 1 -- 172.21.15.111:0/1373304824 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089c0325a0 con 0x7f08a006eb50 2024-09-13T17:58:12.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.418+0000 7f08a539c640 1 -- 172.21.15.111:0/1373304824 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c032930 con 0x7f08a006eb50 2024-09-13T17:58:12.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.419+0000 7f08a73a0640 1 -- 172.21.15.111:0/1373304824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08a006eb50 msgr2=0x7f08a00ac810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.419+0000 7f08a73a0640 1 --2- 172.21.15.111:0/1373304824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08a006eb50 0x7f08a00ac810 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f089c0097a0 tx=0x7f089c02a1e0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.419+0000 7f08a73a0640 1 -- 172.21.15.111:0/1373304824 shutdown_connections 2024-09-13T17:58:12.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.419+0000 7f08a73a0640 1 --2- 172.21.15.111:0/1373304824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08a006eb50 0x7f08a00ac810 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.419+0000 7f08a73a0640 1 -- 172.21.15.111:0/1373304824 >> 172.21.15.111:0/1373304824 conn(0x7f08a00ceb30 msgr2=0x7f08a00cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:12.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.419+0000 7f08a73a0640 1 -- 172.21.15.111:0/1373304824 shutdown_connections 2024-09-13T17:58:12.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.419+0000 7f08a73a0640 1 -- 172.21.15.111:0/1373304824 wait complete. 2024-09-13T17:58:12.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.420+0000 7f08a73a0640 1 Processor -- start 2024-09-13T17:58:12.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.420+0000 7f08a73a0640 1 -- start start 2024-09-13T17:58:12.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f08a73a0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08a006eb50 0x7f08a0143d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f08a73a0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08a00acd50 0x7f08a0156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f08a73a0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a0160480 0x7f08a0157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f08a73a0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a00cd920 con 0x7f08a00acd50 2024-09-13T17:58:12.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f08a73a0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a00ccca0 con 0x7f08a0160480 2024-09-13T17:58:12.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f08a73a0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a00ccf80 con 0x7f08a006eb50 2024-09-13T17:58:12.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f08a5b9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08a00acd50 0x7f08a0156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f08a5b9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08a00acd50 0x7f08a0156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43480/0 (socket says 172.21.15.111:43480) 2024-09-13T17:58:12.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f08a5b9d640 1 -- 172.21.15.111:0/1024095160 learned_addr learned my addr 172.21.15.111:0/1024095160 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:12.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f08a639e640 1 --2- 172.21.15.111:0/1024095160 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08a006eb50 0x7f08a0143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f08a6b9f640 1 --2- 172.21.15.111:0/1024095160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a0160480 0x7f08a0157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f08a5b9d640 1 -- 172.21.15.111:0/1024095160 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08a006eb50 msgr2=0x7f08a0143d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f08a5b9d640 1 --2- 172.21.15.111:0/1024095160 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08a006eb50 0x7f08a0143d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f08a5b9d640 1 -- 172.21.15.111:0/1024095160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a0160480 msgr2=0x7f08a0157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f08a5b9d640 1 --2- 172.21.15.111:0/1024095160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a0160480 0x7f08a0157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f08a5b9d640 1 -- 172.21.15.111:0/1024095160 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c009400 con 0x7f08a00acd50 2024-09-13T17:58:12.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f08a5b9d640 1 --2- 172.21.15.111:0/1024095160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08a00acd50 0x7f08a0156e40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f089000c430 tx=0x7f089000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:12.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f0896ffd640 1 -- 172.21.15.111:0/1024095160 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0890011790 con 0x7f08a00acd50 2024-09-13T17:58:12.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.421+0000 7f0896ffd640 1 -- 172.21.15.111:0/1024095160 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08900118f0 con 0x7f08a00acd50 2024-09-13T17:58:12.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.422+0000 7f08a73a0640 1 -- 172.21.15.111:0/1024095160 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a00b3cf0 con 0x7f08a00acd50 2024-09-13T17:58:12.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.422+0000 7f0896ffd640 1 -- 172.21.15.111:0/1024095160 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0890011ce0 con 0x7f08a00acd50 2024-09-13T17:58:12.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.422+0000 7f08a73a0640 1 -- 172.21.15.111:0/1024095160 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a00cd540 con 0x7f08a00acd50 2024-09-13T17:58:12.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.422+0000 7f08a73a0640 1 -- 172.21.15.111:0/1024095160 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a00b3b50 con 0x7f08a00acd50 2024-09-13T17:58:12.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.423+0000 7f0896ffd640 1 -- 172.21.15.111:0/1024095160 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f089001c460 con 0x7f08a00acd50 2024-09-13T17:58:12.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.423+0000 7f0896ffd640 1 --2- 172.21.15.111:0/1024095160 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f088406ac30 0x7f088406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.423+0000 7f0896ffd640 1 -- 172.21.15.111:0/1024095160 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0890089500 con 0x7f08a00acd50 2024-09-13T17:58:12.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.423+0000 7f08a639e640 1 --2- 172.21.15.111:0/1024095160 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f088406ac30 0x7f088406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.424+0000 7f08a639e640 1 --2- 172.21.15.111:0/1024095160 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f088406ac30 0x7f088406d0f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f089c034000 tx=0x7f089c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:12.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.428+0000 7f0896ffd640 1 -- 172.21.15.111:0/1024095160 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f08a00b3b50 con 0x7f08a00acd50 2024-09-13T17:58:12.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.588+0000 7f08a73a0640 1 -- 172.21.15.111:0/1024095160 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f08a00b3b50 con 0x7f08a00acd50 2024-09-13T17:58:12.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.589+0000 7f0896ffd640 1 -- 172.21.15.111:0/1024095160 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f08a00b3b50 con 0x7f08a00acd50 2024-09-13T17:58:12.591 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:12.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.590+0000 7f08a73a0640 1 -- 172.21.15.111:0/1024095160 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f088406ac30 msgr2=0x7f088406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.590+0000 7f08a73a0640 1 --2- 172.21.15.111:0/1024095160 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f088406ac30 0x7f088406d0f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f089c034000 tx=0x7f089c02e000 comp rx=0 tx=0).stop 2024-09-13T17:58:12.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.590+0000 7f08a73a0640 1 -- 172.21.15.111:0/1024095160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08a00acd50 msgr2=0x7f08a0156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.590+0000 7f08a73a0640 1 --2- 172.21.15.111:0/1024095160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08a00acd50 0x7f08a0156e40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f089000c430 tx=0x7f089000c8f0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.590+0000 7f08a73a0640 1 -- 172.21.15.111:0/1024095160 shutdown_connections 2024-09-13T17:58:12.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.590+0000 7f08a73a0640 1 --2- 172.21.15.111:0/1024095160 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f088406ac30 0x7f088406d0f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.590+0000 7f08a73a0640 1 --2- 172.21.15.111:0/1024095160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a0160480 0x7f08a0157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.590+0000 7f08a73a0640 1 --2- 172.21.15.111:0/1024095160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08a00acd50 0x7f08a0156e40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.590+0000 7f08a73a0640 1 --2- 172.21.15.111:0/1024095160 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08a006eb50 0x7f08a0143d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.590+0000 7f08a73a0640 1 -- 172.21.15.111:0/1024095160 >> 172.21.15.111:0/1024095160 conn(0x7f08a00ceb30 msgr2=0x7f08a006cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:12.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.591+0000 7f08a73a0640 1 -- 172.21.15.111:0/1024095160 shutdown_connections 2024-09-13T17:58:12.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.591+0000 7f08a73a0640 1 -- 172.21.15.111:0/1024095160 wait complete. 2024-09-13T17:58:12.603 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create' 2024-09-13T17:58:12.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.699+0000 7fed95533640 1 Processor -- start 2024-09-13T17:58:12.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.699+0000 7fed95533640 1 -- start start 2024-09-13T17:58:12.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.700+0000 7fed95533640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fed900cdd60 0x7fed900703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.700+0000 7fed95533640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed9015d480 con 0x7fed9015a3a0 2024-09-13T17:58:12.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.700+0000 7fed95533640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed900ac1f0 con 0x7fed9015d910 2024-09-13T17:58:12.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.700+0000 7fed95533640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed90071280 con 0x7fed900cdd60 2024-09-13T17:58:12.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.700+0000 7fed8effd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fed9015a3a0 0x7fed900cd650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33490/0 (socket says 172.21.15.111:33490) 2024-09-13T17:58:12.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.700+0000 7fed8effd640 1 -- 172.21.15.111:0/2883151537 learned_addr learned my addr 172.21.15.111:0/2883151537 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:12.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.700+0000 7fed8e7fc640 1 --2- 172.21.15.111:0/2883151537 >> v2:172.21.15.188:3300/0 conn(0x7fed900cdd60 0x7fed900703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.700+0000 7fed8dffb640 1 -- 172.21.15.111:0/2883151537 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 272255631 0 0) 0x7fed9015d480 con 0x7fed9015a3a0 2024-09-13T17:58:12.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.700+0000 7fed8dffb640 1 -- 172.21.15.111:0/2883151537 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed78003520 con 0x7fed9015a3a0 2024-09-13T17:58:12.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8dffb640 1 -- 172.21.15.111:0/2883151537 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2544201383 0 0) 0x7fed78003520 con 0x7fed9015a3a0 2024-09-13T17:58:12.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8dffb640 1 -- 172.21.15.111:0/2883151537 >> v2:172.21.15.188:3300/0 conn(0x7fed900cdd60 msgr2=0x7fed900703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:12.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8dffb640 1 --2- 172.21.15.111:0/2883151537 >> v2:172.21.15.188:3300/0 conn(0x7fed900cdd60 0x7fed900703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8dffb640 1 -- 172.21.15.111:0/2883151537 >> v1:172.21.15.188:6789/0 conn(0x7fed9015d910 legacy=0x7fed90160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8dffb640 1 -- 172.21.15.111:0/2883151537 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed90070fa0 con 0x7fed9015a3a0 2024-09-13T17:58:12.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8dffb640 1 -- 172.21.15.111:0/2883151537 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7fed800019f0 con 0x7fed9015a3a0 2024-09-13T17:58:12.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8dffb640 1 -- 172.21.15.111:0/2883151537 >> v1:172.21.15.111:6789/0 conn(0x7fed9015a3a0 legacy=0x7fed900cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8dffb640 1 --2- 172.21.15.111:0/2883151537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed78003970 0x7fed78003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8dffb640 1 --2- 172.21.15.111:0/2883151537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed780047c0 0x7fed78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8dffb640 1 --2- 172.21.15.111:0/2883151537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed78005180 0x7fed7800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8dffb640 1 -- 172.21.15.111:0/2883151537 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed90071280 con 0x7fed78003970 2024-09-13T17:58:12.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8dffb640 1 -- 172.21.15.111:0/2883151537 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7800c020 con 0x7fed78005180 2024-09-13T17:58:12.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8dffb640 1 -- 172.21.15.111:0/2883151537 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7800c1e0 con 0x7fed780047c0 2024-09-13T17:58:12.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8effd640 1 --2- 172.21.15.111:0/2883151537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed78003970 0x7fed78003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8f7fe640 1 --2- 172.21.15.111:0/2883151537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed78005180 0x7fed7800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8effd640 1 -- 172.21.15.111:0/2883151537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed780047c0 msgr2=0x7fed78004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:12.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8effd640 1 --2- 172.21.15.111:0/2883151537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed780047c0 0x7fed78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8effd640 1 -- 172.21.15.111:0/2883151537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed78005180 msgr2=0x7fed7800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:12.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8effd640 1 --2- 172.21.15.111:0/2883151537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed78005180 0x7fed7800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8effd640 1 -- 172.21.15.111:0/2883151537 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed78003520 con 0x7fed78003970 2024-09-13T17:58:12.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.701+0000 7fed8f7fe640 1 -- 172.21.15.111:0/2883151537 reap_dead start 2024-09-13T17:58:12.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.702+0000 7fed8effd640 1 --2- 172.21.15.111:0/2883151537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed78003970 0x7fed78003d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fed8000a830 tx=0x7fed780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aab1e2b85e7ba8d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:12.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.702+0000 7fed8dffb640 1 -- 172.21.15.111:0/2883151537 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8002ba50 con 0x7fed78003970 2024-09-13T17:58:12.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.702+0000 7fed8dffb640 1 -- 172.21.15.111:0/2883151537 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8002bbb0 con 0x7fed78003970 2024-09-13T17:58:12.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.702+0000 7fed8dffb640 1 -- 172.21.15.111:0/2883151537 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80034470 con 0x7fed78003970 2024-09-13T17:58:12.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.702+0000 7fed95533640 1 -- 172.21.15.111:0/2883151537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed78003970 msgr2=0x7fed78003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.702+0000 7fed95533640 1 --2- 172.21.15.111:0/2883151537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed78003970 0x7fed78003d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fed8000a830 tx=0x7fed780057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.702+0000 7fed95533640 1 -- 172.21.15.111:0/2883151537 shutdown_connections 2024-09-13T17:58:12.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.703+0000 7fed95533640 1 --2- 172.21.15.111:0/2883151537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed78003970 0x7fed78003d90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.703+0000 7fed95533640 1 -- 172.21.15.111:0/2883151537 >> 172.21.15.111:0/2883151537 conn(0x7fed90069ed0 msgr2=0x7fed9006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:12.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.703+0000 7fed95533640 1 -- 172.21.15.111:0/2883151537 shutdown_connections 2024-09-13T17:58:12.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.703+0000 7fed95533640 1 -- 172.21.15.111:0/2883151537 wait complete. 2024-09-13T17:58:12.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.704+0000 7fed95533640 1 Processor -- start 2024-09-13T17:58:12.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.704+0000 7fed95533640 1 -- start start 2024-09-13T17:58:12.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed95533640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed78003970 0x7fed90151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed95533640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed901567f0 0x7fed90157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed95533640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed90156c70 0x7fed90157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed95533640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed90152760 con 0x7fed90156c70 2024-09-13T17:58:12.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed95533640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed901f2260 con 0x7fed78003970 2024-09-13T17:58:12.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed95533640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed901f2540 con 0x7fed901567f0 2024-09-13T17:58:12.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed8f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed90156c70 0x7fed90157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed8f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed90156c70 0x7fed90157740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43488/0 (socket says 172.21.15.111:43488) 2024-09-13T17:58:12.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed8f7fe640 1 -- 172.21.15.111:0/3260649691 learned_addr learned my addr 172.21.15.111:0/3260649691 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:12.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed8e7fc640 1 --2- 172.21.15.111:0/3260649691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed901567f0 0x7fed90157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed8effd640 1 --2- 172.21.15.111:0/3260649691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed78003970 0x7fed90151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed8f7fe640 1 -- 172.21.15.111:0/3260649691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed901567f0 msgr2=0x7fed90157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed8f7fe640 1 --2- 172.21.15.111:0/3260649691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed901567f0 0x7fed90157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed8f7fe640 1 -- 172.21.15.111:0/3260649691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed78003970 msgr2=0x7fed90151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:12.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed8f7fe640 1 --2- 172.21.15.111:0/3260649691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed78003970 0x7fed90151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:12.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed8f7fe640 1 -- 172.21.15.111:0/3260649691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed901f2820 con 0x7fed90156c70 2024-09-13T17:58:12.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed8f7fe640 1 --2- 172.21.15.111:0/3260649691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed90156c70 0x7fed90157740 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fed9006ad40 tx=0x7fed8800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:12.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed777fe640 1 -- 172.21.15.111:0/3260649691 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8800eec0 con 0x7fed90156c70 2024-09-13T17:58:12.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed777fe640 1 -- 172.21.15.111:0/3260649691 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed88005070 con 0x7fed90156c70 2024-09-13T17:58:12.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed777fe640 1 -- 172.21.15.111:0/3260649691 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88005420 con 0x7fed90156c70 2024-09-13T17:58:12.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.705+0000 7fed95533640 1 -- 172.21.15.111:0/3260649691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed901f2b00 con 0x7fed90156c70 2024-09-13T17:58:12.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.706+0000 7fed95533640 1 -- 172.21.15.111:0/3260649691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed901f1ee0 con 0x7fed90156c70 2024-09-13T17:58:12.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.706+0000 7fed95533640 1 -- 172.21.15.111:0/3260649691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed901530f0 con 0x7fed90156c70 2024-09-13T17:58:12.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.707+0000 7fed777fe640 1 -- 172.21.15.111:0/3260649691 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fed88007030 con 0x7fed90156c70 2024-09-13T17:58:12.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.707+0000 7fed777fe640 1 --2- 172.21.15.111:0/3260649691 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fed7c06ac00 0x7fed7c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:12.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.707+0000 7fed8effd640 1 --2- 172.21.15.111:0/3260649691 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fed7c06ac00 0x7fed7c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.707+0000 7fed777fe640 1 -- 172.21.15.111:0/3260649691 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fed88088690 con 0x7fed90156c70 2024-09-13T17:58:12.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.708+0000 7fed8effd640 1 --2- 172.21.15.111:0/3260649691 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fed7c06ac00 0x7fed7c06d0c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fed80033000 tx=0x7fed8002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:12.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.711+0000 7fed777fe640 1 -- 172.21.15.111:0/3260649691 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fed901530f0 con 0x7fed90156c70 2024-09-13T17:58:12.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:12.934+0000 7fed95533640 1 -- 172.21.15.111:0/3260649691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]} v 0) v1 -- 0x7fed901530f0 con 0x7fed90156c70 2024-09-13T17:58:13.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.776+0000 7fed777fe640 1 -- 172.21.15.111:0/3260649691 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]}]=0 v67) v1 ==== 141+0+0 (secure 0 0 0) 0x7fed901530f0 con 0x7fed90156c70 2024-09-13T17:58:13.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.778+0000 7fed95533640 1 -- 172.21.15.111:0/3260649691 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fed7c06ac00 msgr2=0x7fed7c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:13.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.778+0000 7fed95533640 1 --2- 172.21.15.111:0/3260649691 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fed7c06ac00 0x7fed7c06d0c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fed80033000 tx=0x7fed8002e000 comp rx=0 tx=0).stop 2024-09-13T17:58:13.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.778+0000 7fed95533640 1 -- 172.21.15.111:0/3260649691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed90156c70 msgr2=0x7fed90157740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:13.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.778+0000 7fed95533640 1 --2- 172.21.15.111:0/3260649691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed90156c70 0x7fed90157740 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fed9006ad40 tx=0x7fed8800e530 comp rx=0 tx=0).stop 2024-09-13T17:58:13.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.778+0000 7fed95533640 1 -- 172.21.15.111:0/3260649691 shutdown_connections 2024-09-13T17:58:13.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.778+0000 7fed95533640 1 --2- 172.21.15.111:0/3260649691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed90156c70 0x7fed90157740 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:13.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.778+0000 7fed95533640 1 --2- 172.21.15.111:0/3260649691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed901567f0 0x7fed90157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:13.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.778+0000 7fed95533640 1 --2- 172.21.15.111:0/3260649691 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fed7c06ac00 0x7fed7c06d0c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:13.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.778+0000 7fed95533640 1 --2- 172.21.15.111:0/3260649691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed78003970 0x7fed90151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:13.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.778+0000 7fed95533640 1 -- 172.21.15.111:0/3260649691 >> 172.21.15.111:0/3260649691 conn(0x7fed90069ed0 msgr2=0x7fed9015c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:13.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.778+0000 7fed95533640 1 -- 172.21.15.111:0/3260649691 shutdown_connections 2024-09-13T17:58:13.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.778+0000 7fed95533640 1 -- 172.21.15.111:0/3260649691 wait complete. 2024-09-13T17:58:13.792 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T17:58:13.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.891+0000 7f6200fb5640 1 Processor -- start 2024-09-13T17:58:13.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.891+0000 7f6200fb5640 1 -- start start 2024-09-13T17:58:13.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.891+0000 7f6200fb5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61fc0b2b40 0x7f61fc0ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:13.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.891+0000 7f6200fb5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61fc0ccbe0 con 0x7f61fc0ce4f0 2024-09-13T17:58:13.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.891+0000 7f6200fb5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61fc0b1810 con 0x7f61fc159310 2024-09-13T17:58:13.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.891+0000 7f6200fb5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc0ad940 con 0x7f61fc0b2b40 2024-09-13T17:58:13.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61fad9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f61fc0ce4f0 0x7f61fc0b1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33494/0 (socket says 172.21.15.111:33494) 2024-09-13T17:58:13.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61fad9d640 1 -- 172.21.15.111:0/2655549462 learned_addr learned my addr 172.21.15.111:0/2655549462 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:13.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61f1d9b640 1 --2- 172.21.15.111:0/2655549462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61fc0b2b40 0x7f61fc0ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:13.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61f9d9b640 1 -- 172.21.15.111:0/2655549462 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1586421361 0 0) 0x7f61fc0ccbe0 con 0x7f61fc0ce4f0 2024-09-13T17:58:13.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61f9d9b640 1 -- 172.21.15.111:0/2655549462 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61e8003520 con 0x7f61fc0ce4f0 2024-09-13T17:58:13.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61f9d9b640 1 -- 172.21.15.111:0/2655549462 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 598465016 0 0) 0x7f61e8003520 con 0x7f61fc0ce4f0 2024-09-13T17:58:13.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61f9d9b640 1 -- 172.21.15.111:0/2655549462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61fc0b2b40 msgr2=0x7f61fc0ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:13.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61f9d9b640 1 --2- 172.21.15.111:0/2655549462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61fc0b2b40 0x7f61fc0ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:13.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61f9d9b640 1 -- 172.21.15.111:0/2655549462 >> v1:172.21.15.188:6789/0 conn(0x7f61fc159310 legacy=0x7f61fc0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:13.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61f9d9b640 1 -- 172.21.15.111:0/2655549462 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc06d9e0 con 0x7f61fc0ce4f0 2024-09-13T17:58:13.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61f9d9b640 1 -- 172.21.15.111:0/2655549462 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f61f40019f0 con 0x7f61fc0ce4f0 2024-09-13T17:58:13.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61f9d9b640 1 -- 172.21.15.111:0/2655549462 >> v1:172.21.15.111:6789/0 conn(0x7f61fc0ce4f0 legacy=0x7f61fc0b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:13.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61f9d9b640 1 --2- 172.21.15.111:0/2655549462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61fc0b2b40 0x7f61e80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:13.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61f9d9b640 1 --2- 172.21.15.111:0/2655549462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61e8003f20 0x7f61e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:13.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61f9d9b640 1 --2- 172.21.15.111:0/2655549462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61e8004d00 0x7f61e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:13.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61f9d9b640 1 -- 172.21.15.111:0/2655549462 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e800bb90 con 0x7f61e8004d00 2024-09-13T17:58:13.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61f9d9b640 1 -- 172.21.15.111:0/2655549462 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e800bd00 con 0x7f61e8003f20 2024-09-13T17:58:13.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61f9d9b640 1 -- 172.21.15.111:0/2655549462 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e800bec0 con 0x7f61fc0b2b40 2024-09-13T17:58:13.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61fad9d640 1 --2- 172.21.15.111:0/2655549462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61e8004d00 0x7f61e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:13.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61f1d9b640 1 --2- 172.21.15.111:0/2655549462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61e8003f20 0x7f61e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:13.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.892+0000 7f61fa59c640 1 --2- 172.21.15.111:0/2655549462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61fc0b2b40 0x7f61e80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:13.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.893+0000 7f61fad9d640 1 -- 172.21.15.111:0/2655549462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61fc0b2b40 msgr2=0x7f61e80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:13.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.893+0000 7f61fad9d640 1 --2- 172.21.15.111:0/2655549462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61fc0b2b40 0x7f61e80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:13.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.893+0000 7f61fad9d640 1 -- 172.21.15.111:0/2655549462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61e8003f20 msgr2=0x7f61e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:13.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.893+0000 7f61fad9d640 1 --2- 172.21.15.111:0/2655549462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61e8003f20 0x7f61e80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:13.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.893+0000 7f61fad9d640 1 -- 172.21.15.111:0/2655549462 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e8003520 con 0x7f61e8004d00 2024-09-13T17:58:13.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.893+0000 7f61fad9d640 1 --2- 172.21.15.111:0/2655549462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61e8004d00 0x7f61e8005180 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f61f400cac0 tx=0x7f61e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df17d78bb0496f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:13.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.893+0000 7f61f9d9b640 1 -- 172.21.15.111:0/2655549462 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f40072d0 con 0x7f61e8004d00 2024-09-13T17:58:13.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.893+0000 7f61f9d9b640 1 -- 172.21.15.111:0/2655549462 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f4030070 con 0x7f61e8004d00 2024-09-13T17:58:13.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.893+0000 7f61f9d9b640 1 -- 172.21.15.111:0/2655549462 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4007690 con 0x7f61e8004d00 2024-09-13T17:58:13.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.894+0000 7f6200fb5640 1 -- 172.21.15.111:0/2655549462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61e8004d00 msgr2=0x7f61e8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:13.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.894+0000 7f6200fb5640 1 --2- 172.21.15.111:0/2655549462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61e8004d00 0x7f61e8005180 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f61f400cac0 tx=0x7f61e80057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:13.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.894+0000 7f61fad9d640 1 -- 172.21.15.111:0/2655549462 reap_dead start 2024-09-13T17:58:13.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.894+0000 7f6200fb5640 1 -- 172.21.15.111:0/2655549462 shutdown_connections 2024-09-13T17:58:13.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.894+0000 7f6200fb5640 1 -- 172.21.15.111:0/2655549462 >> 172.21.15.111:0/2655549462 conn(0x7f61fc06a0b0 msgr2=0x7f61fc06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:13.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.894+0000 7f6200fb5640 1 -- 172.21.15.111:0/2655549462 shutdown_connections 2024-09-13T17:58:13.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.894+0000 7f6200fb5640 1 -- 172.21.15.111:0/2655549462 wait complete. 2024-09-13T17:58:13.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.895+0000 7f6200fb5640 1 Processor -- start 2024-09-13T17:58:13.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.895+0000 7f6200fb5640 1 -- start start 2024-09-13T17:58:13.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.896+0000 7f6200fb5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61e8004d00 0x7f61fc0b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:13.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.896+0000 7f6200fb5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61fc1596f0 0x7f61fc0c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:13.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.896+0000 7f6200fb5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61fc0affa0 0x7f61fc0b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:13.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.896+0000 7f6200fb5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc15a8a0 con 0x7f61fc0affa0 2024-09-13T17:58:13.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.896+0000 7f6200fb5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc0b0960 con 0x7f61fc1596f0 2024-09-13T17:58:13.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.896+0000 7f6200fb5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc159c20 con 0x7f61e8004d00 2024-09-13T17:58:13.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.896+0000 7f61fad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61fc0affa0 0x7f61fc0b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:13.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.896+0000 7f61fad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61fc0affa0 0x7f61fc0b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43512/0 (socket says 172.21.15.111:43512) 2024-09-13T17:58:13.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.896+0000 7f61fad9d640 1 -- 172.21.15.111:0/3901840055 learned_addr learned my addr 172.21.15.111:0/3901840055 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:13.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.896+0000 7f61f1d9b640 1 --2- 172.21.15.111:0/3901840055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61fc1596f0 0x7f61fc0c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:13.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.896+0000 7f61fa59c640 1 --2- 172.21.15.111:0/3901840055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61e8004d00 0x7f61fc0b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:13.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.897+0000 7f61fad9d640 1 -- 172.21.15.111:0/3901840055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61e8004d00 msgr2=0x7f61fc0b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:13.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.897+0000 7f61fad9d640 1 --2- 172.21.15.111:0/3901840055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61e8004d00 0x7f61fc0b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:13.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.897+0000 7f61fad9d640 1 -- 172.21.15.111:0/3901840055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61fc1596f0 msgr2=0x7f61fc0c4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:13.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.897+0000 7f61fad9d640 1 --2- 172.21.15.111:0/3901840055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61fc1596f0 0x7f61fc0c4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:13.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.897+0000 7f61fad9d640 1 -- 172.21.15.111:0/3901840055 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc159f00 con 0x7f61fc0affa0 2024-09-13T17:58:13.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.897+0000 7f61fad9d640 1 --2- 172.21.15.111:0/3901840055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61fc0affa0 0x7f61fc0b0420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f61f4001580 tx=0x7f61fc0cdb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:13.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.897+0000 7f61f37fe640 1 -- 172.21.15.111:0/3901840055 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f40070d0 con 0x7f61fc0affa0 2024-09-13T17:58:13.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.897+0000 7f61f37fe640 1 -- 172.21.15.111:0/3901840055 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f4007230 con 0x7f61fc0affa0 2024-09-13T17:58:13.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.897+0000 7f61f37fe640 1 -- 172.21.15.111:0/3901840055 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f40075a0 con 0x7f61fc0affa0 2024-09-13T17:58:13.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.897+0000 7f6200fb5640 1 -- 172.21.15.111:0/3901840055 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61fc15a1e0 con 0x7f61fc0affa0 2024-09-13T17:58:13.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.897+0000 7f6200fb5640 1 -- 172.21.15.111:0/3901840055 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61fc0ba320 con 0x7f61fc0affa0 2024-09-13T17:58:13.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.898+0000 7f6200fb5640 1 -- 172.21.15.111:0/3901840055 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61fc15a9e0 con 0x7f61fc0affa0 2024-09-13T17:58:13.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.898+0000 7f61f37fe640 1 -- 172.21.15.111:0/3901840055 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f61f4032030 con 0x7f61fc0affa0 2024-09-13T17:58:13.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.899+0000 7f61f37fe640 1 --2- 172.21.15.111:0/3901840055 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f61dc06ac30 0x7f61dc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:13.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.899+0000 7f61f37fe640 1 -- 172.21.15.111:0/3901840055 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f61f40aca60 con 0x7f61fc0affa0 2024-09-13T17:58:13.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.899+0000 7f61fa59c640 1 --2- 172.21.15.111:0/3901840055 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f61dc06ac30 0x7f61dc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:13.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.899+0000 7f61fa59c640 1 --2- 172.21.15.111:0/3901840055 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f61dc06ac30 0x7f61dc06d0f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0017e0 tx=0x7f61ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:13.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:13.903+0000 7f61f37fe640 1 -- 172.21.15.111:0/3901840055 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f61fc15a9e0 con 0x7f61fc0affa0 2024-09-13T17:58:14.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.064+0000 7f6200fb5640 1 -- 172.21.15.111:0/3901840055 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f61fc15a9e0 con 0x7f61fc0affa0 2024-09-13T17:58:14.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.068+0000 7f61f37fe640 1 -- 172.21.15.111:0/3901840055 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f61fc15a9e0 con 0x7f61fc0affa0 2024-09-13T17:58:14.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.070+0000 7f6200fb5640 1 -- 172.21.15.111:0/3901840055 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f61dc06ac30 msgr2=0x7f61dc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.070+0000 7f6200fb5640 1 --2- 172.21.15.111:0/3901840055 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f61dc06ac30 0x7f61dc06d0f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0017e0 tx=0x7f61ec009040 comp rx=0 tx=0).stop 2024-09-13T17:58:14.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.070+0000 7f6200fb5640 1 -- 172.21.15.111:0/3901840055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61fc0affa0 msgr2=0x7f61fc0b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.070+0000 7f6200fb5640 1 --2- 172.21.15.111:0/3901840055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61fc0affa0 0x7f61fc0b0420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f61f4001580 tx=0x7f61fc0cdb60 comp rx=0 tx=0).stop 2024-09-13T17:58:14.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.070+0000 7f6200fb5640 1 -- 172.21.15.111:0/3901840055 shutdown_connections 2024-09-13T17:58:14.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.070+0000 7f6200fb5640 1 --2- 172.21.15.111:0/3901840055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61fc0affa0 0x7f61fc0b0420 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.070+0000 7f6200fb5640 1 --2- 172.21.15.111:0/3901840055 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f61dc06ac30 0x7f61dc06d0f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.070+0000 7f6200fb5640 1 --2- 172.21.15.111:0/3901840055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61fc1596f0 0x7f61fc0c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.070+0000 7f6200fb5640 1 --2- 172.21.15.111:0/3901840055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61e8004d00 0x7f61fc0b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.070+0000 7f6200fb5640 1 -- 172.21.15.111:0/3901840055 >> 172.21.15.111:0/3901840055 conn(0x7f61fc06a0b0 msgr2=0x7f61fc0cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:14.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.070+0000 7f6200fb5640 1 -- 172.21.15.111:0/3901840055 shutdown_connections 2024-09-13T17:58:14.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.070+0000 7f6200fb5640 1 -- 172.21.15.111:0/3901840055 wait complete. 2024-09-13T17:58:14.084 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T17:58:14.084 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T17:58:14.131 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T17:58:14.179 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T17:58:14.179 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:58:14.180 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:58:14.196 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:58:14.197 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:58:14.254 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:14.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.390+0000 7f293795a640 1 Processor -- start 2024-09-13T17:58:14.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.390+0000 7f293795a640 1 -- start start 2024-09-13T17:58:14.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.391+0000 7f293795a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2930072110 0x7f2930070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.391+0000 7f293795a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f29300c8920 0x7f29300c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.391+0000 7f293795a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29300abdf0 con 0x7f2930071d30 2024-09-13T17:58:14.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.391+0000 7f293795a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29300c92f0 con 0x7f2930072110 2024-09-13T17:58:14.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.391+0000 7f293795a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29300b2280 con 0x7f29300c8920 2024-09-13T17:58:14.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.391+0000 7f29356f6640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f2930071d30 0x7f29300b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33502/0 (socket says 172.21.15.111:33502) 2024-09-13T17:58:14.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.391+0000 7f29356f6640 1 -- 172.21.15.111:0/704255872 learned_addr learned my addr 172.21.15.111:0/704255872 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:14.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.391+0000 7f2934ef5640 1 --2- 172.21.15.111:0/704255872 >> v2:172.21.15.111:3300/0 conn(0x7f2930072110 0x7f2930070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.391+0000 7f2934ef5640 1 -- 172.21.15.111:0/704255872 >> v2:172.21.15.188:3300/0 conn(0x7f29300c8920 msgr2=0x7f29300c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:14.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.391+0000 7f2934ef5640 1 --2- 172.21.15.111:0/704255872 >> v2:172.21.15.188:3300/0 conn(0x7f29300c8920 0x7f29300c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.391+0000 7f2934ef5640 1 -- 172.21.15.111:0/704255872 >> v1:172.21.15.111:6789/0 conn(0x7f2930071d30 legacy=0x7f29300b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.391+0000 7f2934ef5640 1 -- 172.21.15.111:0/704255872 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29300ab1e0 con 0x7f2930072110 2024-09-13T17:58:14.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.392+0000 7f2934ef5640 1 --2- 172.21.15.111:0/704255872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2930072110 0x7f2930070190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f293006ff50 tx=0x7f292002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b729654122a8d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:14.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.392+0000 7f2927fff640 1 -- 172.21.15.111:0/704255872 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920032440 con 0x7f2930072110 2024-09-13T17:58:14.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.392+0000 7f2927fff640 1 -- 172.21.15.111:0/704255872 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29200325a0 con 0x7f2930072110 2024-09-13T17:58:14.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.392+0000 7f2927fff640 1 -- 172.21.15.111:0/704255872 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920032930 con 0x7f2930072110 2024-09-13T17:58:14.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.393+0000 7f293795a640 1 -- 172.21.15.111:0/704255872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2930072110 msgr2=0x7f2930070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.393+0000 7f293795a640 1 --2- 172.21.15.111:0/704255872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2930072110 0x7f2930070190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f293006ff50 tx=0x7f292002a1a0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.393+0000 7f293795a640 1 -- 172.21.15.111:0/704255872 shutdown_connections 2024-09-13T17:58:14.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.393+0000 7f293795a640 1 --2- 172.21.15.111:0/704255872 >> v2:172.21.15.188:3300/0 conn(0x7f29300c8920 0x7f29300c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.393+0000 7f293795a640 1 --2- 172.21.15.111:0/704255872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2930072110 0x7f2930070190 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.393+0000 7f293795a640 1 -- 172.21.15.111:0/704255872 >> 172.21.15.111:0/704255872 conn(0x7f2930069ed0 msgr2=0x7f293006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:14.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.393+0000 7f293795a640 1 -- 172.21.15.111:0/704255872 shutdown_connections 2024-09-13T17:58:14.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.393+0000 7f293795a640 1 -- 172.21.15.111:0/704255872 wait complete. 2024-09-13T17:58:14.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.394+0000 7f293795a640 1 Processor -- start 2024-09-13T17:58:14.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.394+0000 7f293795a640 1 -- start start 2024-09-13T17:58:14.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f293795a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2930071d30 0x7f29301efcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f293795a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2930072110 0x7f2930157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f293795a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29300c8920 0x7f2930158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f293795a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29301f0230 con 0x7f29300c8920 2024-09-13T17:58:14.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f293795a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293015b180 con 0x7f2930071d30 2024-09-13T17:58:14.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f293795a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29300c6020 con 0x7f2930072110 2024-09-13T17:58:14.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f2935ef7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29300c8920 0x7f2930158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f2935ef7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29300c8920 0x7f2930158330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43530/0 (socket says 172.21.15.111:43530) 2024-09-13T17:58:14.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f2935ef7640 1 -- 172.21.15.111:0/1152494669 learned_addr learned my addr 172.21.15.111:0/1152494669 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:14.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f2934ef5640 1 --2- 172.21.15.111:0/1152494669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2930072110 0x7f2930157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f29356f6640 1 --2- 172.21.15.111:0/1152494669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2930071d30 0x7f29301efcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f2935ef7640 1 -- 172.21.15.111:0/1152494669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2930072110 msgr2=0x7f2930157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f2935ef7640 1 --2- 172.21.15.111:0/1152494669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2930072110 0x7f2930157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f2935ef7640 1 -- 172.21.15.111:0/1152494669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2930071d30 msgr2=0x7f29301efcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f2935ef7640 1 --2- 172.21.15.111:0/1152494669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2930071d30 0x7f29301efcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f2935ef7640 1 -- 172.21.15.111:0/1152494669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2920009400 con 0x7f29300c8920 2024-09-13T17:58:14.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f2934ef5640 1 --2- 172.21.15.111:0/1152494669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2930072110 0x7f2930157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:14.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f2935ef7640 1 --2- 172.21.15.111:0/1152494669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29300c8920 0x7f2930158330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f292800b970 tx=0x7f292800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:14.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f2925ffb640 1 -- 172.21.15.111:0/1152494669 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292800cb40 con 0x7f29300c8920 2024-09-13T17:58:14.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f2925ffb640 1 -- 172.21.15.111:0/1152494669 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292800ccc0 con 0x7f29300c8920 2024-09-13T17:58:14.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f293795a640 1 -- 172.21.15.111:0/1152494669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29300c62c0 con 0x7f29300c8920 2024-09-13T17:58:14.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.395+0000 7f2925ffb640 1 -- 172.21.15.111:0/1152494669 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928014680 con 0x7f29300c8920 2024-09-13T17:58:14.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.396+0000 7f293795a640 1 -- 172.21.15.111:0/1152494669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29300c6880 con 0x7f29300c8920 2024-09-13T17:58:14.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.397+0000 7f293795a640 1 -- 172.21.15.111:0/1152494669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2930070190 con 0x7f29300c8920 2024-09-13T17:58:14.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.397+0000 7f2925ffb640 1 -- 172.21.15.111:0/1152494669 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f292801d3b0 con 0x7f29300c8920 2024-09-13T17:58:14.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.397+0000 7f2925ffb640 1 --2- 172.21.15.111:0/1152494669 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f291406ac30 0x7f291406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.398+0000 7f29356f6640 1 --2- 172.21.15.111:0/1152494669 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f291406ac30 0x7f291406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.398+0000 7f2925ffb640 1 -- 172.21.15.111:0/1152494669 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2928089660 con 0x7f29300c8920 2024-09-13T17:58:14.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.398+0000 7f29356f6640 1 --2- 172.21.15.111:0/1152494669 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f291406ac30 0x7f291406d0f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f292c005880 tx=0x7f292c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:14.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.402+0000 7f2925ffb640 1 -- 172.21.15.111:0/1152494669 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f2928054480 con 0x7f29300c8920 2024-09-13T17:58:14.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.565+0000 7f293795a640 1 -- 172.21.15.111:0/1152494669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f293015cb20 con 0x7f29300c8920 2024-09-13T17:58:14.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.566+0000 7f2925ffb640 1 -- 172.21.15.111:0/1152494669 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18794 (secure 0 0 0) 0x7f2928057aa0 con 0x7f29300c8920 2024-09-13T17:58:14.568 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:14.568 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":26,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:07.749113+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:14.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.568+0000 7f293795a640 1 -- 172.21.15.111:0/1152494669 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f291406ac30 msgr2=0x7f291406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.568+0000 7f293795a640 1 --2- 172.21.15.111:0/1152494669 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f291406ac30 0x7f291406d0f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f292c005880 tx=0x7f292c009040 comp rx=0 tx=0).stop 2024-09-13T17:58:14.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.568+0000 7f293795a640 1 -- 172.21.15.111:0/1152494669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29300c8920 msgr2=0x7f2930158330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.568+0000 7f293795a640 1 --2- 172.21.15.111:0/1152494669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29300c8920 0x7f2930158330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f292800b970 tx=0x7f292800be30 comp rx=0 tx=0).stop 2024-09-13T17:58:14.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.568+0000 7f293795a640 1 -- 172.21.15.111:0/1152494669 shutdown_connections 2024-09-13T17:58:14.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.568+0000 7f293795a640 1 --2- 172.21.15.111:0/1152494669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29300c8920 0x7f2930158330 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.568+0000 7f293795a640 1 --2- 172.21.15.111:0/1152494669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2930072110 0x7f2930157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.568+0000 7f293795a640 1 --2- 172.21.15.111:0/1152494669 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f291406ac30 0x7f291406d0f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.568+0000 7f293795a640 1 --2- 172.21.15.111:0/1152494669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2930071d30 0x7f29301efcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.568+0000 7f293795a640 1 -- 172.21.15.111:0/1152494669 >> 172.21.15.111:0/1152494669 conn(0x7f2930069ed0 msgr2=0x7f29300ac280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:14.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.568+0000 7f293795a640 1 -- 172.21.15.111:0/1152494669 shutdown_connections 2024-09-13T17:58:14.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.568+0000 7f293795a640 1 -- 172.21.15.111:0/1152494669 wait complete. 2024-09-13T17:58:14.581 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:58:14.581 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:58:14.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.680+0000 7fb774f45640 1 Processor -- start 2024-09-13T17:58:14.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.680+0000 7fb774f45640 1 -- start start 2024-09-13T17:58:14.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.681+0000 7fb774f45640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb7700b2b40 0x7fb7700ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.681+0000 7fb774f45640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb7700ce4f0 0x7fb7700ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.681+0000 7fb774f45640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7700ccbe0 con 0x7fb770159310 2024-09-13T17:58:14.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.681+0000 7fb774f45640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7700ad940 con 0x7fb7700b2b40 2024-09-13T17:58:14.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.681+0000 7fb774f45640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7700b1810 con 0x7fb7700ce4f0 2024-09-13T17:58:14.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.681+0000 7fb767fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb7700b2b40 0x7fb7700ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.681+0000 7fb767fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb7700b2b40 0x7fb7700ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43540/0 (socket says 172.21.15.111:43540) 2024-09-13T17:58:14.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.681+0000 7fb767fff640 1 -- 172.21.15.111:0/425034492 learned_addr learned my addr 172.21.15.111:0/425034492 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:14.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.681+0000 7fb767fff640 1 -- 172.21.15.111:0/425034492 >> v2:172.21.15.188:3300/0 conn(0x7fb7700ce4f0 msgr2=0x7fb7700ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:14.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.681+0000 7fb76ed9d640 1 --2- 172.21.15.111:0/425034492 >> v2:172.21.15.188:3300/0 conn(0x7fb7700ce4f0 0x7fb7700ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.681+0000 7fb767fff640 1 --2- 172.21.15.111:0/425034492 >> v2:172.21.15.188:3300/0 conn(0x7fb7700ce4f0 0x7fb7700ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.681+0000 7fb767fff640 1 -- 172.21.15.111:0/425034492 >> v1:172.21.15.188:6789/0 conn(0x7fb770159310 legacy=0x7fb7700ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:14.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.681+0000 7fb767fff640 1 -- 172.21.15.111:0/425034492 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7700cee10 con 0x7fb7700b2b40 2024-09-13T17:58:14.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.681+0000 7fb76ed9d640 1 --2- 172.21.15.111:0/425034492 >> v2:172.21.15.188:3300/0 conn(0x7fb7700ce4f0 0x7fb7700ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:14.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.682+0000 7fb767fff640 1 --2- 172.21.15.111:0/425034492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7700b2b40 0x7fb7700ad400 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb7700b2fc0 tx=0x7fb75802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316602ecfafcd57a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:14.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.682+0000 7fb76dd9b640 1 -- 172.21.15.111:0/425034492 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758032440 con 0x7fb7700b2b40 2024-09-13T17:58:14.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.682+0000 7fb76dd9b640 1 -- 172.21.15.111:0/425034492 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7580325a0 con 0x7fb7700b2b40 2024-09-13T17:58:14.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.682+0000 7fb76dd9b640 1 -- 172.21.15.111:0/425034492 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758032930 con 0x7fb7700b2b40 2024-09-13T17:58:14.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.682+0000 7fb774f45640 1 -- 172.21.15.111:0/425034492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7700b2b40 msgr2=0x7fb7700ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.682+0000 7fb774f45640 1 --2- 172.21.15.111:0/425034492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7700b2b40 0x7fb7700ad400 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb7700b2fc0 tx=0x7fb75802a1a0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.682+0000 7fb774f45640 1 -- 172.21.15.111:0/425034492 shutdown_connections 2024-09-13T17:58:14.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.682+0000 7fb774f45640 1 --2- 172.21.15.111:0/425034492 >> v2:172.21.15.188:3300/0 conn(0x7fb7700ce4f0 0x7fb7700ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.682+0000 7fb774f45640 1 --2- 172.21.15.111:0/425034492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7700b2b40 0x7fb7700ad400 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.682+0000 7fb774f45640 1 -- 172.21.15.111:0/425034492 >> 172.21.15.111:0/425034492 conn(0x7fb77006a0b0 msgr2=0x7fb77006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:14.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.683+0000 7fb774f45640 1 -- 172.21.15.111:0/425034492 shutdown_connections 2024-09-13T17:58:14.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.683+0000 7fb774f45640 1 -- 172.21.15.111:0/425034492 wait complete. 2024-09-13T17:58:14.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.684+0000 7fb774f45640 1 Processor -- start 2024-09-13T17:58:14.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.684+0000 7fb774f45640 1 -- start start 2024-09-13T17:58:14.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.684+0000 7fb774f45640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7700b2b40 0x7fb770153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.684+0000 7fb774f45640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7700ce4f0 0x7fb770097820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb774f45640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb770159310 0x7fb7701520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb774f45640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb770154270 con 0x7fb7700ce4f0 2024-09-13T17:58:14.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb774f45640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb770154550 con 0x7fb770159310 2024-09-13T17:58:14.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb774f45640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb770154830 con 0x7fb7700b2b40 2024-09-13T17:58:14.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb767fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7700ce4f0 0x7fb770097820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb767fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7700ce4f0 0x7fb770097820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43552/0 (socket says 172.21.15.111:43552) 2024-09-13T17:58:14.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb767fff640 1 -- 172.21.15.111:0/1536758744 learned_addr learned my addr 172.21.15.111:0/1536758744 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:14.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb76e59c640 1 --2- 172.21.15.111:0/1536758744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7700b2b40 0x7fb770153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb76ed9d640 1 --2- 172.21.15.111:0/1536758744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb770159310 0x7fb7701520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb767fff640 1 -- 172.21.15.111:0/1536758744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7700b2b40 msgr2=0x7fb770153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb767fff640 1 --2- 172.21.15.111:0/1536758744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7700b2b40 0x7fb770153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb767fff640 1 -- 172.21.15.111:0/1536758744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb770159310 msgr2=0x7fb7701520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb767fff640 1 --2- 172.21.15.111:0/1536758744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb770159310 0x7fb7701520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb767fff640 1 -- 172.21.15.111:0/1536758744 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7580094d0 con 0x7fb7700ce4f0 2024-09-13T17:58:14.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb76ed9d640 1 --2- 172.21.15.111:0/1536758744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb770159310 0x7fb7701520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:14.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb767fff640 1 --2- 172.21.15.111:0/1536758744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7700ce4f0 0x7fb770097820 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb758000c00 tx=0x7fb7580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:14.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb766ffd640 1 -- 172.21.15.111:0/1536758744 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758032440 con 0x7fb7700ce4f0 2024-09-13T17:58:14.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb766ffd640 1 -- 172.21.15.111:0/1536758744 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb758030b80 con 0x7fb7700ce4f0 2024-09-13T17:58:14.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb774f45640 1 -- 172.21.15.111:0/1536758744 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb770154b10 con 0x7fb7700ce4f0 2024-09-13T17:58:14.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb766ffd640 1 -- 172.21.15.111:0/1536758744 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75803d420 con 0x7fb7700ce4f0 2024-09-13T17:58:14.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.685+0000 7fb774f45640 1 -- 172.21.15.111:0/1536758744 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb770097d60 con 0x7fb7700ce4f0 2024-09-13T17:58:14.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.686+0000 7fb774f45640 1 -- 172.21.15.111:0/1536758744 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb770155030 con 0x7fb7700ce4f0 2024-09-13T17:58:14.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.687+0000 7fb766ffd640 1 -- 172.21.15.111:0/1536758744 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb758039040 con 0x7fb7700ce4f0 2024-09-13T17:58:14.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.687+0000 7fb766ffd640 1 --2- 172.21.15.111:0/1536758744 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb75006ac00 0x7fb75006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.687+0000 7fb76e59c640 1 --2- 172.21.15.111:0/1536758744 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb75006ac00 0x7fb75006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.687+0000 7fb766ffd640 1 -- 172.21.15.111:0/1536758744 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb7580aa420 con 0x7fb7700ce4f0 2024-09-13T17:58:14.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.688+0000 7fb76e59c640 1 --2- 172.21.15.111:0/1536758744 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb75006ac00 0x7fb75006d0c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb760006fd0 tx=0x7fb760009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:14.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.691+0000 7fb766ffd640 1 -- 172.21.15.111:0/1536758744 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fb770155030 con 0x7fb7700ce4f0 2024-09-13T17:58:14.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.853+0000 7fb774f45640 1 -- 172.21.15.111:0/1536758744 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb770155030 con 0x7fb7700ce4f0 2024-09-13T17:58:14.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.853+0000 7fb766ffd640 1 -- 172.21.15.111:0/1536758744 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb770155030 con 0x7fb7700ce4f0 2024-09-13T17:58:14.855 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:58:14.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.855+0000 7fb774f45640 1 -- 172.21.15.111:0/1536758744 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb75006ac00 msgr2=0x7fb75006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.855+0000 7fb774f45640 1 --2- 172.21.15.111:0/1536758744 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb75006ac00 0x7fb75006d0c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb760006fd0 tx=0x7fb760009060 comp rx=0 tx=0).stop 2024-09-13T17:58:14.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.855+0000 7fb774f45640 1 -- 172.21.15.111:0/1536758744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7700ce4f0 msgr2=0x7fb770097820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.855+0000 7fb774f45640 1 --2- 172.21.15.111:0/1536758744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7700ce4f0 0x7fb770097820 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb758000c00 tx=0x7fb7580046c0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.855+0000 7fb774f45640 1 -- 172.21.15.111:0/1536758744 shutdown_connections 2024-09-13T17:58:14.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.855+0000 7fb774f45640 1 --2- 172.21.15.111:0/1536758744 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb75006ac00 0x7fb75006d0c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.855+0000 7fb774f45640 1 --2- 172.21.15.111:0/1536758744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb770159310 0x7fb7701520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.855+0000 7fb774f45640 1 --2- 172.21.15.111:0/1536758744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7700ce4f0 0x7fb770097820 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.855+0000 7fb774f45640 1 --2- 172.21.15.111:0/1536758744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7700b2b40 0x7fb770153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.855+0000 7fb774f45640 1 -- 172.21.15.111:0/1536758744 >> 172.21.15.111:0/1536758744 conn(0x7fb77006a0b0 msgr2=0x7fb7700ac5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:14.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.855+0000 7fb774f45640 1 -- 172.21.15.111:0/1536758744 shutdown_connections 2024-09-13T17:58:14.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.855+0000 7fb774f45640 1 -- 172.21.15.111:0/1536758744 wait complete. 2024-09-13T17:58:14.869 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:58:14.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.967+0000 7f5a6c725640 1 Processor -- start 2024-09-13T17:58:14.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.967+0000 7f5a6c725640 1 -- start start 2024-09-13T17:58:14.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a6c725640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a640b20a0 0x7f5a640ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a6c725640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6415a790 con 0x7f5a6415a3b0 2024-09-13T17:58:14.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a6c725640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a640b2680 con 0x7f5a640cea30 2024-09-13T17:58:14.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a6c725640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640ceec0 con 0x7f5a640b20a0 2024-09-13T17:58:14.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a6a4c1640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5a6415a3b0 0x7f5a640b1720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33518/0 (socket says 172.21.15.111:33518) 2024-09-13T17:58:14.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a6a4c1640 1 -- 172.21.15.111:0/1128094575 learned_addr learned my addr 172.21.15.111:0/1128094575 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:14.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a694bf640 1 -- 172.21.15.111:0/1128094575 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707087939 0 0) 0x7f5a6415a790 con 0x7f5a6415a3b0 2024-09-13T17:58:14.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a69cc0640 1 --2- 172.21.15.111:0/1128094575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a640b20a0 0x7f5a640ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a694bf640 1 -- 172.21.15.111:0/1128094575 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a50003520 con 0x7f5a6415a3b0 2024-09-13T17:58:14.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a694bf640 1 -- 172.21.15.111:0/1128094575 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1428692899 0 0) 0x7f5a50003520 con 0x7f5a6415a3b0 2024-09-13T17:58:14.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a694bf640 1 -- 172.21.15.111:0/1128094575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a640b20a0 msgr2=0x7f5a640ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:14.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a694bf640 1 --2- 172.21.15.111:0/1128094575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a640b20a0 0x7f5a640ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a694bf640 1 -- 172.21.15.111:0/1128094575 >> v1:172.21.15.188:6789/0 conn(0x7f5a640cea30 legacy=0x7f5a640ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a694bf640 1 -- 172.21.15.111:0/1128094575 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a640b2960 con 0x7f5a6415a3b0 2024-09-13T17:58:14.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a694bf640 1 -- 172.21.15.111:0/1128094575 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f5a600019f0 con 0x7f5a6415a3b0 2024-09-13T17:58:14.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a694bf640 1 -- 172.21.15.111:0/1128094575 >> v1:172.21.15.111:6789/0 conn(0x7f5a6415a3b0 legacy=0x7f5a640b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a694bf640 1 --2- 172.21.15.111:0/1128094575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a500039a0 0x7f5a50003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a694bf640 1 --2- 172.21.15.111:0/1128094575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a640b20a0 0x7f5a500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.968+0000 7f5a694bf640 1 --2- 172.21.15.111:0/1128094575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a50004d00 0x7f5a50005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.969+0000 7f5a694bf640 1 -- 172.21.15.111:0/1128094575 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640ceec0 con 0x7f5a500039a0 2024-09-13T17:58:14.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.969+0000 7f5a694bf640 1 -- 172.21.15.111:0/1128094575 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5000ba60 con 0x7f5a50004d00 2024-09-13T17:58:14.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.969+0000 7f5a694bf640 1 -- 172.21.15.111:0/1128094575 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5000bc20 con 0x7f5a640b20a0 2024-09-13T17:58:14.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.969+0000 7f5a6a4c1640 1 --2- 172.21.15.111:0/1128094575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a500039a0 0x7f5a50003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.969+0000 7f5a6acc2640 1 --2- 172.21.15.111:0/1128094575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a50004d00 0x7f5a50005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.969+0000 7f5a69cc0640 1 --2- 172.21.15.111:0/1128094575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a640b20a0 0x7f5a500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.969+0000 7f5a6a4c1640 1 -- 172.21.15.111:0/1128094575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a640b20a0 msgr2=0x7f5a500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:14.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.969+0000 7f5a6a4c1640 1 --2- 172.21.15.111:0/1128094575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a640b20a0 0x7f5a500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.969+0000 7f5a6a4c1640 1 -- 172.21.15.111:0/1128094575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a50004d00 msgr2=0x7f5a50005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:14.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.969+0000 7f5a6a4c1640 1 --2- 172.21.15.111:0/1128094575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a50004d00 0x7f5a50005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.969+0000 7f5a6a4c1640 1 -- 172.21.15.111:0/1128094575 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a50003520 con 0x7f5a500039a0 2024-09-13T17:58:14.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.969+0000 7f5a6acc2640 1 --2- 172.21.15.111:0/1128094575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a50004d00 0x7f5a50005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:14.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.970+0000 7f5a6a4c1640 1 --2- 172.21.15.111:0/1128094575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a500039a0 0x7f5a50003dc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000a800 tx=0x7f5a500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643b348144fcdf58 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:14.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.970+0000 7f5a694bf640 1 -- 172.21.15.111:0/1128094575 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6002ba10 con 0x7f5a500039a0 2024-09-13T17:58:14.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.970+0000 7f5a694bf640 1 -- 172.21.15.111:0/1128094575 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6002bb70 con 0x7f5a500039a0 2024-09-13T17:58:14.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.970+0000 7f5a694bf640 1 -- 172.21.15.111:0/1128094575 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60034470 con 0x7f5a500039a0 2024-09-13T17:58:14.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.970+0000 7f5a6c725640 1 -- 172.21.15.111:0/1128094575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a500039a0 msgr2=0x7f5a50003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.970+0000 7f5a6c725640 1 --2- 172.21.15.111:0/1128094575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a500039a0 0x7f5a50003dc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000a800 tx=0x7f5a500057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.970+0000 7f5a6acc2640 1 -- 172.21.15.111:0/1128094575 reap_dead start 2024-09-13T17:58:14.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.970+0000 7f5a6c725640 1 -- 172.21.15.111:0/1128094575 shutdown_connections 2024-09-13T17:58:14.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.970+0000 7f5a6c725640 1 -- 172.21.15.111:0/1128094575 >> 172.21.15.111:0/1128094575 conn(0x7f5a6406a0b0 msgr2=0x7f5a6406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:14.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.970+0000 7f5a6c725640 1 -- 172.21.15.111:0/1128094575 shutdown_connections 2024-09-13T17:58:14.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.970+0000 7f5a6c725640 1 -- 172.21.15.111:0/1128094575 wait complete. 2024-09-13T17:58:14.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.971+0000 7f5a6c725640 1 Processor -- start 2024-09-13T17:58:14.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.971+0000 7f5a6c725640 1 -- start start 2024-09-13T17:58:14.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.972+0000 7f5a6c725640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a500039a0 0x7f5a64097990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.972+0000 7f5a6c725640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a640afd40 0x7f5a640b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.972+0000 7f5a6c725640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a640c40a0 0x7f5a640c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.972+0000 7f5a6c725640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640c3af0 con 0x7f5a640c40a0 2024-09-13T17:58:14.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.972+0000 7f5a6c725640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a64097ed0 con 0x7f5a500039a0 2024-09-13T17:58:14.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.972+0000 7f5a6c725640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640b06e0 con 0x7f5a640afd40 2024-09-13T17:58:14.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.972+0000 7f5a6acc2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a640c40a0 0x7f5a640c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.972+0000 7f5a6acc2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a640c40a0 0x7f5a640c4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43572/0 (socket says 172.21.15.111:43572) 2024-09-13T17:58:14.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.972+0000 7f5a6acc2640 1 -- 172.21.15.111:0/2787720330 learned_addr learned my addr 172.21.15.111:0/2787720330 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:14.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.972+0000 7f5a6a4c1640 1 --2- 172.21.15.111:0/2787720330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a500039a0 0x7f5a64097990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.972+0000 7f5a69cc0640 1 --2- 172.21.15.111:0/2787720330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a640afd40 0x7f5a640b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.973+0000 7f5a6acc2640 1 -- 172.21.15.111:0/2787720330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a640afd40 msgr2=0x7f5a640b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.973+0000 7f5a6acc2640 1 --2- 172.21.15.111:0/2787720330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a640afd40 0x7f5a640b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.973+0000 7f5a6acc2640 1 -- 172.21.15.111:0/2787720330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a500039a0 msgr2=0x7f5a64097990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.973+0000 7f5a6acc2640 1 --2- 172.21.15.111:0/2787720330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a500039a0 0x7f5a64097990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.973+0000 7f5a6acc2640 1 -- 172.21.15.111:0/2787720330 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a640c4a60 con 0x7f5a640c40a0 2024-09-13T17:58:14.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.973+0000 7f5a6acc2640 1 --2- 172.21.15.111:0/2787720330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a640c40a0 0x7f5a640c4520 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5a640cbd40 tx=0x7f5a5c00e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:14.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.973+0000 7f5a5affd640 1 -- 172.21.15.111:0/2787720330 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0052e0 con 0x7f5a640c40a0 2024-09-13T17:58:14.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.973+0000 7f5a5affd640 1 -- 172.21.15.111:0/2787720330 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c005460 con 0x7f5a640c40a0 2024-09-13T17:58:14.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.973+0000 7f5a6c725640 1 -- 172.21.15.111:0/2787720330 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a640c2e70 con 0x7f5a640c40a0 2024-09-13T17:58:14.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.973+0000 7f5a5affd640 1 -- 172.21.15.111:0/2787720330 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c003830 con 0x7f5a640c40a0 2024-09-13T17:58:14.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.973+0000 7f5a6c725640 1 -- 172.21.15.111:0/2787720330 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a640c3430 con 0x7f5a640c40a0 2024-09-13T17:58:14.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.974+0000 7f5a6c725640 1 -- 172.21.15.111:0/2787720330 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a640c3c30 con 0x7f5a640c40a0 2024-09-13T17:58:14.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.974+0000 7f5a5affd640 1 -- 172.21.15.111:0/2787720330 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5a5c003990 con 0x7f5a640c40a0 2024-09-13T17:58:14.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.975+0000 7f5a5affd640 1 --2- 172.21.15.111:0/2787720330 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5a4806ac00 0x7f5a4806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.975+0000 7f5a5affd640 1 -- 172.21.15.111:0/2787720330 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5a5c088cb0 con 0x7f5a640c40a0 2024-09-13T17:58:14.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.975+0000 7f5a6a4c1640 1 --2- 172.21.15.111:0/2787720330 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5a4806ac00 0x7f5a4806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.975+0000 7f5a6a4c1640 1 --2- 172.21.15.111:0/2787720330 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5a4806ac00 0x7f5a4806d0c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5a60033000 tx=0x7f5a6002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:14.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:14.979+0000 7f5a5affd640 1 -- 172.21.15.111:0/2787720330 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f5a640c3c30 con 0x7f5a640c40a0 2024-09-13T17:58:15.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.138+0000 7f5a6c725640 1 -- 172.21.15.111:0/2787720330 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a640c3c30 con 0x7f5a640c40a0 2024-09-13T17:58:15.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.138+0000 7f5a5affd640 1 -- 172.21.15.111:0/2787720330 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a640c3c30 con 0x7f5a640c40a0 2024-09-13T17:58:15.141 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:15.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.140+0000 7f5a6c725640 1 -- 172.21.15.111:0/2787720330 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5a4806ac00 msgr2=0x7f5a4806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:15.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.140+0000 7f5a6c725640 1 --2- 172.21.15.111:0/2787720330 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5a4806ac00 0x7f5a4806d0c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5a60033000 tx=0x7f5a6002e000 comp rx=0 tx=0).stop 2024-09-13T17:58:15.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.140+0000 7f5a6c725640 1 -- 172.21.15.111:0/2787720330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a640c40a0 msgr2=0x7f5a640c4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:15.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.140+0000 7f5a6c725640 1 --2- 172.21.15.111:0/2787720330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a640c40a0 0x7f5a640c4520 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5a640cbd40 tx=0x7f5a5c00e820 comp rx=0 tx=0).stop 2024-09-13T17:58:15.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.140+0000 7f5a6c725640 1 -- 172.21.15.111:0/2787720330 shutdown_connections 2024-09-13T17:58:15.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.140+0000 7f5a6c725640 1 --2- 172.21.15.111:0/2787720330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a640c40a0 0x7f5a640c4520 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.140+0000 7f5a6c725640 1 --2- 172.21.15.111:0/2787720330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a640afd40 0x7f5a640b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.140+0000 7f5a6c725640 1 --2- 172.21.15.111:0/2787720330 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5a4806ac00 0x7f5a4806d0c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.140+0000 7f5a6c725640 1 --2- 172.21.15.111:0/2787720330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a500039a0 0x7f5a64097990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.140+0000 7f5a6c725640 1 -- 172.21.15.111:0/2787720330 >> 172.21.15.111:0/2787720330 conn(0x7f5a6406a0b0 msgr2=0x7f5a640ab8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:15.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.140+0000 7f5a6c725640 1 -- 172.21.15.111:0/2787720330 shutdown_connections 2024-09-13T17:58:15.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.140+0000 7f5a6c725640 1 -- 172.21.15.111:0/2787720330 wait complete. 2024-09-13T17:58:15.154 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:58:15.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.254+0000 7f37a9b9c640 1 Processor -- start 2024-09-13T17:58:15.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.254+0000 7f37a9b9c640 1 -- start start 2024-09-13T17:58:15.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.255+0000 7f37a9b9c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f37a415a7f0 0x7f37a40ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.255+0000 7f37a9b9c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f37a40989d0 0x7f37a4098e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.255+0000 7f37a9b9c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a40cc300 con 0x7f37a40993a0 2024-09-13T17:58:15.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.255+0000 7f37a9b9c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a40b3d20 con 0x7f37a415a7f0 2024-09-13T17:58:15.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.255+0000 7f37a9b9c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a40997d0 con 0x7f37a40989d0 2024-09-13T17:58:15.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.255+0000 7f37a37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f37a415a7f0 0x7f37a40ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:15.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.255+0000 7f37a37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f37a415a7f0 0x7f37a40ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43586/0 (socket says 172.21.15.111:43586) 2024-09-13T17:58:15.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.255+0000 7f37a37fe640 1 -- 172.21.15.111:0/1350227115 learned_addr learned my addr 172.21.15.111:0/1350227115 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:15.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.256+0000 7f37a37fe640 1 -- 172.21.15.111:0/1350227115 >> v2:172.21.15.188:3300/0 conn(0x7f37a40989d0 msgr2=0x7f37a4098e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:15.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.256+0000 7f37a37fe640 1 --2- 172.21.15.111:0/1350227115 >> v2:172.21.15.188:3300/0 conn(0x7f37a40989d0 0x7f37a4098e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.256+0000 7f37a37fe640 1 -- 172.21.15.111:0/1350227115 >> v1:172.21.15.188:6789/0 conn(0x7f37a40993a0 legacy=0x7f37a40caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:15.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.256+0000 7f37a37fe640 1 -- 172.21.15.111:0/1350227115 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a40b33f0 con 0x7f37a415a7f0 2024-09-13T17:58:15.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.256+0000 7f37a37fe640 1 --2- 172.21.15.111:0/1350227115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37a415a7f0 0x7f37a40ce3e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f37a40ce920 tx=0x7f379402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9a1280dc032458 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:15.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.256+0000 7f37a2ffd640 1 -- 172.21.15.111:0/1350227115 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379402ae20 con 0x7f37a415a7f0 2024-09-13T17:58:15.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.256+0000 7f37a2ffd640 1 -- 172.21.15.111:0/1350227115 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3794032440 con 0x7f37a415a7f0 2024-09-13T17:58:15.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.256+0000 7f37a2ffd640 1 -- 172.21.15.111:0/1350227115 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37940327d0 con 0x7f37a415a7f0 2024-09-13T17:58:15.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.257+0000 7f37a9b9c640 1 -- 172.21.15.111:0/1350227115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37a415a7f0 msgr2=0x7f37a40ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:15.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.257+0000 7f37a9b9c640 1 --2- 172.21.15.111:0/1350227115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37a415a7f0 0x7f37a40ce3e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f37a40ce920 tx=0x7f379402a110 comp rx=0 tx=0).stop 2024-09-13T17:58:15.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.257+0000 7f37a9b9c640 1 -- 172.21.15.111:0/1350227115 shutdown_connections 2024-09-13T17:58:15.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.257+0000 7f37a9b9c640 1 --2- 172.21.15.111:0/1350227115 >> v2:172.21.15.188:3300/0 conn(0x7f37a40989d0 0x7f37a4098e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.257+0000 7f37a9b9c640 1 --2- 172.21.15.111:0/1350227115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37a415a7f0 0x7f37a40ce3e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.257+0000 7f37a9b9c640 1 -- 172.21.15.111:0/1350227115 >> 172.21.15.111:0/1350227115 conn(0x7f37a40b1f60 msgr2=0x7f37a40b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:15.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.257+0000 7f37a9b9c640 1 -- 172.21.15.111:0/1350227115 shutdown_connections 2024-09-13T17:58:15.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.257+0000 7f37a9b9c640 1 -- 172.21.15.111:0/1350227115 wait complete. 2024-09-13T17:58:15.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.258+0000 7f37a9b9c640 1 Processor -- start 2024-09-13T17:58:15.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.258+0000 7f37a9b9c640 1 -- start start 2024-09-13T17:58:15.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.259+0000 7f37a9b9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37a40989d0 0x7f37a40ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.259+0000 7f37a9b9c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37a40993a0 0x7f37a4146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.259+0000 7f37a9b9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37a415a7f0 0x7f37a41485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.259+0000 7f37a9b9c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a40c6580 con 0x7f37a40993a0 2024-09-13T17:58:15.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.259+0000 7f37a9b9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a40c5be0 con 0x7f37a415a7f0 2024-09-13T17:58:15.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.259+0000 7f37a9b9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a40c5ec0 con 0x7f37a40989d0 2024-09-13T17:58:15.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.259+0000 7f379affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37a40993a0 0x7f37a4146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:15.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.259+0000 7f379affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37a40993a0 0x7f37a4146560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43590/0 (socket says 172.21.15.111:43590) 2024-09-13T17:58:15.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.259+0000 7f379affd640 1 -- 172.21.15.111:0/1620994826 learned_addr learned my addr 172.21.15.111:0/1620994826 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:15.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.260+0000 7f37a3fff640 1 --2- 172.21.15.111:0/1620994826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37a415a7f0 0x7f37a41485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:15.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.260+0000 7f37a37fe640 1 --2- 172.21.15.111:0/1620994826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37a40989d0 0x7f37a40ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:15.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.260+0000 7f379affd640 1 -- 172.21.15.111:0/1620994826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37a40989d0 msgr2=0x7f37a40ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:15.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.260+0000 7f379affd640 1 --2- 172.21.15.111:0/1620994826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37a40989d0 0x7f37a40ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.260+0000 7f379affd640 1 -- 172.21.15.111:0/1620994826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37a415a7f0 msgr2=0x7f37a41485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:15.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.260+0000 7f379affd640 1 --2- 172.21.15.111:0/1620994826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37a415a7f0 0x7f37a41485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.260+0000 7f379affd640 1 -- 172.21.15.111:0/1620994826 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37940094d0 con 0x7f37a40993a0 2024-09-13T17:58:15.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.260+0000 7f37a3fff640 1 --2- 172.21.15.111:0/1620994826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37a415a7f0 0x7f37a41485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:15.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.260+0000 7f379affd640 1 --2- 172.21.15.111:0/1620994826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37a40993a0 0x7f37a4146560 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f378c00b410 tx=0x7f378c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:15.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.260+0000 7f37a0ff9640 1 -- 172.21.15.111:0/1620994826 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c011830 con 0x7f37a40993a0 2024-09-13T17:58:15.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.260+0000 7f37a0ff9640 1 -- 172.21.15.111:0/1620994826 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c0119b0 con 0x7f37a40993a0 2024-09-13T17:58:15.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.260+0000 7f37a9b9c640 1 -- 172.21.15.111:0/1620994826 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a40c61a0 con 0x7f37a40993a0 2024-09-13T17:58:15.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.260+0000 7f37a0ff9640 1 -- 172.21.15.111:0/1620994826 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c011d60 con 0x7f37a40993a0 2024-09-13T17:58:15.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.260+0000 7f37a9b9c640 1 -- 172.21.15.111:0/1620994826 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a40c5890 con 0x7f37a40993a0 2024-09-13T17:58:15.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.261+0000 7f37a9b9c640 1 -- 172.21.15.111:0/1620994826 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a40b55f0 con 0x7f37a40993a0 2024-09-13T17:58:15.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.261+0000 7f37a0ff9640 1 -- 172.21.15.111:0/1620994826 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f378c01a400 con 0x7f37a40993a0 2024-09-13T17:58:15.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.262+0000 7f37a0ff9640 1 --2- 172.21.15.111:0/1620994826 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f378406ac00 0x7f378406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.262+0000 7f37a0ff9640 1 -- 172.21.15.111:0/1620994826 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f378c089830 con 0x7f37a40993a0 2024-09-13T17:58:15.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.262+0000 7f37a37fe640 1 --2- 172.21.15.111:0/1620994826 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f378406ac00 0x7f378406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:15.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.262+0000 7f37a37fe640 1 --2- 172.21.15.111:0/1620994826 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f378406ac00 0x7f378406d0c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3794034000 tx=0x7f379402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:15.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.266+0000 7f37a0ff9640 1 -- 172.21.15.111:0/1620994826 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f378c0546e0 con 0x7f37a40993a0 2024-09-13T17:58:15.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.461+0000 7f37a9b9c640 1 -- 172.21.15.111:0/1620994826 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f37a40c6330 con 0x7f37a40993a0 2024-09-13T17:58:15.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.461+0000 7f37a0ff9640 1 -- 172.21.15.111:0/1620994826 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f378c0150a0 con 0x7f37a40993a0 2024-09-13T17:58:15.463 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:58:15.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.463+0000 7f37a9b9c640 1 -- 172.21.15.111:0/1620994826 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f378406ac00 msgr2=0x7f378406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:15.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.463+0000 7f37a9b9c640 1 --2- 172.21.15.111:0/1620994826 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f378406ac00 0x7f378406d0c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3794034000 tx=0x7f379402e000 comp rx=0 tx=0).stop 2024-09-13T17:58:15.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.463+0000 7f37a9b9c640 1 -- 172.21.15.111:0/1620994826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37a40993a0 msgr2=0x7f37a4146560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:15.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.463+0000 7f37a9b9c640 1 --2- 172.21.15.111:0/1620994826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37a40993a0 0x7f37a4146560 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f378c00b410 tx=0x7f378c00b8d0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.463+0000 7f37a9b9c640 1 -- 172.21.15.111:0/1620994826 shutdown_connections 2024-09-13T17:58:15.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.463+0000 7f37a9b9c640 1 --2- 172.21.15.111:0/1620994826 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f378406ac00 0x7f378406d0c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.463+0000 7f37a9b9c640 1 --2- 172.21.15.111:0/1620994826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37a415a7f0 0x7f37a41485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.463+0000 7f37a9b9c640 1 --2- 172.21.15.111:0/1620994826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37a40993a0 0x7f37a4146560 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.463+0000 7f37a9b9c640 1 --2- 172.21.15.111:0/1620994826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37a40989d0 0x7f37a40ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.463+0000 7f37a9b9c640 1 -- 172.21.15.111:0/1620994826 >> 172.21.15.111:0/1620994826 conn(0x7f37a40b1f60 msgr2=0x7f37a40b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:15.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.463+0000 7f37a9b9c640 1 -- 172.21.15.111:0/1620994826 shutdown_connections 2024-09-13T17:58:15.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.463+0000 7f37a9b9c640 1 -- 172.21.15.111:0/1620994826 wait complete. 2024-09-13T17:58:15.477 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:58:15.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.578+0000 7f7cfe45a640 1 Processor -- start 2024-09-13T17:58:15.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.578+0000 7f7cfe45a640 1 -- start start 2024-09-13T17:58:15.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.579+0000 7f7cfe45a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf815c500 0x7f7cf80d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.579+0000 7f7cfe45a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7cf80d2590 0x7f7cf80cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.579+0000 7f7cfe45a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf80cd450 con 0x7f7cf80cde20 2024-09-13T17:58:15.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.579+0000 7f7cfe45a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf80d29f0 con 0x7f7cf80d2590 2024-09-13T17:58:15.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.579+0000 7f7cfe45a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80d0cf0 con 0x7f7cf815c500 2024-09-13T17:58:15.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.579+0000 7f7ceffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7cf80d2590 0x7f7cf80cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:15.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.579+0000 7f7ceffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7cf80d2590 0x7f7cf80cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43596/0 (socket says 172.21.15.111:43596) 2024-09-13T17:58:15.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.579+0000 7f7ceffff640 1 -- 172.21.15.111:0/2291251094 learned_addr learned my addr 172.21.15.111:0/2291251094 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:15.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.579+0000 7f7cf7fff640 1 --2- 172.21.15.111:0/2291251094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf815c500 0x7f7cf80d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:15.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.579+0000 7f7ceffff640 1 -- 172.21.15.111:0/2291251094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf815c500 msgr2=0x7f7cf80d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:15.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.579+0000 7f7ceffff640 1 --2- 172.21.15.111:0/2291251094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf815c500 0x7f7cf80d2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.579+0000 7f7ceffff640 1 -- 172.21.15.111:0/2291251094 >> v1:172.21.15.111:6789/0 conn(0x7f7cf80cde20 legacy=0x7f7cf80d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:15.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.579+0000 7f7ceffff640 1 -- 172.21.15.111:0/2291251094 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf80d0fd0 con 0x7f7cf80d2590 2024-09-13T17:58:15.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.580+0000 7f7ceffff640 1 --2- 172.21.15.111:0/2291251094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cf80d2590 0x7f7cf80cd8e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009870 tx=0x7f7ce002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a697e9f9e54d75 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:15.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.580+0000 7f7cf77fe640 1 -- 172.21.15.111:0/2291251094 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0032440 con 0x7f7cf80d2590 2024-09-13T17:58:15.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.580+0000 7f7cf77fe640 1 -- 172.21.15.111:0/2291251094 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce00325a0 con 0x7f7cf80d2590 2024-09-13T17:58:15.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.580+0000 7f7cf77fe640 1 -- 172.21.15.111:0/2291251094 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0032930 con 0x7f7cf80d2590 2024-09-13T17:58:15.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.581+0000 7f7cfe45a640 1 -- 172.21.15.111:0/2291251094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cf80d2590 msgr2=0x7f7cf80cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:15.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.581+0000 7f7cfe45a640 1 --2- 172.21.15.111:0/2291251094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cf80d2590 0x7f7cf80cd8e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009870 tx=0x7f7ce002a270 comp rx=0 tx=0).stop 2024-09-13T17:58:15.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.581+0000 7f7cfe45a640 1 -- 172.21.15.111:0/2291251094 shutdown_connections 2024-09-13T17:58:15.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.581+0000 7f7cfe45a640 1 --2- 172.21.15.111:0/2291251094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cf80d2590 0x7f7cf80cd8e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.581+0000 7f7cfe45a640 1 --2- 172.21.15.111:0/2291251094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf815c500 0x7f7cf80d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.581+0000 7f7cfe45a640 1 -- 172.21.15.111:0/2291251094 >> 172.21.15.111:0/2291251094 conn(0x7f7cf8089f60 msgr2=0x7f7cf809b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:15.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.581+0000 7f7cfe45a640 1 -- 172.21.15.111:0/2291251094 shutdown_connections 2024-09-13T17:58:15.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.581+0000 7f7cfe45a640 1 -- 172.21.15.111:0/2291251094 wait complete. 2024-09-13T17:58:15.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.582+0000 7f7cfe45a640 1 Processor -- start 2024-09-13T17:58:15.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.582+0000 7f7cfe45a640 1 -- start start 2024-09-13T17:58:15.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.583+0000 7f7cfe45a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cf80cde20 0x7f7cf80ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.583+0000 7f7cfe45a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cf80d2590 0x7f7cf8151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.583+0000 7f7cfe45a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf815c500 0x7f7cf80cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.583+0000 7f7cfe45a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80b65f0 con 0x7f7cf80d2590 2024-09-13T17:58:15.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.583+0000 7f7cfe45a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80b7cf0 con 0x7f7cf80cde20 2024-09-13T17:58:15.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.583+0000 7f7cfe45a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80ca420 con 0x7f7cf815c500 2024-09-13T17:58:15.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.583+0000 7f7ceffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cf80d2590 0x7f7cf8151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:15.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.583+0000 7f7ceffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cf80d2590 0x7f7cf8151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43600/0 (socket says 172.21.15.111:43600) 2024-09-13T17:58:15.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.583+0000 7f7ceffff640 1 -- 172.21.15.111:0/1193011234 learned_addr learned my addr 172.21.15.111:0/1193011234 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:15.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.584+0000 7f7cfc9f7640 1 --2- 172.21.15.111:0/1193011234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf815c500 0x7f7cf80cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:15.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.584+0000 7f7cf7fff640 1 --2- 172.21.15.111:0/1193011234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cf80cde20 0x7f7cf80ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:15.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.584+0000 7f7ceffff640 1 -- 172.21.15.111:0/1193011234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf815c500 msgr2=0x7f7cf80cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:15.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.584+0000 7f7ceffff640 1 --2- 172.21.15.111:0/1193011234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf815c500 0x7f7cf80cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.584+0000 7f7ceffff640 1 -- 172.21.15.111:0/1193011234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cf80cde20 msgr2=0x7f7cf80ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:15.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.584+0000 7f7ceffff640 1 --2- 172.21.15.111:0/1193011234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cf80cde20 0x7f7cf80ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.584+0000 7f7ceffff640 1 -- 172.21.15.111:0/1193011234 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce00094d0 con 0x7f7cf80d2590 2024-09-13T17:58:15.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.584+0000 7f7ceffff640 1 --2- 172.21.15.111:0/1193011234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cf80d2590 0x7f7cf8151c60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0000c00 tx=0x7f7ce00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:15.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.584+0000 7f7cf57fa640 1 -- 172.21.15.111:0/1193011234 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0032440 con 0x7f7cf80d2590 2024-09-13T17:58:15.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.584+0000 7f7cf57fa640 1 -- 172.21.15.111:0/1193011234 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce0030b80 con 0x7f7cf80d2590 2024-09-13T17:58:15.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.584+0000 7f7cfe45a640 1 -- 172.21.15.111:0/1193011234 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf80b6030 con 0x7f7cf80d2590 2024-09-13T17:58:15.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.584+0000 7f7cf57fa640 1 -- 172.21.15.111:0/1193011234 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce003d440 con 0x7f7cf80d2590 2024-09-13T17:58:15.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.584+0000 7f7cfe45a640 1 -- 172.21.15.111:0/1193011234 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf80ca0b0 con 0x7f7cf80d2590 2024-09-13T17:58:15.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.585+0000 7f7cfe45a640 1 -- 172.21.15.111:0/1193011234 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf80c9f10 con 0x7f7cf80d2590 2024-09-13T17:58:15.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.586+0000 7f7cf57fa640 1 -- 172.21.15.111:0/1193011234 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7ce0039040 con 0x7f7cf80d2590 2024-09-13T17:58:15.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.586+0000 7f7cf57fa640 1 --2- 172.21.15.111:0/1193011234 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cd806ac00 0x7f7cd806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.586+0000 7f7cf57fa640 1 -- 172.21.15.111:0/1193011234 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7ce00aa2d0 con 0x7f7cf80d2590 2024-09-13T17:58:15.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.586+0000 7f7cf7fff640 1 --2- 172.21.15.111:0/1193011234 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cd806ac00 0x7f7cd806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:15.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.587+0000 7f7cf7fff640 1 --2- 172.21.15.111:0/1193011234 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cd806ac00 0x7f7cd806d0c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8006fd0 tx=0x7f7ce8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:15.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.590+0000 7f7cf57fa640 1 -- 172.21.15.111:0/1193011234 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f7cf80c9f10 con 0x7f7cf80d2590 2024-09-13T17:58:15.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.760+0000 7f7cfe45a640 1 -- 172.21.15.111:0/1193011234 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7cf80c9f10 con 0x7f7cf80d2590 2024-09-13T17:58:15.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.761+0000 7f7cf57fa640 1 -- 172.21.15.111:0/1193011234 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7cf80c9f10 con 0x7f7cf80d2590 2024-09-13T17:58:15.763 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:15.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.763+0000 7f7cfe45a640 1 -- 172.21.15.111:0/1193011234 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cd806ac00 msgr2=0x7f7cd806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:15.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.763+0000 7f7cfe45a640 1 --2- 172.21.15.111:0/1193011234 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cd806ac00 0x7f7cd806d0c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8006fd0 tx=0x7f7ce8008040 comp rx=0 tx=0).stop 2024-09-13T17:58:15.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.763+0000 7f7cfe45a640 1 -- 172.21.15.111:0/1193011234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cf80d2590 msgr2=0x7f7cf8151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:15.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.763+0000 7f7cfe45a640 1 --2- 172.21.15.111:0/1193011234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cf80d2590 0x7f7cf8151c60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0000c00 tx=0x7f7ce00046c0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.763+0000 7f7cfe45a640 1 -- 172.21.15.111:0/1193011234 shutdown_connections 2024-09-13T17:58:15.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.763+0000 7f7cfe45a640 1 --2- 172.21.15.111:0/1193011234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf815c500 0x7f7cf80cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.763+0000 7f7cfe45a640 1 --2- 172.21.15.111:0/1193011234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cf80d2590 0x7f7cf8151c60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.763+0000 7f7cfe45a640 1 --2- 172.21.15.111:0/1193011234 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cd806ac00 0x7f7cd806d0c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.763+0000 7f7cfe45a640 1 --2- 172.21.15.111:0/1193011234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cf80cde20 0x7f7cf80ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.763+0000 7f7cfe45a640 1 -- 172.21.15.111:0/1193011234 >> 172.21.15.111:0/1193011234 conn(0x7f7cf8089f60 msgr2=0x7f7cf80b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:15.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.763+0000 7f7cfe45a640 1 -- 172.21.15.111:0/1193011234 shutdown_connections 2024-09-13T17:58:15.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.763+0000 7f7cfe45a640 1 -- 172.21.15.111:0/1193011234 wait complete. 2024-09-13T17:58:15.777 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:58:15.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.876+0000 7f4d73f7d640 1 Processor -- start 2024-09-13T17:58:15.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.876+0000 7f4d73f7d640 1 -- start start 2024-09-13T17:58:15.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.877+0000 7f4d73f7d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4d6c15c6e0 0x7f4d6c0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.877+0000 7f4d73f7d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d6c0d07b0 0x7f4d6c0b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.877+0000 7f4d73f7d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6c0d1840 con 0x7f4d6c0ccb20 2024-09-13T17:58:15.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.877+0000 7f4d73f7d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d6c06bd20 con 0x7f4d6c15c6e0 2024-09-13T17:58:15.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.877+0000 7f4d73f7d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0d0c30 con 0x7f4d6c0d07b0 2024-09-13T17:58:15.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.877+0000 7f4d71518640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d6c0d07b0 0x7f4d6c0b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:15.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.877+0000 7f4d71518640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d6c0d07b0 0x7f4d6c0b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:33054/0 (socket says 172.21.15.111:33054) 2024-09-13T17:58:15.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.877+0000 7f4d71d19640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4d6c15c6e0 0x7f4d6c0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:15.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.877+0000 7f4d71518640 1 -- 172.21.15.111:0/2604073443 learned_addr learned my addr 172.21.15.111:0/2604073443 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:15.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.877+0000 7f4d71d19640 1 -- 172.21.15.111:0/2604073443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d6c0d07b0 msgr2=0x7f4d6c0b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:15.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.877+0000 7f4d71d19640 1 --2- 172.21.15.111:0/2604073443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d6c0d07b0 0x7f4d6c0b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.877+0000 7f4d71d19640 1 -- 172.21.15.111:0/2604073443 >> v1:172.21.15.188:6789/0 conn(0x7f4d6c0ccb20 legacy=0x7f4d6c160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:15.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.877+0000 7f4d71d19640 1 -- 172.21.15.111:0/2604073443 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c0d0f10 con 0x7f4d6c15c6e0 2024-09-13T17:58:15.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.877+0000 7f4d71518640 1 --2- 172.21.15.111:0/2604073443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d6c0d07b0 0x7f4d6c0b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:15.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.878+0000 7f4d71d19640 1 --2- 172.21.15.111:0/2604073443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d6c15c6e0 0x7f4d6c0b3f50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4d680097a0 tx=0x7f4d6802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4af6ba35ec9cc91a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:15.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.878+0000 7f4d70d17640 1 -- 172.21.15.111:0/2604073443 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6802ae50 con 0x7f4d6c15c6e0 2024-09-13T17:58:15.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.878+0000 7f4d70d17640 1 -- 172.21.15.111:0/2604073443 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d68032440 con 0x7f4d6c15c6e0 2024-09-13T17:58:15.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.878+0000 7f4d70d17640 1 -- 172.21.15.111:0/2604073443 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d680327d0 con 0x7f4d6c15c6e0 2024-09-13T17:58:15.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.879+0000 7f4d73f7d640 1 -- 172.21.15.111:0/2604073443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d6c15c6e0 msgr2=0x7f4d6c0b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:15.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.879+0000 7f4d73f7d640 1 --2- 172.21.15.111:0/2604073443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d6c15c6e0 0x7f4d6c0b3f50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4d680097a0 tx=0x7f4d6802a100 comp rx=0 tx=0).stop 2024-09-13T17:58:15.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.879+0000 7f4d73f7d640 1 -- 172.21.15.111:0/2604073443 shutdown_connections 2024-09-13T17:58:15.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.879+0000 7f4d73f7d640 1 --2- 172.21.15.111:0/2604073443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d6c0d07b0 0x7f4d6c0b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.879+0000 7f4d73f7d640 1 --2- 172.21.15.111:0/2604073443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d6c15c6e0 0x7f4d6c0b3f50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.879+0000 7f4d73f7d640 1 -- 172.21.15.111:0/2604073443 >> 172.21.15.111:0/2604073443 conn(0x7f4d6c089f60 msgr2=0x7f4d6c0d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:15.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.879+0000 7f4d73f7d640 1 -- 172.21.15.111:0/2604073443 shutdown_connections 2024-09-13T17:58:15.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.879+0000 7f4d73f7d640 1 -- 172.21.15.111:0/2604073443 wait complete. 2024-09-13T17:58:15.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.881+0000 7f4d73f7d640 1 Processor -- start 2024-09-13T17:58:15.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.881+0000 7f4d73f7d640 1 -- start start 2024-09-13T17:58:15.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.881+0000 7f4d73f7d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6c0ccb20 0x7f4d6c14b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.881+0000 7f4d73f7d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d6c0d07b0 0x7f4d6c0b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.881+0000 7f4d73f7d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d6c15c6e0 0x7f4d6c15eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.881+0000 7f4d73f7d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0b6990 con 0x7f4d6c0ccb20 2024-09-13T17:58:15.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.881+0000 7f4d73f7d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0b5ff0 con 0x7f4d6c0d07b0 2024-09-13T17:58:15.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.881+0000 7f4d73f7d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0b62d0 con 0x7f4d6c15c6e0 2024-09-13T17:58:15.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.881+0000 7f4d71d19640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6c0ccb20 0x7f4d6c14b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:15.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.881+0000 7f4d71d19640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6c0ccb20 0x7f4d6c14b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43602/0 (socket says 172.21.15.111:43602) 2024-09-13T17:58:15.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.881+0000 7f4d71d19640 1 -- 172.21.15.111:0/3448613821 learned_addr learned my addr 172.21.15.111:0/3448613821 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:15.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.882+0000 7f4d7251a640 1 --2- 172.21.15.111:0/3448613821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d6c15c6e0 0x7f4d6c15eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:15.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.882+0000 7f4d71518640 1 --2- 172.21.15.111:0/3448613821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d6c0d07b0 0x7f4d6c0b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:15.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.882+0000 7f4d71d19640 1 -- 172.21.15.111:0/3448613821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d6c15c6e0 msgr2=0x7f4d6c15eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:15.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.882+0000 7f4d71d19640 1 --2- 172.21.15.111:0/3448613821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d6c15c6e0 0x7f4d6c15eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.882+0000 7f4d71d19640 1 -- 172.21.15.111:0/3448613821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d6c0d07b0 msgr2=0x7f4d6c0b7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:15.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.882+0000 7f4d71d19640 1 --2- 172.21.15.111:0/3448613821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d6c0d07b0 0x7f4d6c0b7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:15.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.882+0000 7f4d71d19640 1 -- 172.21.15.111:0/3448613821 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c0094d0 con 0x7f4d6c0ccb20 2024-09-13T17:58:15.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.882+0000 7f4d7251a640 1 --2- 172.21.15.111:0/3448613821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d6c15c6e0 0x7f4d6c15eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:15.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.882+0000 7f4d71518640 1 --2- 172.21.15.111:0/3448613821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d6c0d07b0 0x7f4d6c0b7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:15.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.882+0000 7f4d71d19640 1 --2- 172.21.15.111:0/3448613821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6c0ccb20 0x7f4d6c14b890 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4d68000c00 tx=0x7f4d680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:15.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.882+0000 7f4d627fc640 1 -- 172.21.15.111:0/3448613821 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6803c610 con 0x7f4d6c0ccb20 2024-09-13T17:58:15.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.882+0000 7f4d627fc640 1 -- 172.21.15.111:0/3448613821 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6803c790 con 0x7f4d6c0ccb20 2024-09-13T17:58:15.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.882+0000 7f4d627fc640 1 -- 172.21.15.111:0/3448613821 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6803cb40 con 0x7f4d6c0ccb20 2024-09-13T17:58:15.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.882+0000 7f4d73f7d640 1 -- 172.21.15.111:0/3448613821 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d68009400 con 0x7f4d6c0ccb20 2024-09-13T17:58:15.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.882+0000 7f4d73f7d640 1 -- 172.21.15.111:0/3448613821 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6c14bdd0 con 0x7f4d6c0ccb20 2024-09-13T17:58:15.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.883+0000 7f4d73f7d640 1 -- 172.21.15.111:0/3448613821 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6c0d2020 con 0x7f4d6c0ccb20 2024-09-13T17:58:15.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.884+0000 7f4d627fc640 1 -- 172.21.15.111:0/3448613821 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4d68039050 con 0x7f4d6c0ccb20 2024-09-13T17:58:15.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.884+0000 7f4d627fc640 1 --2- 172.21.15.111:0/3448613821 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d5006ac00 0x7f4d5006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:15.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.884+0000 7f4d71518640 1 --2- 172.21.15.111:0/3448613821 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d5006ac00 0x7f4d5006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:15.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.884+0000 7f4d627fc640 1 -- 172.21.15.111:0/3448613821 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4d680a9e70 con 0x7f4d6c0ccb20 2024-09-13T17:58:15.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.885+0000 7f4d71518640 1 --2- 172.21.15.111:0/3448613821 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d5006ac00 0x7f4d5006d0c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0ce080 tx=0x7f4d5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:15.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:15.888+0000 7f4d627fc640 1 -- 172.21.15.111:0/3448613821 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4d68074e20 con 0x7f4d6c0ccb20 2024-09-13T17:58:16.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.066+0000 7f4d73f7d640 1 -- 172.21.15.111:0/3448613821 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4d6c0b4c80 con 0x7f4d6c0ccb20 2024-09-13T17:58:16.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.067+0000 7f4d627fc640 1 -- 172.21.15.111:0/3448613821 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4d68037020 con 0x7f4d6c0ccb20 2024-09-13T17:58:16.069 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2024-09-13T17:58:16.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.069+0000 7f4d73f7d640 1 -- 172.21.15.111:0/3448613821 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d5006ac00 msgr2=0x7f4d5006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.069+0000 7f4d73f7d640 1 --2- 172.21.15.111:0/3448613821 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d5006ac00 0x7f4d5006d0c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0ce080 tx=0x7f4d5c008040 comp rx=0 tx=0).stop 2024-09-13T17:58:16.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.069+0000 7f4d73f7d640 1 -- 172.21.15.111:0/3448613821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6c0ccb20 msgr2=0x7f4d6c14b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.069+0000 7f4d73f7d640 1 --2- 172.21.15.111:0/3448613821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6c0ccb20 0x7f4d6c14b890 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4d68000c00 tx=0x7f4d680046c0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.069+0000 7f4d73f7d640 1 -- 172.21.15.111:0/3448613821 shutdown_connections 2024-09-13T17:58:16.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.069+0000 7f4d73f7d640 1 --2- 172.21.15.111:0/3448613821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d6c15c6e0 0x7f4d6c15eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.069+0000 7f4d73f7d640 1 --2- 172.21.15.111:0/3448613821 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d5006ac00 0x7f4d5006d0c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.069+0000 7f4d73f7d640 1 --2- 172.21.15.111:0/3448613821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d6c0d07b0 0x7f4d6c0b7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.069+0000 7f4d73f7d640 1 --2- 172.21.15.111:0/3448613821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6c0ccb20 0x7f4d6c14b890 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.069+0000 7f4d73f7d640 1 -- 172.21.15.111:0/3448613821 >> 172.21.15.111:0/3448613821 conn(0x7f4d6c089f60 msgr2=0x7f4d6c06be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:16.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.069+0000 7f4d73f7d640 1 -- 172.21.15.111:0/3448613821 shutdown_connections 2024-09-13T17:58:16.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.069+0000 7f4d73f7d640 1 -- 172.21.15.111:0/3448613821 wait complete. 2024-09-13T17:58:16.081 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:58:16.081 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:58:16.099 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:16.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.237+0000 7f7ce1cb8640 1 Processor -- start 2024-09-13T17:58:16.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.237+0000 7f7ce1cb8640 1 -- start start 2024-09-13T17:58:16.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.238+0000 7f7ce1cb8640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7cdc15c6e0 0x7f7cdc0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.238+0000 7f7ce1cb8640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cdc0d1840 con 0x7f7cdc0d07b0 2024-09-13T17:58:16.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.238+0000 7f7ce1cb8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cdc06bd20 con 0x7f7cdc0ccb20 2024-09-13T17:58:16.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.238+0000 7f7ce1cb8640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cdc160e50 con 0x7f7cdc15c6e0 2024-09-13T17:58:16.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.238+0000 7f7cdb7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7cdc15c6e0 0x7f7cdc0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.238+0000 7f7cdb7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7cdc15c6e0 0x7f7cdc0b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43610/0 (socket says 172.21.15.111:43610) 2024-09-13T17:58:16.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.238+0000 7f7cdb7fe640 1 -- 172.21.15.111:0/2981739867 learned_addr learned my addr 172.21.15.111:0/2981739867 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:16.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.239+0000 7f7cdb7fe640 1 -- 172.21.15.111:0/2981739867 >> v1:172.21.15.188:6789/0 conn(0x7f7cdc0ccb20 legacy=0x7f7cdc160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:16.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.239+0000 7f7cdb7fe640 1 -- 172.21.15.111:0/2981739867 >> v1:172.21.15.111:6789/0 conn(0x7f7cdc0d07b0 legacy=0x7f7cdc160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.239+0000 7f7cdb7fe640 1 -- 172.21.15.111:0/2981739867 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cdc0d0c30 con 0x7f7cdc15c6e0 2024-09-13T17:58:16.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.239+0000 7f7cdb7fe640 1 --2- 172.21.15.111:0/2981739867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cdc15c6e0 0x7f7cdc0b3f50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009870 tx=0x7f7ccc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e99ac2cdb98ce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:16.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.239+0000 7f7cdaffd640 1 -- 172.21.15.111:0/2981739867 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc032440 con 0x7f7cdc15c6e0 2024-09-13T17:58:16.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.239+0000 7f7cdaffd640 1 -- 172.21.15.111:0/2981739867 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ccc0325a0 con 0x7f7cdc15c6e0 2024-09-13T17:58:16.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.239+0000 7f7cdaffd640 1 -- 172.21.15.111:0/2981739867 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc032930 con 0x7f7cdc15c6e0 2024-09-13T17:58:16.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.240+0000 7f7ce1cb8640 1 -- 172.21.15.111:0/2981739867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cdc15c6e0 msgr2=0x7f7cdc0b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.240+0000 7f7ce1cb8640 1 --2- 172.21.15.111:0/2981739867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cdc15c6e0 0x7f7cdc0b3f50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009870 tx=0x7f7ccc02a210 comp rx=0 tx=0).stop 2024-09-13T17:58:16.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.240+0000 7f7ce1cb8640 1 -- 172.21.15.111:0/2981739867 shutdown_connections 2024-09-13T17:58:16.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.240+0000 7f7ce1cb8640 1 --2- 172.21.15.111:0/2981739867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cdc15c6e0 0x7f7cdc0b3f50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.240+0000 7f7ce1cb8640 1 -- 172.21.15.111:0/2981739867 >> 172.21.15.111:0/2981739867 conn(0x7f7cdc089f60 msgr2=0x7f7cdc0d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:16.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.240+0000 7f7ce1cb8640 1 -- 172.21.15.111:0/2981739867 shutdown_connections 2024-09-13T17:58:16.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.240+0000 7f7ce1cb8640 1 -- 172.21.15.111:0/2981739867 wait complete. 2024-09-13T17:58:16.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.241+0000 7f7ce1cb8640 1 Processor -- start 2024-09-13T17:58:16.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.241+0000 7f7ce1cb8640 1 -- start start 2024-09-13T17:58:16.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.242+0000 7f7ce1cb8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cdc0ccb20 0x7f7cdc154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.242+0000 7f7ce1cb8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cdc0d07b0 0x7f7cdc0b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.242+0000 7f7ce1cb8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cdc15c6e0 0x7f7cdc0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.242+0000 7f7ce1cb8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc1551f0 con 0x7f7cdc0ccb20 2024-09-13T17:58:16.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.242+0000 7f7ce1cb8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc0c9b40 con 0x7f7cdc0d07b0 2024-09-13T17:58:16.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.242+0000 7f7ce1cb8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc0b7f30 con 0x7f7cdc15c6e0 2024-09-13T17:58:16.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.242+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cdc0ccb20 0x7f7cdc154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.242+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cdc0ccb20 0x7f7cdc154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43620/0 (socket says 172.21.15.111:43620) 2024-09-13T17:58:16.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.242+0000 7f7cdb7fe640 1 -- 172.21.15.111:0/2995954181 learned_addr learned my addr 172.21.15.111:0/2995954181 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:16.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.242+0000 7f7cd3fff640 1 --2- 172.21.15.111:0/2995954181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cdc0d07b0 0x7f7cdc0b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.242+0000 7f7cdbfff640 1 --2- 172.21.15.111:0/2995954181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cdc15c6e0 0x7f7cdc0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.242+0000 7f7cdb7fe640 1 -- 172.21.15.111:0/2995954181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cdc15c6e0 msgr2=0x7f7cdc0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.242+0000 7f7cdb7fe640 1 --2- 172.21.15.111:0/2995954181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cdc15c6e0 0x7f7cdc0b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.242+0000 7f7cdb7fe640 1 -- 172.21.15.111:0/2995954181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cdc0d07b0 msgr2=0x7f7cdc0b79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.242+0000 7f7cdb7fe640 1 --2- 172.21.15.111:0/2995954181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cdc0d07b0 0x7f7cdc0b79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.242+0000 7f7cdb7fe640 1 -- 172.21.15.111:0/2995954181 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc0094d0 con 0x7f7cdc0ccb20 2024-09-13T17:58:16.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.243+0000 7f7cdb7fe640 1 --2- 172.21.15.111:0/2995954181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cdc0ccb20 0x7f7cdc154cb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc000c00 tx=0x7f7ccc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:16.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.243+0000 7f7cd8ff9640 1 -- 172.21.15.111:0/2995954181 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc030b00 con 0x7f7cdc0ccb20 2024-09-13T17:58:16.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.243+0000 7f7cd8ff9640 1 -- 172.21.15.111:0/2995954181 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ccc030c60 con 0x7f7cdc0ccb20 2024-09-13T17:58:16.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.243+0000 7f7cd8ff9640 1 -- 172.21.15.111:0/2995954181 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc03d440 con 0x7f7cdc0ccb20 2024-09-13T17:58:16.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.243+0000 7f7ce1cb8640 1 -- 172.21.15.111:0/2995954181 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cdc0b76a0 con 0x7f7cdc0ccb20 2024-09-13T17:58:16.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.243+0000 7f7ce1cb8640 1 -- 172.21.15.111:0/2995954181 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cdc0b5a90 con 0x7f7cdc0ccb20 2024-09-13T17:58:16.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.244+0000 7f7ce1cb8640 1 -- 172.21.15.111:0/2995954181 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cdc0b4c80 con 0x7f7cdc0ccb20 2024-09-13T17:58:16.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.244+0000 7f7cd8ff9640 1 -- 172.21.15.111:0/2995954181 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7ccc039040 con 0x7f7cdc0ccb20 2024-09-13T17:58:16.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.245+0000 7f7cd8ff9640 1 --2- 172.21.15.111:0/2995954181 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cbc06ac00 0x7f7cbc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.245+0000 7f7cd8ff9640 1 -- 172.21.15.111:0/2995954181 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7ccc0aa3b0 con 0x7f7cdc0ccb20 2024-09-13T17:58:16.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.245+0000 7f7cd3fff640 1 --2- 172.21.15.111:0/2995954181 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cbc06ac00 0x7f7cbc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.245+0000 7f7cd3fff640 1 --2- 172.21.15.111:0/2995954181 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cbc06ac00 0x7f7cbc06d0c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc15b690 tx=0x7f7cc4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:16.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.249+0000 7f7cd8ff9640 1 -- 172.21.15.111:0/2995954181 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f7ccc0751d0 con 0x7f7cdc0ccb20 2024-09-13T17:58:16.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.419+0000 7f7ce1cb8640 1 -- 172.21.15.111:0/2995954181 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cdc08c610 con 0x7f7cdc0ccb20 2024-09-13T17:58:16.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.421+0000 7f7cd8ff9640 1 -- 172.21.15.111:0/2995954181 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18794 (secure 0 0 0) 0x7f7ccc0787f0 con 0x7f7cdc0ccb20 2024-09-13T17:58:16.423 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:16.424 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":26,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:07.749113+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:16.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.424+0000 7f7ce1cb8640 1 -- 172.21.15.111:0/2995954181 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cbc06ac00 msgr2=0x7f7cbc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.424+0000 7f7ce1cb8640 1 --2- 172.21.15.111:0/2995954181 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cbc06ac00 0x7f7cbc06d0c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc15b690 tx=0x7f7cc4009040 comp rx=0 tx=0).stop 2024-09-13T17:58:16.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.424+0000 7f7ce1cb8640 1 -- 172.21.15.111:0/2995954181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cdc0ccb20 msgr2=0x7f7cdc154cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.424+0000 7f7ce1cb8640 1 --2- 172.21.15.111:0/2995954181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cdc0ccb20 0x7f7cdc154cb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc000c00 tx=0x7f7ccc0046c0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.424+0000 7f7ce1cb8640 1 -- 172.21.15.111:0/2995954181 shutdown_connections 2024-09-13T17:58:16.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.424+0000 7f7ce1cb8640 1 --2- 172.21.15.111:0/2995954181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cdc15c6e0 0x7f7cdc0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.424+0000 7f7ce1cb8640 1 --2- 172.21.15.111:0/2995954181 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cbc06ac00 0x7f7cbc06d0c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.424+0000 7f7ce1cb8640 1 --2- 172.21.15.111:0/2995954181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cdc0d07b0 0x7f7cdc0b79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.424+0000 7f7ce1cb8640 1 --2- 172.21.15.111:0/2995954181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cdc0ccb20 0x7f7cdc154cb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.424+0000 7f7ce1cb8640 1 -- 172.21.15.111:0/2995954181 >> 172.21.15.111:0/2995954181 conn(0x7f7cdc089f60 msgr2=0x7f7cdc0cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:16.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.424+0000 7f7ce1cb8640 1 -- 172.21.15.111:0/2995954181 shutdown_connections 2024-09-13T17:58:16.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.424+0000 7f7ce1cb8640 1 -- 172.21.15.111:0/2995954181 wait complete. 2024-09-13T17:58:16.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:58:16.438 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:58:16.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.536+0000 7f64d1e0d640 1 Processor -- start 2024-09-13T17:58:16.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.537+0000 7f64d1e0d640 1 -- start start 2024-09-13T17:58:16.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.537+0000 7f64d1e0d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64cc0cdff0 0x7f64cc06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.537+0000 7f64d1e0d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64cc0cd450 con 0x7f64cc15c500 2024-09-13T17:58:16.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.537+0000 7f64d1e0d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64cc0d0cf0 con 0x7f64cc0d2760 2024-09-13T17:58:16.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.537+0000 7f64d1e0d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc0d0fd0 con 0x7f64cc0cdff0 2024-09-13T17:58:16.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.537+0000 7f64cb7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f64cc15c500 0x7f64cc0d2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33542/0 (socket says 172.21.15.111:33542) 2024-09-13T17:58:16.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.537+0000 7f64cb7fe640 1 -- 172.21.15.111:0/3738333776 learned_addr learned my addr 172.21.15.111:0/3738333776 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:16.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64caffd640 1 -- 172.21.15.111:0/3738333776 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265510356 0 0) 0x7f64cc0cd450 con 0x7f64cc15c500 2024-09-13T17:58:16.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64caffd640 1 -- 172.21.15.111:0/3738333776 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64b8003520 con 0x7f64cc15c500 2024-09-13T17:58:16.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64cbfff640 1 --2- 172.21.15.111:0/3738333776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64cc0cdff0 0x7f64cc06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64caffd640 1 -- 172.21.15.111:0/3738333776 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 258170057 0 0) 0x7f64b8003520 con 0x7f64cc15c500 2024-09-13T17:58:16.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64caffd640 1 -- 172.21.15.111:0/3738333776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64cc0cdff0 msgr2=0x7f64cc06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:16.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64caffd640 1 --2- 172.21.15.111:0/3738333776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64cc0cdff0 0x7f64cc06c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64caffd640 1 -- 172.21.15.111:0/3738333776 >> v1:172.21.15.188:6789/0 conn(0x7f64cc0d2760 legacy=0x7f64cc0cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64caffd640 1 -- 172.21.15.111:0/3738333776 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc0cd0e0 con 0x7f64cc15c500 2024-09-13T17:58:16.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64caffd640 1 -- 172.21.15.111:0/3738333776 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f64bc001670 con 0x7f64cc15c500 2024-09-13T17:58:16.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64caffd640 1 -- 172.21.15.111:0/3738333776 >> v1:172.21.15.111:6789/0 conn(0x7f64cc15c500 legacy=0x7f64cc0d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64caffd640 1 --2- 172.21.15.111:0/3738333776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64cc0cdff0 0x7f64b80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64caffd640 1 --2- 172.21.15.111:0/3738333776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64b8003f20 0x7f64b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64caffd640 1 --2- 172.21.15.111:0/3738333776 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64b8004d00 0x7f64b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64caffd640 1 -- 172.21.15.111:0/3738333776 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc0d0fd0 con 0x7f64b8004d00 2024-09-13T17:58:16.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64caffd640 1 -- 172.21.15.111:0/3738333776 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b800ba60 con 0x7f64b8003f20 2024-09-13T17:58:16.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64caffd640 1 -- 172.21.15.111:0/3738333776 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b800bc20 con 0x7f64cc0cdff0 2024-09-13T17:58:16.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64cbfff640 1 --2- 172.21.15.111:0/3738333776 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64b8004d00 0x7f64b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64cb7fe640 1 --2- 172.21.15.111:0/3738333776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64cc0cdff0 0x7f64b80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64c2ffd640 1 --2- 172.21.15.111:0/3738333776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64b8003f20 0x7f64b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64cbfff640 1 -- 172.21.15.111:0/3738333776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64cc0cdff0 msgr2=0x7f64b80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:16.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64cbfff640 1 --2- 172.21.15.111:0/3738333776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64cc0cdff0 0x7f64b80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64cbfff640 1 -- 172.21.15.111:0/3738333776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64b8003f20 msgr2=0x7f64b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:16.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64cbfff640 1 --2- 172.21.15.111:0/3738333776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64b8003f20 0x7f64b80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.538+0000 7f64cbfff640 1 -- 172.21.15.111:0/3738333776 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b8003520 con 0x7f64b8004d00 2024-09-13T17:58:16.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.539+0000 7f64cbfff640 1 --2- 172.21.15.111:0/3738333776 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64b8004d00 0x7f64b8005180 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009fc0 tx=0x7f64b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5101355988a7920f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:16.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.539+0000 7f64caffd640 1 -- 172.21.15.111:0/3738333776 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4031640 con 0x7f64b8004d00 2024-09-13T17:58:16.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.539+0000 7f64caffd640 1 -- 172.21.15.111:0/3738333776 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c402d070 con 0x7f64b8004d00 2024-09-13T17:58:16.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.539+0000 7f64caffd640 1 -- 172.21.15.111:0/3738333776 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c40319e0 con 0x7f64b8004d00 2024-09-13T17:58:16.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.540+0000 7f64d1e0d640 1 -- 172.21.15.111:0/3738333776 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64b8004d00 msgr2=0x7f64b8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.540+0000 7f64d1e0d640 1 --2- 172.21.15.111:0/3738333776 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64b8004d00 0x7f64b8005180 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009fc0 tx=0x7f64b80057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.540+0000 7f64cbfff640 1 -- 172.21.15.111:0/3738333776 reap_dead start 2024-09-13T17:58:16.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.540+0000 7f64d1e0d640 1 -- 172.21.15.111:0/3738333776 shutdown_connections 2024-09-13T17:58:16.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.540+0000 7f64d1e0d640 1 -- 172.21.15.111:0/3738333776 >> 172.21.15.111:0/3738333776 conn(0x7f64cc089f60 msgr2=0x7f64cc09b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:16.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.540+0000 7f64d1e0d640 1 -- 172.21.15.111:0/3738333776 shutdown_connections 2024-09-13T17:58:16.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.540+0000 7f64d1e0d640 1 -- 172.21.15.111:0/3738333776 wait complete. 2024-09-13T17:58:16.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.541+0000 7f64d1e0d640 1 Processor -- start 2024-09-13T17:58:16.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.541+0000 7f64d1e0d640 1 -- start start 2024-09-13T17:58:16.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64d1e0d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64b8004d00 0x7f64cc1f2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64d1e0d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64cc1ef7e0 0x7f64cc15dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64d1e0d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64cc0b2a90 0x7f64cc0b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64d1e0d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc0b3450 con 0x7f64b8004d00 2024-09-13T17:58:16.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64d1e0d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc15e510 con 0x7f64cc1ef7e0 2024-09-13T17:58:16.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64d1e0d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc15cc90 con 0x7f64cc0b2a90 2024-09-13T17:58:16.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64cb7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64b8004d00 0x7f64cc1f2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64cb7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64b8004d00 0x7f64cc1f2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43650/0 (socket says 172.21.15.111:43650) 2024-09-13T17:58:16.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64cb7fe640 1 -- 172.21.15.111:0/800183482 learned_addr learned my addr 172.21.15.111:0/800183482 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:16.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64cbfff640 1 --2- 172.21.15.111:0/800183482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64cc0b2a90 0x7f64cc0b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64c2ffd640 1 --2- 172.21.15.111:0/800183482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64cc1ef7e0 0x7f64cc15dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64cb7fe640 1 -- 172.21.15.111:0/800183482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64cc0b2a90 msgr2=0x7f64cc0b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64cb7fe640 1 --2- 172.21.15.111:0/800183482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64cc0b2a90 0x7f64cc0b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64cb7fe640 1 -- 172.21.15.111:0/800183482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64cc1ef7e0 msgr2=0x7f64cc15dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64cb7fe640 1 --2- 172.21.15.111:0/800183482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64cc1ef7e0 0x7f64cc15dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64cb7fe640 1 -- 172.21.15.111:0/800183482 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc15cf70 con 0x7f64b8004d00 2024-09-13T17:58:16.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64cbfff640 1 --2- 172.21.15.111:0/800183482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64cc0b2a90 0x7f64cc0b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:16.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.542+0000 7f64cb7fe640 1 --2- 172.21.15.111:0/800183482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64b8004d00 0x7f64cc1f2d00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f64bc007290 tx=0x7f64bc007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:16.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.543+0000 7f64c8ff9640 1 -- 172.21.15.111:0/800183482 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc001540 con 0x7f64b8004d00 2024-09-13T17:58:16.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.543+0000 7f64c8ff9640 1 -- 172.21.15.111:0/800183482 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64bc00f940 con 0x7f64b8004d00 2024-09-13T17:58:16.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.543+0000 7f64c8ff9640 1 -- 172.21.15.111:0/800183482 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc00fcd0 con 0x7f64b8004d00 2024-09-13T17:58:16.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.543+0000 7f64d1e0d640 1 -- 172.21.15.111:0/800183482 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64cc15d250 con 0x7f64b8004d00 2024-09-13T17:58:16.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.543+0000 7f64d1e0d640 1 -- 172.21.15.111:0/800183482 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc15dc30 con 0x7f64b8004d00 2024-09-13T17:58:16.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.544+0000 7f64d1e0d640 1 -- 172.21.15.111:0/800183482 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64cc06c8e0 con 0x7f64b8004d00 2024-09-13T17:58:16.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.544+0000 7f64c8ff9640 1 -- 172.21.15.111:0/800183482 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f64bc01d460 con 0x7f64b8004d00 2024-09-13T17:58:16.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.544+0000 7f64c8ff9640 1 --2- 172.21.15.111:0/800183482 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f64ac06ac30 0x7f64ac06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.544+0000 7f64c8ff9640 1 -- 172.21.15.111:0/800183482 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f64bc0898d0 con 0x7f64b8004d00 2024-09-13T17:58:16.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.545+0000 7f64c2ffd640 1 --2- 172.21.15.111:0/800183482 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f64ac06ac30 0x7f64ac06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.545+0000 7f64c2ffd640 1 --2- 172.21.15.111:0/800183482 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f64ac06ac30 0x7f64ac06d0f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0cf600 tx=0x7f64b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:16.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.549+0000 7f64c8ff9640 1 -- 172.21.15.111:0/800183482 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f64bc0546f0 con 0x7f64b8004d00 2024-09-13T17:58:16.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.708+0000 7f64d1e0d640 1 -- 172.21.15.111:0/800183482 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f64cc15d8a0 con 0x7f64b8004d00 2024-09-13T17:58:16.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.708+0000 7f64c8ff9640 1 -- 172.21.15.111:0/800183482 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f64bc087e60 con 0x7f64b8004d00 2024-09-13T17:58:16.710 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:58:16.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.710+0000 7f64d1e0d640 1 -- 172.21.15.111:0/800183482 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f64ac06ac30 msgr2=0x7f64ac06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.710+0000 7f64d1e0d640 1 --2- 172.21.15.111:0/800183482 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f64ac06ac30 0x7f64ac06d0f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0cf600 tx=0x7f64b4009040 comp rx=0 tx=0).stop 2024-09-13T17:58:16.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.710+0000 7f64d1e0d640 1 -- 172.21.15.111:0/800183482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64b8004d00 msgr2=0x7f64cc1f2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.710+0000 7f64d1e0d640 1 --2- 172.21.15.111:0/800183482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64b8004d00 0x7f64cc1f2d00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f64bc007290 tx=0x7f64bc007750 comp rx=0 tx=0).stop 2024-09-13T17:58:16.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.710+0000 7f64d1e0d640 1 -- 172.21.15.111:0/800183482 shutdown_connections 2024-09-13T17:58:16.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.710+0000 7f64d1e0d640 1 --2- 172.21.15.111:0/800183482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64cc0b2a90 0x7f64cc0b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.710+0000 7f64d1e0d640 1 --2- 172.21.15.111:0/800183482 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f64ac06ac30 0x7f64ac06d0f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.710+0000 7f64d1e0d640 1 --2- 172.21.15.111:0/800183482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64cc1ef7e0 0x7f64cc15dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.710+0000 7f64d1e0d640 1 --2- 172.21.15.111:0/800183482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64b8004d00 0x7f64cc1f2d00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.710+0000 7f64d1e0d640 1 -- 172.21.15.111:0/800183482 >> 172.21.15.111:0/800183482 conn(0x7f64cc089f60 msgr2=0x7f64cc0b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:16.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.710+0000 7f64d1e0d640 1 -- 172.21.15.111:0/800183482 shutdown_connections 2024-09-13T17:58:16.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.710+0000 7f64d1e0d640 1 -- 172.21.15.111:0/800183482 wait complete. 2024-09-13T17:58:16.723 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:58:16.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.823+0000 7f636e061640 1 Processor -- start 2024-09-13T17:58:16.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.823+0000 7f636e061640 1 -- start start 2024-09-13T17:58:16.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.823+0000 7f636e061640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636815d8b0 0x7f63680cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.823+0000 7f636e061640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f63680af4b0 0x7f63680ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.823+0000 7f636e061640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f63680cedd0 0x7f63680cae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.823+0000 7f636e061640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63680cf200 con 0x7f63680cedd0 2024-09-13T17:58:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.823+0000 7f636e061640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f636815dc90 con 0x7f63680af4b0 2024-09-13T17:58:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.823+0000 7f636e061640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6368099d20 con 0x7f636815d8b0 2024-09-13T17:58:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.824+0000 7f6367fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f63680cedd0 0x7f63680cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.824+0000 7f6367fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f63680cedd0 0x7f63680cae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43652/0 (socket says 172.21.15.111:43652) 2024-09-13T17:58:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.824+0000 7f6367fff640 1 -- 172.21.15.111:0/3099973355 learned_addr learned my addr 172.21.15.111:0/3099973355 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.824+0000 7f6366ffd640 1 --2- 172.21.15.111:0/3099973355 >> v2:172.21.15.188:3300/0 conn(0x7f63680af4b0 0x7f63680ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.824+0000 7f63677fe640 1 --2- 172.21.15.111:0/3099973355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636815d8b0 0x7f63680cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.824+0000 7f6367fff640 1 -- 172.21.15.111:0/3099973355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636815d8b0 msgr2=0x7f63680cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.824+0000 7f6367fff640 1 --2- 172.21.15.111:0/3099973355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636815d8b0 0x7f63680cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.824+0000 7f6367fff640 1 -- 172.21.15.111:0/3099973355 >> v2:172.21.15.188:3300/0 conn(0x7f63680af4b0 msgr2=0x7f63680ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.824+0000 7f6367fff640 1 --2- 172.21.15.111:0/3099973355 >> v2:172.21.15.188:3300/0 conn(0x7f63680af4b0 0x7f63680ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.824+0000 7f6367fff640 1 -- 172.21.15.111:0/3099973355 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6368099480 con 0x7f63680cedd0 2024-09-13T17:58:16.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.824+0000 7f63677fe640 1 --2- 172.21.15.111:0/3099973355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636815d8b0 0x7f63680cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:16.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.825+0000 7f6367fff640 1 --2- 172.21.15.111:0/3099973355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63680cedd0 0x7f63680cae30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6368071ec0 tx=0x7f636002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b68515cade7e6d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:16.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.825+0000 7f63667fc640 1 -- 172.21.15.111:0/3099973355 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360032440 con 0x7f63680cedd0 2024-09-13T17:58:16.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.825+0000 7f63667fc640 1 -- 172.21.15.111:0/3099973355 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63600325a0 con 0x7f63680cedd0 2024-09-13T17:58:16.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.825+0000 7f63667fc640 1 -- 172.21.15.111:0/3099973355 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360032930 con 0x7f63680cedd0 2024-09-13T17:58:16.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.825+0000 7f636e061640 1 -- 172.21.15.111:0/3099973355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63680cedd0 msgr2=0x7f63680cae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.825+0000 7f636e061640 1 --2- 172.21.15.111:0/3099973355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63680cedd0 0x7f63680cae30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6368071ec0 tx=0x7f636002a1e0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.825+0000 7f636e061640 1 -- 172.21.15.111:0/3099973355 shutdown_connections 2024-09-13T17:58:16.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.825+0000 7f636e061640 1 --2- 172.21.15.111:0/3099973355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63680cedd0 0x7f63680cae30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.825+0000 7f636e061640 1 --2- 172.21.15.111:0/3099973355 >> v2:172.21.15.188:3300/0 conn(0x7f63680af4b0 0x7f63680ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.825+0000 7f636e061640 1 --2- 172.21.15.111:0/3099973355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636815d8b0 0x7f63680cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.825+0000 7f636e061640 1 -- 172.21.15.111:0/3099973355 >> 172.21.15.111:0/3099973355 conn(0x7f63680b1da0 msgr2=0x7f63680b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:16.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.825+0000 7f636e061640 1 -- 172.21.15.111:0/3099973355 shutdown_connections 2024-09-13T17:58:16.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.825+0000 7f636e061640 1 -- 172.21.15.111:0/3099973355 wait complete. 2024-09-13T17:58:16.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.826+0000 7f636e061640 1 Processor -- start 2024-09-13T17:58:16.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.827+0000 7f636e061640 1 -- start start 2024-09-13T17:58:16.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.827+0000 7f636e061640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63680af4b0 0x7f63680c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.827+0000 7f636e061640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63680cedd0 0x7f63681454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.827+0000 7f636e061640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636815d8b0 0x7f63680b5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.827+0000 7f636e061640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63680c5440 con 0x7f63680af4b0 2024-09-13T17:58:16.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.827+0000 7f636e061640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63680c5720 con 0x7f63680cedd0 2024-09-13T17:58:16.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.827+0000 7f636e061640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63680c5a00 con 0x7f636815d8b0 2024-09-13T17:58:16.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.827+0000 7f63677fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63680af4b0 0x7f63680c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.827+0000 7f63677fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63680af4b0 0x7f63680c4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43668/0 (socket says 172.21.15.111:43668) 2024-09-13T17:58:16.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.827+0000 7f63677fe640 1 -- 172.21.15.111:0/358975925 learned_addr learned my addr 172.21.15.111:0/358975925 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:16.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.828+0000 7f6366ffd640 1 --2- 172.21.15.111:0/358975925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63680cedd0 0x7f63681454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.828+0000 7f6367fff640 1 --2- 172.21.15.111:0/358975925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636815d8b0 0x7f63680b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.828+0000 7f63677fe640 1 -- 172.21.15.111:0/358975925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636815d8b0 msgr2=0x7f63680b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.828+0000 7f63677fe640 1 --2- 172.21.15.111:0/358975925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636815d8b0 0x7f63680b5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.828+0000 7f63677fe640 1 -- 172.21.15.111:0/358975925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63680cedd0 msgr2=0x7f63681454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.828+0000 7f63677fe640 1 --2- 172.21.15.111:0/358975925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63680cedd0 0x7f63681454e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.828+0000 7f63677fe640 1 -- 172.21.15.111:0/358975925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63600094d0 con 0x7f63680af4b0 2024-09-13T17:58:16.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.828+0000 7f6367fff640 1 --2- 172.21.15.111:0/358975925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636815d8b0 0x7f63680b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:16.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.828+0000 7f63677fe640 1 --2- 172.21.15.111:0/358975925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63680af4b0 0x7f63680c4dd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f635800cf60 tx=0x7f635800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:16.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.828+0000 7f634ffff640 1 -- 172.21.15.111:0/358975925 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635800d840 con 0x7f63680af4b0 2024-09-13T17:58:16.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.828+0000 7f634ffff640 1 -- 172.21.15.111:0/358975925 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635800d9a0 con 0x7f63680af4b0 2024-09-13T17:58:16.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.828+0000 7f636e061640 1 -- 172.21.15.111:0/358975925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63680c5ce0 con 0x7f63680af4b0 2024-09-13T17:58:16.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.828+0000 7f634ffff640 1 -- 172.21.15.111:0/358975925 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635800dd30 con 0x7f63680af4b0 2024-09-13T17:58:16.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.828+0000 7f636e061640 1 -- 172.21.15.111:0/358975925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6368145a20 con 0x7f63680af4b0 2024-09-13T17:58:16.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.829+0000 7f636e061640 1 -- 172.21.15.111:0/358975925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63680d1220 con 0x7f63680af4b0 2024-09-13T17:58:16.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.829+0000 7f634ffff640 1 -- 172.21.15.111:0/358975925 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6358015440 con 0x7f63680af4b0 2024-09-13T17:58:16.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.830+0000 7f634ffff640 1 --2- 172.21.15.111:0/358975925 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f635406ac00 0x7f635406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:16.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.830+0000 7f634ffff640 1 -- 172.21.15.111:0/358975925 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f635808a010 con 0x7f63680af4b0 2024-09-13T17:58:16.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.830+0000 7f6366ffd640 1 --2- 172.21.15.111:0/358975925 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f635406ac00 0x7f635406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:16.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.830+0000 7f6366ffd640 1 --2- 172.21.15.111:0/358975925 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f635406ac00 0x7f635406d0c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f63680739f0 tx=0x7f635c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:16.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.834+0000 7f634ffff640 1 -- 172.21.15.111:0/358975925 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f6358054f70 con 0x7f63680af4b0 2024-09-13T17:58:16.996 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.994+0000 7f636e061640 1 -- 172.21.15.111:0/358975925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f63680c6200 con 0x7f63680af4b0 2024-09-13T17:58:16.996 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.995+0000 7f634ffff640 1 -- 172.21.15.111:0/358975925 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f635801a020 con 0x7f63680af4b0 2024-09-13T17:58:16.997 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:16.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.996+0000 7f636e061640 1 -- 172.21.15.111:0/358975925 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f635406ac00 msgr2=0x7f635406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.996+0000 7f636e061640 1 --2- 172.21.15.111:0/358975925 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f635406ac00 0x7f635406d0c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f63680739f0 tx=0x7f635c008040 comp rx=0 tx=0).stop 2024-09-13T17:58:16.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.996+0000 7f636e061640 1 -- 172.21.15.111:0/358975925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63680af4b0 msgr2=0x7f63680c4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:16.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.996+0000 7f636e061640 1 --2- 172.21.15.111:0/358975925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63680af4b0 0x7f63680c4dd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f635800cf60 tx=0x7f635800cf90 comp rx=0 tx=0).stop 2024-09-13T17:58:16.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.997+0000 7f636e061640 1 -- 172.21.15.111:0/358975925 shutdown_connections 2024-09-13T17:58:16.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.997+0000 7f636e061640 1 --2- 172.21.15.111:0/358975925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636815d8b0 0x7f63680b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.997+0000 7f636e061640 1 --2- 172.21.15.111:0/358975925 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f635406ac00 0x7f635406d0c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.997+0000 7f636e061640 1 --2- 172.21.15.111:0/358975925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63680cedd0 0x7f63681454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.997+0000 7f636e061640 1 --2- 172.21.15.111:0/358975925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63680af4b0 0x7f63680c4dd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:16.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.997+0000 7f636e061640 1 -- 172.21.15.111:0/358975925 >> 172.21.15.111:0/358975925 conn(0x7f63680b1da0 msgr2=0x7f63680ad420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:16.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.997+0000 7f636e061640 1 -- 172.21.15.111:0/358975925 shutdown_connections 2024-09-13T17:58:16.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:16.997+0000 7f636e061640 1 -- 172.21.15.111:0/358975925 wait complete. 2024-09-13T17:58:17.009 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:58:17.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.107+0000 7f3270345640 1 Processor -- start 2024-09-13T17:58:17.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.107+0000 7f3270345640 1 -- start start 2024-09-13T17:58:17.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.108+0000 7f3270345640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f32680cec00 0x7f32680ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.108+0000 7f3270345640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32680ccbe0 con 0x7f32680b2b40 2024-09-13T17:58:17.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.108+0000 7f3270345640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32680b1810 con 0x7f3268159310 2024-09-13T17:58:17.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.108+0000 7f3270345640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32680ad940 con 0x7f32680cec00 2024-09-13T17:58:17.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.108+0000 7f326d8e0640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f32680b2b40 0x7f32680ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33550/0 (socket says 172.21.15.111:33550) 2024-09-13T17:58:17.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.108+0000 7f326d8e0640 1 -- 172.21.15.111:0/4209285934 learned_addr learned my addr 172.21.15.111:0/4209285934 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:17.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.108+0000 7f326e8e2640 1 --2- 172.21.15.111:0/4209285934 >> v2:172.21.15.188:3300/0 conn(0x7f32680cec00 0x7f32680ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.108+0000 7f326d0df640 1 -- 172.21.15.111:0/4209285934 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202244794 0 0) 0x7f32680ccbe0 con 0x7f32680b2b40 2024-09-13T17:58:17.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.108+0000 7f326d0df640 1 -- 172.21.15.111:0/4209285934 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3250003520 con 0x7f32680b2b40 2024-09-13T17:58:17.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.108+0000 7f326e8e2640 1 -- 172.21.15.111:0/4209285934 >> v1:172.21.15.188:6789/0 conn(0x7f3268159310 legacy=0x7f32680ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.108+0000 7f326e8e2640 1 -- 172.21.15.111:0/4209285934 >> v1:172.21.15.111:6789/0 conn(0x7f32680b2b40 legacy=0x7f32680ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.108+0000 7f326e8e2640 1 -- 172.21.15.111:0/4209285934 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326806d9e0 con 0x7f32680cec00 2024-09-13T17:58:17.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.109+0000 7f326e8e2640 1 --2- 172.21.15.111:0/4209285934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32680cec00 0x7f32680ad400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3258009d90 tx=0x7f325802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fffb0bdb75b88d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:17.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.109+0000 7f326d0df640 1 -- 172.21.15.111:0/4209285934 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258007740 con 0x7f32680cec00 2024-09-13T17:58:17.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.109+0000 7f326d0df640 1 -- 172.21.15.111:0/4209285934 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325802d050 con 0x7f32680cec00 2024-09-13T17:58:17.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.109+0000 7f326d0df640 1 -- 172.21.15.111:0/4209285934 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258007ac0 con 0x7f32680cec00 2024-09-13T17:58:17.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.110+0000 7f3270345640 1 -- 172.21.15.111:0/4209285934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32680cec00 msgr2=0x7f32680ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.110+0000 7f3270345640 1 --2- 172.21.15.111:0/4209285934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32680cec00 0x7f32680ad400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3258009d90 tx=0x7f325802a660 comp rx=0 tx=0).stop 2024-09-13T17:58:17.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.110+0000 7f3270345640 1 -- 172.21.15.111:0/4209285934 shutdown_connections 2024-09-13T17:58:17.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.110+0000 7f3270345640 1 --2- 172.21.15.111:0/4209285934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32680cec00 0x7f32680ad400 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.110+0000 7f3270345640 1 -- 172.21.15.111:0/4209285934 >> 172.21.15.111:0/4209285934 conn(0x7f326806a0b0 msgr2=0x7f326806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:17.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.110+0000 7f3270345640 1 -- 172.21.15.111:0/4209285934 shutdown_connections 2024-09-13T17:58:17.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.110+0000 7f3270345640 1 -- 172.21.15.111:0/4209285934 wait complete. 2024-09-13T17:58:17.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.111+0000 7f3270345640 1 Processor -- start 2024-09-13T17:58:17.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.111+0000 7f3270345640 1 -- start start 2024-09-13T17:58:17.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.112+0000 7f3270345640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32680b2b40 0x7f32680c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.112+0000 7f3270345640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32680cec00 0x7f32680addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.112+0000 7f3270345640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3268159310 0x7f32681510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.112+0000 7f3270345640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32680ae620 con 0x7f32680b2b40 2024-09-13T17:58:17.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.112+0000 7f3270345640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32680ae900 con 0x7f32680cec00 2024-09-13T17:58:17.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.112+0000 7f3270345640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32680aebe0 con 0x7f3268159310 2024-09-13T17:58:17.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.112+0000 7f326e0e1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32680b2b40 0x7f32680c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.112+0000 7f326e0e1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32680b2b40 0x7f32680c4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43684/0 (socket says 172.21.15.111:43684) 2024-09-13T17:58:17.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.112+0000 7f326e0e1640 1 -- 172.21.15.111:0/2751798890 learned_addr learned my addr 172.21.15.111:0/2751798890 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:17.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.112+0000 7f326d8e0640 1 --2- 172.21.15.111:0/2751798890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32680cec00 0x7f32680addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.112+0000 7f326e0e1640 1 -- 172.21.15.111:0/2751798890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3268159310 msgr2=0x7f32681510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.112+0000 7f326e8e2640 1 --2- 172.21.15.111:0/2751798890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3268159310 0x7f32681510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.112+0000 7f326e0e1640 1 --2- 172.21.15.111:0/2751798890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3268159310 0x7f32681510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.112+0000 7f326e0e1640 1 -- 172.21.15.111:0/2751798890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32680cec00 msgr2=0x7f32680addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.112+0000 7f326e0e1640 1 --2- 172.21.15.111:0/2751798890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32680cec00 0x7f32680addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.112+0000 7f326e0e1640 1 -- 172.21.15.111:0/2751798890 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3258009a70 con 0x7f32680b2b40 2024-09-13T17:58:17.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.113+0000 7f326e0e1640 1 --2- 172.21.15.111:0/2751798890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32680b2b40 0x7f32680c4520 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f326000c400 tx=0x7f326000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:17.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.113+0000 7f3266ffd640 1 -- 172.21.15.111:0/2751798890 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32600052d0 con 0x7f32680b2b40 2024-09-13T17:58:17.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.113+0000 7f3266ffd640 1 -- 172.21.15.111:0/2751798890 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3260005430 con 0x7f32680b2b40 2024-09-13T17:58:17.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.113+0000 7f3270345640 1 -- 172.21.15.111:0/2751798890 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32680aeec0 con 0x7f32680b2b40 2024-09-13T17:58:17.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.113+0000 7f3266ffd640 1 -- 172.21.15.111:0/2751798890 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260003830 con 0x7f32680b2b40 2024-09-13T17:58:17.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.113+0000 7f3270345640 1 -- 172.21.15.111:0/2751798890 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32680ae310 con 0x7f32680b2b40 2024-09-13T17:58:17.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.114+0000 7f3270345640 1 -- 172.21.15.111:0/2751798890 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32680af3e0 con 0x7f32680b2b40 2024-09-13T17:58:17.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.114+0000 7f3266ffd640 1 -- 172.21.15.111:0/2751798890 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3260003990 con 0x7f32680b2b40 2024-09-13T17:58:17.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.114+0000 7f3266ffd640 1 --2- 172.21.15.111:0/2751798890 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f325406ac00 0x7f325406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.115+0000 7f3266ffd640 1 -- 172.21.15.111:0/2751798890 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3260088c90 con 0x7f32680b2b40 2024-09-13T17:58:17.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.115+0000 7f326d8e0640 1 --2- 172.21.15.111:0/2751798890 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f325406ac00 0x7f325406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.115+0000 7f326d8e0640 1 --2- 172.21.15.111:0/2751798890 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f325406ac00 0x7f325406d0c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f32680c5110 tx=0x7f325c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:17.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.119+0000 7f3266ffd640 1 -- 172.21.15.111:0/2751798890 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f32680af3e0 con 0x7f32680b2b40 2024-09-13T17:58:17.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.278+0000 7f3270345640 1 -- 172.21.15.111:0/2751798890 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f32680af3e0 con 0x7f32680b2b40 2024-09-13T17:58:17.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.279+0000 7f3266ffd640 1 -- 172.21.15.111:0/2751798890 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f32600170a0 con 0x7f32680b2b40 2024-09-13T17:58:17.281 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:58:17.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.281+0000 7f3270345640 1 -- 172.21.15.111:0/2751798890 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f325406ac00 msgr2=0x7f325406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.281+0000 7f3270345640 1 --2- 172.21.15.111:0/2751798890 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f325406ac00 0x7f325406d0c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f32680c5110 tx=0x7f325c009040 comp rx=0 tx=0).stop 2024-09-13T17:58:17.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.281+0000 7f3270345640 1 -- 172.21.15.111:0/2751798890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32680b2b40 msgr2=0x7f32680c4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.281+0000 7f3270345640 1 --2- 172.21.15.111:0/2751798890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32680b2b40 0x7f32680c4520 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f326000c400 tx=0x7f326000c8c0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.281+0000 7f3270345640 1 -- 172.21.15.111:0/2751798890 shutdown_connections 2024-09-13T17:58:17.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.281+0000 7f3270345640 1 --2- 172.21.15.111:0/2751798890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3268159310 0x7f32681510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.281+0000 7f3270345640 1 --2- 172.21.15.111:0/2751798890 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f325406ac00 0x7f325406d0c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.281+0000 7f3270345640 1 --2- 172.21.15.111:0/2751798890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32680cec00 0x7f32680addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.281+0000 7f3270345640 1 --2- 172.21.15.111:0/2751798890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32680b2b40 0x7f32680c4520 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.281+0000 7f3270345640 1 -- 172.21.15.111:0/2751798890 >> 172.21.15.111:0/2751798890 conn(0x7f326806a0b0 msgr2=0x7f32680cad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:17.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.281+0000 7f3270345640 1 -- 172.21.15.111:0/2751798890 shutdown_connections 2024-09-13T17:58:17.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.281+0000 7f3270345640 1 -- 172.21.15.111:0/2751798890 wait complete. 2024-09-13T17:58:17.293 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:58:17.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.392+0000 7f1a71108640 1 Processor -- start 2024-09-13T17:58:17.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.392+0000 7f1a71108640 1 -- start start 2024-09-13T17:58:17.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.393+0000 7f1a71108640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1a6c0b2880 0x7f1a6c163690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.393+0000 7f1a71108640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6c0b2d10 con 0x7f1a6c158b00 2024-09-13T17:58:17.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.393+0000 7f1a71108640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6c0b2ff0 con 0x7f1a6c06c8a0 2024-09-13T17:58:17.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.393+0000 7f1a71108640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6c070310 con 0x7f1a6c0b2880 2024-09-13T17:58:17.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.393+0000 7f1a6b59e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1a6c0b2880 0x7f1a6c163690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.393+0000 7f1a6b59e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1a6c0b2880 0x7f1a6c163690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43694/0 (socket says 172.21.15.111:43694) 2024-09-13T17:58:17.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.393+0000 7f1a6b59e640 1 -- 172.21.15.111:0/903822636 learned_addr learned my addr 172.21.15.111:0/903822636 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:17.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.393+0000 7f1a6b59e640 1 -- 172.21.15.111:0/903822636 >> v1:172.21.15.188:6789/0 conn(0x7f1a6c06c8a0 legacy=0x7f1a6c0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:17.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.393+0000 7f1a6b59e640 1 -- 172.21.15.111:0/903822636 >> v1:172.21.15.111:6789/0 conn(0x7f1a6c158b00 legacy=0x7f1a6c06c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.393+0000 7f1a6b59e640 1 -- 172.21.15.111:0/903822636 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c0705f0 con 0x7f1a6c0b2880 2024-09-13T17:58:17.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.394+0000 7f1a6b59e640 1 --2- 172.21.15.111:0/903822636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a6c0b2880 0x7f1a6c163690 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009e50 tx=0x7f1a6402a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54c6621005b2fb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:17.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.394+0000 7f1a6a59c640 1 -- 172.21.15.111:0/903822636 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64031600 con 0x7f1a6c0b2880 2024-09-13T17:58:17.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.394+0000 7f1a6a59c640 1 -- 172.21.15.111:0/903822636 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6402d050 con 0x7f1a6c0b2880 2024-09-13T17:58:17.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.394+0000 7f1a6a59c640 1 -- 172.21.15.111:0/903822636 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64031980 con 0x7f1a6c0b2880 2024-09-13T17:58:17.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.395+0000 7f1a71108640 1 -- 172.21.15.111:0/903822636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a6c0b2880 msgr2=0x7f1a6c163690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.395+0000 7f1a71108640 1 --2- 172.21.15.111:0/903822636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a6c0b2880 0x7f1a6c163690 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009e50 tx=0x7f1a6402a760 comp rx=0 tx=0).stop 2024-09-13T17:58:17.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.395+0000 7f1a71108640 1 -- 172.21.15.111:0/903822636 shutdown_connections 2024-09-13T17:58:17.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.395+0000 7f1a71108640 1 --2- 172.21.15.111:0/903822636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a6c0b2880 0x7f1a6c163690 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.395+0000 7f1a71108640 1 -- 172.21.15.111:0/903822636 >> 172.21.15.111:0/903822636 conn(0x7f1a6c06a0b0 msgr2=0x7f1a6c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:17.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.395+0000 7f1a71108640 1 -- 172.21.15.111:0/903822636 shutdown_connections 2024-09-13T17:58:17.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.395+0000 7f1a71108640 1 -- 172.21.15.111:0/903822636 wait complete. 2024-09-13T17:58:17.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.396+0000 7f1a71108640 1 Processor -- start 2024-09-13T17:58:17.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.396+0000 7f1a71108640 1 -- start start 2024-09-13T17:58:17.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.397+0000 7f1a71108640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6c06c8a0 0x7f1a6c09b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.397+0000 7f1a71108640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a6c158b00 0x7f1a6c15b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.397+0000 7f1a71108640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a6c1ee4f0 0x7f1a6c0972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.397+0000 7f1a71108640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c1efab0 con 0x7f1a6c1ee4f0 2024-09-13T17:58:17.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.397+0000 7f1a71108640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c1eea10 con 0x7f1a6c06c8a0 2024-09-13T17:58:17.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.397+0000 7f1a71108640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c1eecf0 con 0x7f1a6c158b00 2024-09-13T17:58:17.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.397+0000 7f1a6b59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a6c1ee4f0 0x7f1a6c0972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.397+0000 7f1a6b59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a6c1ee4f0 0x7f1a6c0972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43698/0 (socket says 172.21.15.111:43698) 2024-09-13T17:58:17.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.397+0000 7f1a6b59e640 1 -- 172.21.15.111:0/1654175936 learned_addr learned my addr 172.21.15.111:0/1654175936 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:17.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.397+0000 7f1a6b59e640 1 -- 172.21.15.111:0/1654175936 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a6c158b00 msgr2=0x7f1a6c15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.397+0000 7f1a6259c640 1 --2- 172.21.15.111:0/1654175936 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a6c158b00 0x7f1a6c15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.397+0000 7f1a6ad9d640 1 --2- 172.21.15.111:0/1654175936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6c06c8a0 0x7f1a6c09b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.397+0000 7f1a6b59e640 1 --2- 172.21.15.111:0/1654175936 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a6c158b00 0x7f1a6c15b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.397+0000 7f1a6b59e640 1 -- 172.21.15.111:0/1654175936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6c06c8a0 msgr2=0x7f1a6c09b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.398+0000 7f1a6b59e640 1 --2- 172.21.15.111:0/1654175936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6c06c8a0 0x7f1a6c09b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.398+0000 7f1a6b59e640 1 -- 172.21.15.111:0/1654175936 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a64009b30 con 0x7f1a6c1ee4f0 2024-09-13T17:58:17.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.398+0000 7f1a6b59e640 1 --2- 172.21.15.111:0/1654175936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a6c1ee4f0 0x7f1a6c0972c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1a64034040 tx=0x7f1a64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:17.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.398+0000 7f1a63fff640 1 -- 172.21.15.111:0/1654175936 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6403d860 con 0x7f1a6c1ee4f0 2024-09-13T17:58:17.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.398+0000 7f1a63fff640 1 -- 172.21.15.111:0/1654175936 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6403d9c0 con 0x7f1a6c1ee4f0 2024-09-13T17:58:17.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.398+0000 7f1a71108640 1 -- 172.21.15.111:0/1654175936 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6c1eefd0 con 0x7f1a6c1ee4f0 2024-09-13T17:58:17.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.398+0000 7f1a63fff640 1 -- 172.21.15.111:0/1654175936 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6403dd70 con 0x7f1a6c1ee4f0 2024-09-13T17:58:17.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.398+0000 7f1a71108640 1 -- 172.21.15.111:0/1654175936 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c1ef590 con 0x7f1a6c1ee4f0 2024-09-13T17:58:17.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.399+0000 7f1a71108640 1 -- 172.21.15.111:0/1654175936 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6c099a60 con 0x7f1a6c1ee4f0 2024-09-13T17:58:17.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.399+0000 7f1a63fff640 1 -- 172.21.15.111:0/1654175936 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1a6403b400 con 0x7f1a6c1ee4f0 2024-09-13T17:58:17.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.400+0000 7f1a63fff640 1 --2- 172.21.15.111:0/1654175936 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1a4c06ac00 0x7f1a4c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.400+0000 7f1a6ad9d640 1 --2- 172.21.15.111:0/1654175936 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1a4c06ac00 0x7f1a4c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.400+0000 7f1a63fff640 1 -- 172.21.15.111:0/1654175936 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1a640aa6a0 con 0x7f1a6c1ee4f0 2024-09-13T17:58:17.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.400+0000 7f1a6ad9d640 1 --2- 172.21.15.111:0/1654175936 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1a4c06ac00 0x7f1a4c06d0c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c000f60 tx=0x7f1a5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:17.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.404+0000 7f1a63fff640 1 -- 172.21.15.111:0/1654175936 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f1a640754c0 con 0x7f1a6c1ee4f0 2024-09-13T17:58:17.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.566+0000 7f1a71108640 1 -- 172.21.15.111:0/1654175936 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1a6c06c190 con 0x7f1a6c1ee4f0 2024-09-13T17:58:17.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.567+0000 7f1a63fff640 1 -- 172.21.15.111:0/1654175936 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1a64009510 con 0x7f1a6c1ee4f0 2024-09-13T17:58:17.569 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:17.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.569+0000 7f1a71108640 1 -- 172.21.15.111:0/1654175936 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1a4c06ac00 msgr2=0x7f1a4c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.569+0000 7f1a71108640 1 --2- 172.21.15.111:0/1654175936 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1a4c06ac00 0x7f1a4c06d0c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c000f60 tx=0x7f1a5c009040 comp rx=0 tx=0).stop 2024-09-13T17:58:17.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.569+0000 7f1a71108640 1 -- 172.21.15.111:0/1654175936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a6c1ee4f0 msgr2=0x7f1a6c0972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.569+0000 7f1a71108640 1 --2- 172.21.15.111:0/1654175936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a6c1ee4f0 0x7f1a6c0972c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1a64034040 tx=0x7f1a64004a40 comp rx=0 tx=0).stop 2024-09-13T17:58:17.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.569+0000 7f1a71108640 1 -- 172.21.15.111:0/1654175936 shutdown_connections 2024-09-13T17:58:17.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.569+0000 7f1a71108640 1 --2- 172.21.15.111:0/1654175936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a6c1ee4f0 0x7f1a6c0972c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.569+0000 7f1a71108640 1 --2- 172.21.15.111:0/1654175936 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a6c158b00 0x7f1a6c15b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.569+0000 7f1a71108640 1 --2- 172.21.15.111:0/1654175936 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1a4c06ac00 0x7f1a4c06d0c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.569+0000 7f1a71108640 1 --2- 172.21.15.111:0/1654175936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6c06c8a0 0x7f1a6c09b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.569+0000 7f1a71108640 1 -- 172.21.15.111:0/1654175936 >> 172.21.15.111:0/1654175936 conn(0x7f1a6c06a0b0 msgr2=0x7f1a6c06a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:17.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.569+0000 7f1a71108640 1 -- 172.21.15.111:0/1654175936 shutdown_connections 2024-09-13T17:58:17.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.569+0000 7f1a71108640 1 -- 172.21.15.111:0/1654175936 wait complete. 2024-09-13T17:58:17.582 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:58:17.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.679+0000 7f6a4a565640 1 Processor -- start 2024-09-13T17:58:17.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.679+0000 7f6a4a565640 1 -- start start 2024-09-13T17:58:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.679+0000 7f6a4a565640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a44071d30 0x7f6a44070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.679+0000 7f6a4a565640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a440abdf0 con 0x7f6a44072110 2024-09-13T17:58:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a4a565640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a440b2280 con 0x7f6a440c8920 2024-09-13T17:58:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a4a565640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a440ab1e0 con 0x7f6a44071d30 2024-09-13T17:58:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a437fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6a44072110 0x7f6a440b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33572/0 (socket says 172.21.15.111:33572) 2024-09-13T17:58:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a437fe640 1 -- 172.21.15.111:0/2561862656 learned_addr learned my addr 172.21.15.111:0/2561862656 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a43fff640 1 --2- 172.21.15.111:0/2561862656 >> v2:172.21.15.188:3300/0 conn(0x7f6a44071d30 0x7f6a44070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a42ffd640 1 -- 172.21.15.111:0/2561862656 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48409441 0 0) 0x7f6a440abdf0 con 0x7f6a44072110 2024-09-13T17:58:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a42ffd640 1 -- 172.21.15.111:0/2561862656 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a2c003520 con 0x7f6a44072110 2024-09-13T17:58:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a42ffd640 1 -- 172.21.15.111:0/2561862656 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3478194473 0 0) 0x7f6a2c003520 con 0x7f6a44072110 2024-09-13T17:58:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a42ffd640 1 -- 172.21.15.111:0/2561862656 >> v2:172.21.15.188:3300/0 conn(0x7f6a44071d30 msgr2=0x7f6a44070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a42ffd640 1 --2- 172.21.15.111:0/2561862656 >> v2:172.21.15.188:3300/0 conn(0x7f6a44071d30 0x7f6a44070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a42ffd640 1 -- 172.21.15.111:0/2561862656 >> v1:172.21.15.188:6789/0 conn(0x7f6a440c8920 legacy=0x7f6a440ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a42ffd640 1 -- 172.21.15.111:0/2561862656 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a440ab4c0 con 0x7f6a44072110 2024-09-13T17:58:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a42ffd640 1 -- 172.21.15.111:0/2561862656 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f6a38001670 con 0x7f6a44072110 2024-09-13T17:58:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a42ffd640 1 -- 172.21.15.111:0/2561862656 >> v1:172.21.15.111:6789/0 conn(0x7f6a44072110 legacy=0x7f6a440b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a42ffd640 1 --2- 172.21.15.111:0/2561862656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a2c003970 0x7f6a2c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a42ffd640 1 --2- 172.21.15.111:0/2561862656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a2c0047c0 0x7f6a2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a42ffd640 1 --2- 172.21.15.111:0/2561862656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a2c005180 0x7f6a2c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a42ffd640 1 -- 172.21.15.111:0/2561862656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a440ab1e0 con 0x7f6a2c0047c0 2024-09-13T17:58:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a42ffd640 1 -- 172.21.15.111:0/2561862656 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c00c020 con 0x7f6a2c005180 2024-09-13T17:58:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a42ffd640 1 -- 172.21.15.111:0/2561862656 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c00e3f0 con 0x7f6a2c003970 2024-09-13T17:58:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a43fff640 1 --2- 172.21.15.111:0/2561862656 >> v2:172.21.15.188:3300/0 conn(0x7f6a44071d30 0x7f6a44070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a437fe640 1 --2- 172.21.15.111:0/2561862656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a2c0047c0 0x7f6a2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.680+0000 7f6a48b02640 1 --2- 172.21.15.111:0/2561862656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a2c005180 0x7f6a2c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.681+0000 7f6a43fff640 1 --2- 172.21.15.111:0/2561862656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a2c003970 0x7f6a2c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.681+0000 7f6a437fe640 1 -- 172.21.15.111:0/2561862656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a2c003970 msgr2=0x7f6a2c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.681+0000 7f6a437fe640 1 --2- 172.21.15.111:0/2561862656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a2c003970 0x7f6a2c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.681+0000 7f6a437fe640 1 -- 172.21.15.111:0/2561862656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a2c005180 msgr2=0x7f6a2c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.681+0000 7f6a437fe640 1 --2- 172.21.15.111:0/2561862656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a2c005180 0x7f6a2c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.681+0000 7f6a437fe640 1 -- 172.21.15.111:0/2561862656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c003520 con 0x7f6a2c0047c0 2024-09-13T17:58:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.681+0000 7f6a48b02640 1 --2- 172.21.15.111:0/2561862656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a2c005180 0x7f6a2c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.681+0000 7f6a48b02640 1 -- 172.21.15.111:0/2561862656 reap_dead start 2024-09-13T17:58:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.681+0000 7f6a437fe640 1 --2- 172.21.15.111:0/2561862656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a2c0047c0 0x7f6a2c004c40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800c240 tx=0x7f6a2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0bcef82a530bbe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.681+0000 7f6a42ffd640 1 -- 172.21.15.111:0/2561862656 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3802e070 con 0x7f6a2c0047c0 2024-09-13T17:58:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.681+0000 7f6a42ffd640 1 -- 172.21.15.111:0/2561862656 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a380058d0 con 0x7f6a2c0047c0 2024-09-13T17:58:17.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.681+0000 7f6a42ffd640 1 -- 172.21.15.111:0/2561862656 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38005d00 con 0x7f6a2c0047c0 2024-09-13T17:58:17.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.682+0000 7f6a4a565640 1 -- 172.21.15.111:0/2561862656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a2c0047c0 msgr2=0x7f6a2c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.682+0000 7f6a4a565640 1 --2- 172.21.15.111:0/2561862656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a2c0047c0 0x7f6a2c004c40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800c240 tx=0x7f6a2c0057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.682+0000 7f6a4a565640 1 -- 172.21.15.111:0/2561862656 shutdown_connections 2024-09-13T17:58:17.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.682+0000 7f6a4a565640 1 --2- 172.21.15.111:0/2561862656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a2c0047c0 0x7f6a2c004c40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.682+0000 7f6a4a565640 1 -- 172.21.15.111:0/2561862656 >> 172.21.15.111:0/2561862656 conn(0x7f6a44069ed0 msgr2=0x7f6a4406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:17.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.682+0000 7f6a4a565640 1 -- 172.21.15.111:0/2561862656 shutdown_connections 2024-09-13T17:58:17.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.682+0000 7f6a4a565640 1 -- 172.21.15.111:0/2561862656 wait complete. 2024-09-13T17:58:17.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.683+0000 7f6a4a565640 1 Processor -- start 2024-09-13T17:58:17.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.683+0000 7f6a4a565640 1 -- start start 2024-09-13T17:58:17.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.684+0000 7f6a4a565640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a2c0047c0 0x7f6a440cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.684+0000 7f6a4a565640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a44157590 0x7f6a44097870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.684+0000 7f6a4a565640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a440c6980 0x7f6a440c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.684+0000 7f6a4a565640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a440cd460 con 0x7f6a44157590 2024-09-13T17:58:17.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.684+0000 7f6a4a565640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a440cc8f0 con 0x7f6a2c0047c0 2024-09-13T17:58:17.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.684+0000 7f6a4a565640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a440ccbc0 con 0x7f6a440c6980 2024-09-13T17:58:17.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.684+0000 7f6a437fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a44157590 0x7f6a44097870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.684+0000 7f6a437fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a44157590 0x7f6a44097870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43712/0 (socket says 172.21.15.111:43712) 2024-09-13T17:58:17.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.684+0000 7f6a437fe640 1 -- 172.21.15.111:0/584596040 learned_addr learned my addr 172.21.15.111:0/584596040 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:17.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.684+0000 7f6a43fff640 1 --2- 172.21.15.111:0/584596040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a2c0047c0 0x7f6a440cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.684+0000 7f6a48b02640 1 --2- 172.21.15.111:0/584596040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a440c6980 0x7f6a440c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.684+0000 7f6a437fe640 1 -- 172.21.15.111:0/584596040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a440c6980 msgr2=0x7f6a440c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.684+0000 7f6a437fe640 1 --2- 172.21.15.111:0/584596040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a440c6980 0x7f6a440c6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.684+0000 7f6a437fe640 1 -- 172.21.15.111:0/584596040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a2c0047c0 msgr2=0x7f6a440cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.684+0000 7f6a437fe640 1 --2- 172.21.15.111:0/584596040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a2c0047c0 0x7f6a440cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.684+0000 7f6a437fe640 1 -- 172.21.15.111:0/584596040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a440ccea0 con 0x7f6a44157590 2024-09-13T17:58:17.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.684+0000 7f6a43fff640 1 --2- 172.21.15.111:0/584596040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a2c0047c0 0x7f6a440cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:17.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.685+0000 7f6a437fe640 1 --2- 172.21.15.111:0/584596040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a44157590 0x7f6a44097870 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6a38001930 tx=0x7f6a38001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:17.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.685+0000 7f6a40ff9640 1 -- 172.21.15.111:0/584596040 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38004e10 con 0x7f6a44157590 2024-09-13T17:58:17.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.685+0000 7f6a40ff9640 1 -- 172.21.15.111:0/584596040 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a38000ed0 con 0x7f6a44157590 2024-09-13T17:58:17.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.685+0000 7f6a4a565640 1 -- 172.21.15.111:0/584596040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a440cd180 con 0x7f6a44157590 2024-09-13T17:58:17.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.685+0000 7f6a40ff9640 1 -- 172.21.15.111:0/584596040 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38005280 con 0x7f6a44157590 2024-09-13T17:58:17.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.685+0000 7f6a4a565640 1 -- 172.21.15.111:0/584596040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a44097db0 con 0x7f6a44157590 2024-09-13T17:58:17.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.686+0000 7f6a4a565640 1 -- 172.21.15.111:0/584596040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4406c090 con 0x7f6a44157590 2024-09-13T17:58:17.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.686+0000 7f6a40ff9640 1 -- 172.21.15.111:0/584596040 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6a380053e0 con 0x7f6a44157590 2024-09-13T17:58:17.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.687+0000 7f6a40ff9640 1 --2- 172.21.15.111:0/584596040 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6a3006ac30 0x7f6a3006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.687+0000 7f6a43fff640 1 --2- 172.21.15.111:0/584596040 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6a3006ac30 0x7f6a3006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.687+0000 7f6a40ff9640 1 -- 172.21.15.111:0/584596040 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6a380a9430 con 0x7f6a44157590 2024-09-13T17:58:17.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.687+0000 7f6a43fff640 1 --2- 172.21.15.111:0/584596040 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6a3006ac30 0x7f6a3006d0f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6a4406a9c0 tx=0x7f6a34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:17.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.691+0000 7f6a40ff9640 1 -- 172.21.15.111:0/584596040 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f6a4406c090 con 0x7f6a44157590 2024-09-13T17:58:17.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.869+0000 7f6a4a565640 1 -- 172.21.15.111:0/584596040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6a4406c090 con 0x7f6a44157590 2024-09-13T17:58:17.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.869+0000 7f6a40ff9640 1 -- 172.21.15.111:0/584596040 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6a4406c090 con 0x7f6a44157590 2024-09-13T17:58:17.871 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2024-09-13T17:58:17.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.871+0000 7f6a4a565640 1 -- 172.21.15.111:0/584596040 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6a3006ac30 msgr2=0x7f6a3006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.871+0000 7f6a4a565640 1 --2- 172.21.15.111:0/584596040 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6a3006ac30 0x7f6a3006d0f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6a4406a9c0 tx=0x7f6a34008040 comp rx=0 tx=0).stop 2024-09-13T17:58:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.871+0000 7f6a4a565640 1 -- 172.21.15.111:0/584596040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a44157590 msgr2=0x7f6a44097870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.871+0000 7f6a4a565640 1 --2- 172.21.15.111:0/584596040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a44157590 0x7f6a44097870 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6a38001930 tx=0x7f6a38001a00 comp rx=0 tx=0).stop 2024-09-13T17:58:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.871+0000 7f6a4a565640 1 -- 172.21.15.111:0/584596040 shutdown_connections 2024-09-13T17:58:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.871+0000 7f6a4a565640 1 --2- 172.21.15.111:0/584596040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a440c6980 0x7f6a440c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.871+0000 7f6a4a565640 1 --2- 172.21.15.111:0/584596040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a44157590 0x7f6a44097870 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.871+0000 7f6a4a565640 1 --2- 172.21.15.111:0/584596040 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6a3006ac30 0x7f6a3006d0f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.871+0000 7f6a4a565640 1 --2- 172.21.15.111:0/584596040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a2c0047c0 0x7f6a440cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.871+0000 7f6a4a565640 1 -- 172.21.15.111:0/584596040 >> 172.21.15.111:0/584596040 conn(0x7f6a44069ed0 msgr2=0x7f6a440ac180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.871+0000 7f6a4a565640 1 -- 172.21.15.111:0/584596040 shutdown_connections 2024-09-13T17:58:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:17.871+0000 7f6a4a565640 1 -- 172.21.15.111:0/584596040 wait complete. 2024-09-13T17:58:17.884 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi111.front.sepia.ceph.com... 2024-09-13T17:58:17.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:17.884 DEBUG:teuthology.orchestra.run.smithi111:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T17:58:17.909 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:58:17.909 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T17:58:17.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:17.910 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T17:58:17.965 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:58:17.965 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:58:17.985 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:58:17.985 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:58:18.045 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:18.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.140+0000 7fc71775e640 1 Processor -- start 2024-09-13T17:58:18.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.141+0000 7fc71775e640 1 -- start start 2024-09-13T17:58:18.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.141+0000 7fc71775e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc7100ca5a0 0x7fc710070930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.141+0000 7fc71775e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc71006dc40 0x7fc71006c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.141+0000 7fc71775e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc710070e70 con 0x7fc71006d610 2024-09-13T17:58:18.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.141+0000 7fc71775e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc71006d180 con 0x7fc71006dc40 2024-09-13T17:58:18.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.141+0000 7fc71775e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc710070250 con 0x7fc7100ca5a0 2024-09-13T17:58:18.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.142+0000 7fc714cf9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc71006dc40 0x7fc71006c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.142+0000 7fc714cf9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc71006dc40 0x7fc71006c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43722/0 (socket says 172.21.15.111:43722) 2024-09-13T17:58:18.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.142+0000 7fc714cf9640 1 -- 172.21.15.111:0/241181186 learned_addr learned my addr 172.21.15.111:0/241181186 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:18.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.142+0000 7fc714cf9640 1 -- 172.21.15.111:0/241181186 >> v2:172.21.15.188:3300/0 conn(0x7fc7100ca5a0 msgr2=0x7fc710070930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:18.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.142+0000 7fc714cf9640 1 --2- 172.21.15.111:0/241181186 >> v2:172.21.15.188:3300/0 conn(0x7fc7100ca5a0 0x7fc710070930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.142+0000 7fc714cf9640 1 -- 172.21.15.111:0/241181186 >> v1:172.21.15.188:6789/0 conn(0x7fc71006d610 legacy=0x7fc71006c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:18.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.142+0000 7fc714cf9640 1 -- 172.21.15.111:0/241181186 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc710070530 con 0x7fc71006dc40 2024-09-13T17:58:18.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.142+0000 7fc714cf9640 1 --2- 172.21.15.111:0/241181186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc71006dc40 0x7fc71006c040 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc7100b0c10 tx=0x7fc70002c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7143232e1b822 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.143+0000 7fc707fff640 1 -- 172.21.15.111:0/241181186 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70002f070 con 0x7fc71006dc40 2024-09-13T17:58:18.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.143+0000 7fc707fff640 1 -- 172.21.15.111:0/241181186 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7000325a0 con 0x7fc71006dc40 2024-09-13T17:58:18.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.143+0000 7fc707fff640 1 -- 172.21.15.111:0/241181186 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700032930 con 0x7fc71006dc40 2024-09-13T17:58:18.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.143+0000 7fc71775e640 1 -- 172.21.15.111:0/241181186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc71006dc40 msgr2=0x7fc71006c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.143+0000 7fc71775e640 1 --2- 172.21.15.111:0/241181186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc71006dc40 0x7fc71006c040 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc7100b0c10 tx=0x7fc70002c8e0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.143+0000 7fc71775e640 1 -- 172.21.15.111:0/241181186 shutdown_connections 2024-09-13T17:58:18.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.143+0000 7fc71775e640 1 --2- 172.21.15.111:0/241181186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc71006dc40 0x7fc71006c040 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.143+0000 7fc71775e640 1 --2- 172.21.15.111:0/241181186 >> v2:172.21.15.188:3300/0 conn(0x7fc7100ca5a0 0x7fc710070930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.143+0000 7fc71775e640 1 -- 172.21.15.111:0/241181186 >> 172.21.15.111:0/241181186 conn(0x7fc7100afbe0 msgr2=0x7fc7100afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:18.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.143+0000 7fc71775e640 1 -- 172.21.15.111:0/241181186 shutdown_connections 2024-09-13T17:58:18.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.143+0000 7fc71775e640 1 -- 172.21.15.111:0/241181186 wait complete. 2024-09-13T17:58:18.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.144+0000 7fc71775e640 1 Processor -- start 2024-09-13T17:58:18.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.144+0000 7fc71775e640 1 -- start start 2024-09-13T17:58:18.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.145+0000 7fc71775e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc71006d610 0x7fc7100aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.145+0000 7fc71775e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc71006dc40 0x7fc7100c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.145+0000 7fc71775e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7100ca5a0 0x7fc7100c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.145+0000 7fc71775e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7101f59e0 con 0x7fc7100ca5a0 2024-09-13T17:58:18.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.145+0000 7fc71775e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7101f5c80 con 0x7fc71006dc40 2024-09-13T17:58:18.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.145+0000 7fc71775e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7101f5f60 con 0x7fc71006d610 2024-09-13T17:58:18.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.145+0000 7fc715cfb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7100ca5a0 0x7fc7100c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.145+0000 7fc715cfb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7100ca5a0 0x7fc7100c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43732/0 (socket says 172.21.15.111:43732) 2024-09-13T17:58:18.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.145+0000 7fc715cfb640 1 -- 172.21.15.111:0/4149026573 learned_addr learned my addr 172.21.15.111:0/4149026573 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:18.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.145+0000 7fc714cf9640 1 --2- 172.21.15.111:0/4149026573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc71006dc40 0x7fc7100c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.146+0000 7fc7154fa640 1 --2- 172.21.15.111:0/4149026573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc71006d610 0x7fc7100aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.146+0000 7fc715cfb640 1 -- 172.21.15.111:0/4149026573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc71006d610 msgr2=0x7fc7100aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.146+0000 7fc715cfb640 1 --2- 172.21.15.111:0/4149026573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc71006d610 0x7fc7100aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.146+0000 7fc715cfb640 1 -- 172.21.15.111:0/4149026573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc71006dc40 msgr2=0x7fc7100c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.146+0000 7fc715cfb640 1 --2- 172.21.15.111:0/4149026573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc71006dc40 0x7fc7100c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.146+0000 7fc715cfb640 1 -- 172.21.15.111:0/4149026573 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70000b770 con 0x7fc7100ca5a0 2024-09-13T17:58:18.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.146+0000 7fc714cf9640 1 --2- 172.21.15.111:0/4149026573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc71006dc40 0x7fc7100c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:18.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.146+0000 7fc715cfb640 1 --2- 172.21.15.111:0/4149026573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7100ca5a0 0x7fc7100c8cb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc70800c780 tx=0x7fc70800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.146+0000 7fc705ffb640 1 -- 172.21.15.111:0/4149026573 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708004b60 con 0x7fc7100ca5a0 2024-09-13T17:58:18.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.146+0000 7fc705ffb640 1 -- 172.21.15.111:0/4149026573 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70800ce90 con 0x7fc7100ca5a0 2024-09-13T17:58:18.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.146+0000 7fc705ffb640 1 -- 172.21.15.111:0/4149026573 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70800a850 con 0x7fc7100ca5a0 2024-09-13T17:58:18.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.146+0000 7fc71775e640 1 -- 172.21.15.111:0/4149026573 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7101f6240 con 0x7fc7100ca5a0 2024-09-13T17:58:18.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.146+0000 7fc71775e640 1 -- 172.21.15.111:0/4149026573 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7100aef40 con 0x7fc7100ca5a0 2024-09-13T17:58:18.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.147+0000 7fc71775e640 1 -- 172.21.15.111:0/4149026573 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7100672e0 con 0x7fc7100ca5a0 2024-09-13T17:58:18.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.147+0000 7fc705ffb640 1 -- 172.21.15.111:0/4149026573 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc70800a9b0 con 0x7fc7100ca5a0 2024-09-13T17:58:18.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.148+0000 7fc705ffb640 1 --2- 172.21.15.111:0/4149026573 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc6f406ac00 0x7fc6f406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.148+0000 7fc705ffb640 1 -- 172.21.15.111:0/4149026573 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc708089880 con 0x7fc7100ca5a0 2024-09-13T17:58:18.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.148+0000 7fc7154fa640 1 --2- 172.21.15.111:0/4149026573 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc6f406ac00 0x7fc6f406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.148+0000 7fc7154fa640 1 --2- 172.21.15.111:0/4149026573 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc6f406ac00 0x7fc6f406d0c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00b810 tx=0x7fc70c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.152+0000 7fc705ffb640 1 -- 172.21.15.111:0/4149026573 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc708054730 con 0x7fc7100ca5a0 2024-09-13T17:58:18.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.314+0000 7fc71775e640 1 -- 172.21.15.111:0/4149026573 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7101f6760 con 0x7fc7100ca5a0 2024-09-13T17:58:18.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.315+0000 7fc705ffb640 1 -- 172.21.15.111:0/4149026573 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18794 (secure 0 0 0) 0x7fc7101f6760 con 0x7fc7100ca5a0 2024-09-13T17:58:18.317 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:18.317 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":26,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:07.749113+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:18.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.318+0000 7fc71775e640 1 -- 172.21.15.111:0/4149026573 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc6f406ac00 msgr2=0x7fc6f406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.318+0000 7fc71775e640 1 --2- 172.21.15.111:0/4149026573 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc6f406ac00 0x7fc6f406d0c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00b810 tx=0x7fc70c008040 comp rx=0 tx=0).stop 2024-09-13T17:58:18.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.318+0000 7fc71775e640 1 -- 172.21.15.111:0/4149026573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7100ca5a0 msgr2=0x7fc7100c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.318+0000 7fc71775e640 1 --2- 172.21.15.111:0/4149026573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7100ca5a0 0x7fc7100c8cb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc70800c780 tx=0x7fc70800cc40 comp rx=0 tx=0).stop 2024-09-13T17:58:18.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.318+0000 7fc71775e640 1 -- 172.21.15.111:0/4149026573 shutdown_connections 2024-09-13T17:58:18.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.318+0000 7fc71775e640 1 --2- 172.21.15.111:0/4149026573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7100ca5a0 0x7fc7100c8cb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.318+0000 7fc71775e640 1 --2- 172.21.15.111:0/4149026573 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc6f406ac00 0x7fc6f406d0c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.318+0000 7fc71775e640 1 --2- 172.21.15.111:0/4149026573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc71006dc40 0x7fc7100c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.318+0000 7fc71775e640 1 --2- 172.21.15.111:0/4149026573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc71006d610 0x7fc7100aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.318+0000 7fc71775e640 1 -- 172.21.15.111:0/4149026573 >> 172.21.15.111:0/4149026573 conn(0x7fc7100afbe0 msgr2=0x7fc7100affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:18.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.318+0000 7fc71775e640 1 -- 172.21.15.111:0/4149026573 shutdown_connections 2024-09-13T17:58:18.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.318+0000 7fc71775e640 1 -- 172.21.15.111:0/4149026573 wait complete. 2024-09-13T17:58:18.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:58:18.330 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:58:18.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.429+0000 7f5f9dcd0640 1 Processor -- start 2024-09-13T17:58:18.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.429+0000 7f5f9dcd0640 1 -- start start 2024-09-13T17:58:18.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.429+0000 7f5f9dcd0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5f9815c500 0x7f5f980d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.429+0000 7f5f9dcd0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5f980cdff0 0x7f5f980d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.429+0000 7f5f9dcd0640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f980cd450 con 0x7f5f980d2590 2024-09-13T17:58:18.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.429+0000 7f5f9dcd0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f980d29f0 con 0x7f5f980cdff0 2024-09-13T17:58:18.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.429+0000 7f5f9dcd0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f980d0cf0 con 0x7f5f9815c500 2024-09-13T17:58:18.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.430+0000 7f5f97fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5f980cdff0 0x7f5f980d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.430+0000 7f5f97fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5f980cdff0 0x7f5f980d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43746/0 (socket says 172.21.15.111:43746) 2024-09-13T17:58:18.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.430+0000 7f5f97fff640 1 -- 172.21.15.111:0/1733244183 learned_addr learned my addr 172.21.15.111:0/1733244183 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:18.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.430+0000 7f5f97fff640 1 -- 172.21.15.111:0/1733244183 >> v2:172.21.15.188:3300/0 conn(0x7f5f9815c500 msgr2=0x7f5f980d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:18.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.430+0000 7f5f97fff640 1 --2- 172.21.15.111:0/1733244183 >> v2:172.21.15.188:3300/0 conn(0x7f5f9815c500 0x7f5f980d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.430+0000 7f5f97fff640 1 -- 172.21.15.111:0/1733244183 >> v1:172.21.15.111:6789/0 conn(0x7f5f980d2590 legacy=0x7f5f980cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.430+0000 7f5f97fff640 1 -- 172.21.15.111:0/1733244183 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f980d0fd0 con 0x7f5f980cdff0 2024-09-13T17:58:18.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.431+0000 7f5f97fff640 1 --2- 172.21.15.111:0/1733244183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f980cdff0 0x7f5f980d0000 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5f980cfec0 tx=0x7f5f9002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a03e0f4a17ae18 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.431+0000 7f5f967fc640 1 -- 172.21.15.111:0/1733244183 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f90031600 con 0x7f5f980cdff0 2024-09-13T17:58:18.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.431+0000 7f5f967fc640 1 -- 172.21.15.111:0/1733244183 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9002d050 con 0x7f5f980cdff0 2024-09-13T17:58:18.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.431+0000 7f5f967fc640 1 -- 172.21.15.111:0/1733244183 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f90031980 con 0x7f5f980cdff0 2024-09-13T17:58:18.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.431+0000 7f5f9dcd0640 1 -- 172.21.15.111:0/1733244183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f980cdff0 msgr2=0x7f5f980d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.431+0000 7f5f9dcd0640 1 --2- 172.21.15.111:0/1733244183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f980cdff0 0x7f5f980d0000 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5f980cfec0 tx=0x7f5f9002a730 comp rx=0 tx=0).stop 2024-09-13T17:58:18.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.431+0000 7f5f9dcd0640 1 -- 172.21.15.111:0/1733244183 shutdown_connections 2024-09-13T17:58:18.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.431+0000 7f5f9dcd0640 1 --2- 172.21.15.111:0/1733244183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f980cdff0 0x7f5f980d0000 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.431+0000 7f5f9dcd0640 1 --2- 172.21.15.111:0/1733244183 >> v2:172.21.15.188:3300/0 conn(0x7f5f9815c500 0x7f5f980d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.431+0000 7f5f9dcd0640 1 -- 172.21.15.111:0/1733244183 >> 172.21.15.111:0/1733244183 conn(0x7f5f98089f60 msgr2=0x7f5f9809b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:18.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.431+0000 7f5f9dcd0640 1 -- 172.21.15.111:0/1733244183 shutdown_connections 2024-09-13T17:58:18.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.431+0000 7f5f9dcd0640 1 -- 172.21.15.111:0/1733244183 wait complete. 2024-09-13T17:58:18.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.432+0000 7f5f9dcd0640 1 Processor -- start 2024-09-13T17:58:18.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.433+0000 7f5f9dcd0640 1 -- start start 2024-09-13T17:58:18.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.433+0000 7f5f9dcd0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f980cdff0 0x7f5f98153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.433+0000 7f5f9dcd0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f980d2590 0x7f5f9815d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.433+0000 7f5f9dcd0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f9815c500 0x7f5f98155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.433+0000 7f5f9dcd0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f980b6310 con 0x7f5f980cdff0 2024-09-13T17:58:18.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.433+0000 7f5f9dcd0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f980b65f0 con 0x7f5f9815c500 2024-09-13T17:58:18.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.433+0000 7f5f9dcd0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f980b68d0 con 0x7f5f980d2590 2024-09-13T17:58:18.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.433+0000 7f5f977fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f980cdff0 0x7f5f98153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.433+0000 7f5f977fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f980cdff0 0x7f5f98153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43748/0 (socket says 172.21.15.111:43748) 2024-09-13T17:58:18.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.433+0000 7f5f977fe640 1 -- 172.21.15.111:0/3827522201 learned_addr learned my addr 172.21.15.111:0/3827522201 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:18.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.434+0000 7f5f97fff640 1 --2- 172.21.15.111:0/3827522201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f9815c500 0x7f5f98155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.434+0000 7f5f96ffd640 1 --2- 172.21.15.111:0/3827522201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f980d2590 0x7f5f9815d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.434+0000 7f5f977fe640 1 -- 172.21.15.111:0/3827522201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f980d2590 msgr2=0x7f5f9815d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.434+0000 7f5f977fe640 1 --2- 172.21.15.111:0/3827522201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f980d2590 0x7f5f9815d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.434+0000 7f5f977fe640 1 -- 172.21.15.111:0/3827522201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f9815c500 msgr2=0x7f5f98155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.434+0000 7f5f977fe640 1 --2- 172.21.15.111:0/3827522201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f9815c500 0x7f5f98155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.434+0000 7f5f977fe640 1 -- 172.21.15.111:0/3827522201 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f90009b30 con 0x7f5f980cdff0 2024-09-13T17:58:18.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.434+0000 7f5f97fff640 1 --2- 172.21.15.111:0/3827522201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f9815c500 0x7f5f98155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:18.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.434+0000 7f5f96ffd640 1 --2- 172.21.15.111:0/3827522201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f980d2590 0x7f5f9815d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:18.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.434+0000 7f5f977fe640 1 --2- 172.21.15.111:0/3827522201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f980cdff0 0x7f5f98153a60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800b410 tx=0x7f5f8800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.434+0000 7f5f7ffff640 1 -- 172.21.15.111:0/3827522201 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f88011830 con 0x7f5f980cdff0 2024-09-13T17:58:18.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.434+0000 7f5f7ffff640 1 -- 172.21.15.111:0/3827522201 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f880119b0 con 0x7f5f980cdff0 2024-09-13T17:58:18.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.434+0000 7f5f9dcd0640 1 -- 172.21.15.111:0/3827522201 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f980b6bb0 con 0x7f5f980cdff0 2024-09-13T17:58:18.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.434+0000 7f5f7ffff640 1 -- 172.21.15.111:0/3827522201 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f88011d60 con 0x7f5f980cdff0 2024-09-13T17:58:18.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.434+0000 7f5f9dcd0640 1 -- 172.21.15.111:0/3827522201 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9815daa0 con 0x7f5f980cdff0 2024-09-13T17:58:18.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.435+0000 7f5f9dcd0640 1 -- 172.21.15.111:0/3827522201 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f980b8440 con 0x7f5f980cdff0 2024-09-13T17:58:18.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.435+0000 7f5f7ffff640 1 -- 172.21.15.111:0/3827522201 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5f8801a400 con 0x7f5f980cdff0 2024-09-13T17:58:18.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.436+0000 7f5f7ffff640 1 --2- 172.21.15.111:0/3827522201 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5f8406ac00 0x7f5f8406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.436+0000 7f5f7ffff640 1 -- 172.21.15.111:0/3827522201 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5f880898c0 con 0x7f5f980cdff0 2024-09-13T17:58:18.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.436+0000 7f5f96ffd640 1 --2- 172.21.15.111:0/3827522201 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5f8406ac00 0x7f5f8406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.436+0000 7f5f96ffd640 1 --2- 172.21.15.111:0/3827522201 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5f8406ac00 0x7f5f8406d0c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5f9806c6a0 tx=0x7f5f8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.440+0000 7f5f7ffff640 1 -- 172.21.15.111:0/3827522201 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f5f880546e0 con 0x7f5f980cdff0 2024-09-13T17:58:18.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.599+0000 7f5f9dcd0640 1 -- 172.21.15.111:0/3827522201 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f980b6450 con 0x7f5f980cdff0 2024-09-13T17:58:18.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.600+0000 7f5f7ffff640 1 -- 172.21.15.111:0/3827522201 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f980b6450 con 0x7f5f980cdff0 2024-09-13T17:58:18.602 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:58:18.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.602+0000 7f5f9dcd0640 1 -- 172.21.15.111:0/3827522201 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5f8406ac00 msgr2=0x7f5f8406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.602+0000 7f5f9dcd0640 1 --2- 172.21.15.111:0/3827522201 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5f8406ac00 0x7f5f8406d0c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5f9806c6a0 tx=0x7f5f8c009040 comp rx=0 tx=0).stop 2024-09-13T17:58:18.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.602+0000 7f5f9dcd0640 1 -- 172.21.15.111:0/3827522201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f980cdff0 msgr2=0x7f5f98153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.602+0000 7f5f9dcd0640 1 --2- 172.21.15.111:0/3827522201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f980cdff0 0x7f5f98153a60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800b410 tx=0x7f5f8800b8d0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.602+0000 7f5f9dcd0640 1 -- 172.21.15.111:0/3827522201 shutdown_connections 2024-09-13T17:58:18.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.602+0000 7f5f9dcd0640 1 --2- 172.21.15.111:0/3827522201 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5f8406ac00 0x7f5f8406d0c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.602+0000 7f5f9dcd0640 1 --2- 172.21.15.111:0/3827522201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f9815c500 0x7f5f98155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.602+0000 7f5f9dcd0640 1 --2- 172.21.15.111:0/3827522201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f980d2590 0x7f5f9815d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.602+0000 7f5f9dcd0640 1 --2- 172.21.15.111:0/3827522201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f980cdff0 0x7f5f98153a60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.602+0000 7f5f9dcd0640 1 -- 172.21.15.111:0/3827522201 >> 172.21.15.111:0/3827522201 conn(0x7f5f98089f60 msgr2=0x7f5f980b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:18.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.602+0000 7f5f9dcd0640 1 -- 172.21.15.111:0/3827522201 shutdown_connections 2024-09-13T17:58:18.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.602+0000 7f5f9dcd0640 1 -- 172.21.15.111:0/3827522201 wait complete. 2024-09-13T17:58:18.615 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:58:18.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.713+0000 7fdf0206b640 1 Processor -- start 2024-09-13T17:58:18.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.713+0000 7fdf0206b640 1 -- start start 2024-09-13T17:58:18.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.713+0000 7fdf0206b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdefc159310 0x7fdefc0ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.714+0000 7fdf0206b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdefc0ce4f0 0x7fdefc0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.714+0000 7fdf0206b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc0ccbe0 con 0x7fdefc0b2b40 2024-09-13T17:58:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.714+0000 7fdf0206b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdefc0ad940 con 0x7fdefc159310 2024-09-13T17:58:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.714+0000 7fdf0206b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdefc0b1810 con 0x7fdefc0ce4f0 2024-09-13T17:58:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.714+0000 7fdefb7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdefc159310 0x7fdefc0ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.714+0000 7fdefb7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdefc159310 0x7fdefc0ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43762/0 (socket says 172.21.15.111:43762) 2024-09-13T17:58:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.714+0000 7fdefb7fe640 1 -- 172.21.15.111:0/3712392924 learned_addr learned my addr 172.21.15.111:0/3712392924 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.714+0000 7fdefbfff640 1 --2- 172.21.15.111:0/3712392924 >> v2:172.21.15.188:3300/0 conn(0x7fdefc0ce4f0 0x7fdefc0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.714+0000 7fdefb7fe640 1 -- 172.21.15.111:0/3712392924 >> v2:172.21.15.188:3300/0 conn(0x7fdefc0ce4f0 msgr2=0x7fdefc0ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.714+0000 7fdefb7fe640 1 --2- 172.21.15.111:0/3712392924 >> v2:172.21.15.188:3300/0 conn(0x7fdefc0ce4f0 0x7fdefc0ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.714+0000 7fdefb7fe640 1 -- 172.21.15.111:0/3712392924 >> v1:172.21.15.188:6789/0 conn(0x7fdefc0b2b40 legacy=0x7fdefc0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.714+0000 7fdefb7fe640 1 -- 172.21.15.111:0/3712392924 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc0cee10 con 0x7fdefc159310 2024-09-13T17:58:18.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.715+0000 7fdefb7fe640 1 --2- 172.21.15.111:0/3712392924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdefc159310 0x7fdefc0ad400 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0b2fc0 tx=0x7fdee402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6daf3d6c6494ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.715+0000 7fdefa7fc640 1 -- 172.21.15.111:0/3712392924 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee4032710 con 0x7fdefc159310 2024-09-13T17:58:18.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.715+0000 7fdefa7fc640 1 -- 172.21.15.111:0/3712392924 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee4030050 con 0x7fdefc159310 2024-09-13T17:58:18.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.715+0000 7fdefa7fc640 1 -- 172.21.15.111:0/3712392924 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee4032ab0 con 0x7fdefc159310 2024-09-13T17:58:18.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.715+0000 7fdf0206b640 1 -- 172.21.15.111:0/3712392924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdefc159310 msgr2=0x7fdefc0ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.715+0000 7fdf0206b640 1 --2- 172.21.15.111:0/3712392924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdefc159310 0x7fdefc0ad400 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0b2fc0 tx=0x7fdee402c850 comp rx=0 tx=0).stop 2024-09-13T17:58:18.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.715+0000 7fdf0206b640 1 -- 172.21.15.111:0/3712392924 shutdown_connections 2024-09-13T17:58:18.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.715+0000 7fdf0206b640 1 --2- 172.21.15.111:0/3712392924 >> v2:172.21.15.188:3300/0 conn(0x7fdefc0ce4f0 0x7fdefc0ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.715+0000 7fdf0206b640 1 --2- 172.21.15.111:0/3712392924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdefc159310 0x7fdefc0ad400 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.716+0000 7fdf0206b640 1 -- 172.21.15.111:0/3712392924 >> 172.21.15.111:0/3712392924 conn(0x7fdefc06a0b0 msgr2=0x7fdefc06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:18.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.716+0000 7fdf0206b640 1 -- 172.21.15.111:0/3712392924 shutdown_connections 2024-09-13T17:58:18.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.716+0000 7fdf0206b640 1 -- 172.21.15.111:0/3712392924 wait complete. 2024-09-13T17:58:18.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.717+0000 7fdf0206b640 1 Processor -- start 2024-09-13T17:58:18.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.717+0000 7fdf0206b640 1 -- start start 2024-09-13T17:58:18.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdf0206b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdefc0b2b40 0x7fdefc0c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdf0206b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdefc0ce4f0 0x7fdefc1518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdf0206b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdefc159310 0x7fdefc0c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdf0206b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc15bb30 con 0x7fdefc0ce4f0 2024-09-13T17:58:18.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdf0206b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0c4a40 con 0x7fdefc0b2b40 2024-09-13T17:58:18.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdf0206b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc15af20 con 0x7fdefc159310 2024-09-13T17:58:18.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdefaffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdefc0ce4f0 0x7fdefc1518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdefaffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdefc0ce4f0 0x7fdefc1518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43764/0 (socket says 172.21.15.111:43764) 2024-09-13T17:58:18.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdefaffd640 1 -- 172.21.15.111:0/304777238 learned_addr learned my addr 172.21.15.111:0/304777238 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:18.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdefb7fe640 1 --2- 172.21.15.111:0/304777238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdefc0b2b40 0x7fdefc0c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdefbfff640 1 --2- 172.21.15.111:0/304777238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdefc159310 0x7fdefc0c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdefaffd640 1 -- 172.21.15.111:0/304777238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdefc159310 msgr2=0x7fdefc0c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdefaffd640 1 --2- 172.21.15.111:0/304777238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdefc159310 0x7fdefc0c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdefaffd640 1 -- 172.21.15.111:0/304777238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdefc0b2b40 msgr2=0x7fdefc0c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdefaffd640 1 --2- 172.21.15.111:0/304777238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdefc0b2b40 0x7fdefc0c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdefaffd640 1 -- 172.21.15.111:0/304777238 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee400b770 con 0x7fdefc0ce4f0 2024-09-13T17:58:18.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdefaffd640 1 --2- 172.21.15.111:0/304777238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdefc0ce4f0 0x7fdefc1518b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00c430 tx=0x7fdeec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdef3fff640 1 -- 172.21.15.111:0/304777238 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec011790 con 0x7fdefc0ce4f0 2024-09-13T17:58:18.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdef3fff640 1 -- 172.21.15.111:0/304777238 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeec0118f0 con 0x7fdefc0ce4f0 2024-09-13T17:58:18.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdef3fff640 1 -- 172.21.15.111:0/304777238 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec011ce0 con 0x7fdefc0ce4f0 2024-09-13T17:58:18.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdf0206b640 1 -- 172.21.15.111:0/304777238 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdefc15b200 con 0x7fdefc0ce4f0 2024-09-13T17:58:18.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.718+0000 7fdf0206b640 1 -- 172.21.15.111:0/304777238 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc15b7c0 con 0x7fdefc0ce4f0 2024-09-13T17:58:18.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.719+0000 7fdf0206b640 1 -- 172.21.15.111:0/304777238 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdefc06dfa0 con 0x7fdefc0ce4f0 2024-09-13T17:58:18.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.720+0000 7fdef3fff640 1 -- 172.21.15.111:0/304777238 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdeec01c460 con 0x7fdefc0ce4f0 2024-09-13T17:58:18.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.720+0000 7fdef3fff640 1 --2- 172.21.15.111:0/304777238 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdedc06ac00 0x7fdedc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.720+0000 7fdef3fff640 1 -- 172.21.15.111:0/304777238 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdeec0895f0 con 0x7fdefc0ce4f0 2024-09-13T17:58:18.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.720+0000 7fdefb7fe640 1 --2- 172.21.15.111:0/304777238 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdedc06ac00 0x7fdedc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.721+0000 7fdefb7fe640 1 --2- 172.21.15.111:0/304777238 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdedc06ac00 0x7fdedc06d0c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdee4037000 tx=0x7fdee4036040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.724+0000 7fdef3fff640 1 -- 172.21.15.111:0/304777238 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fdeec054410 con 0x7fdefc0ce4f0 2024-09-13T17:58:18.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.886+0000 7fdf0206b640 1 -- 172.21.15.111:0/304777238 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdefc06e4b0 con 0x7fdefc0ce4f0 2024-09-13T17:58:18.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.886+0000 7fdef3fff640 1 -- 172.21.15.111:0/304777238 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdeec057a30 con 0x7fdefc0ce4f0 2024-09-13T17:58:18.889 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:18.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.888+0000 7fdf0206b640 1 -- 172.21.15.111:0/304777238 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdedc06ac00 msgr2=0x7fdedc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.888+0000 7fdf0206b640 1 --2- 172.21.15.111:0/304777238 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdedc06ac00 0x7fdedc06d0c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdee4037000 tx=0x7fdee4036040 comp rx=0 tx=0).stop 2024-09-13T17:58:18.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.888+0000 7fdf0206b640 1 -- 172.21.15.111:0/304777238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdefc0ce4f0 msgr2=0x7fdefc1518b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.888+0000 7fdf0206b640 1 --2- 172.21.15.111:0/304777238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdefc0ce4f0 0x7fdefc1518b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00c430 tx=0x7fdeec00c8f0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.888+0000 7fdf0206b640 1 -- 172.21.15.111:0/304777238 shutdown_connections 2024-09-13T17:58:18.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.888+0000 7fdf0206b640 1 --2- 172.21.15.111:0/304777238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdefc159310 0x7fdefc0c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.888+0000 7fdf0206b640 1 --2- 172.21.15.111:0/304777238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdefc0ce4f0 0x7fdefc1518b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.888+0000 7fdf0206b640 1 --2- 172.21.15.111:0/304777238 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdedc06ac00 0x7fdedc06d0c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.888+0000 7fdf0206b640 1 --2- 172.21.15.111:0/304777238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdefc0b2b40 0x7fdefc0c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.888+0000 7fdf0206b640 1 -- 172.21.15.111:0/304777238 >> 172.21.15.111:0/304777238 conn(0x7fdefc06a0b0 msgr2=0x7fdefc0cb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:18.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.889+0000 7fdf0206b640 1 -- 172.21.15.111:0/304777238 shutdown_connections 2024-09-13T17:58:18.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.889+0000 7fdf0206b640 1 -- 172.21.15.111:0/304777238 wait complete. 2024-09-13T17:58:18.901 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:58:19.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.999+0000 7fafe88d0640 1 Processor -- start 2024-09-13T17:58:19.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:18.999+0000 7fafe88d0640 1 -- start start 2024-09-13T17:58:19.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.000+0000 7fafe88d0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fafe4159310 0x7fafe40ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.000+0000 7fafe88d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafe40b2b40 0x7fafe40ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.000+0000 7fafe88d0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fafe40ab680 0x7fafe40ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.000+0000 7fafe88d0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe40ad940 con 0x7fafe4159310 2024-09-13T17:58:19.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.000+0000 7fafe88d0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe40ccbe0 con 0x7fafe40ab680 2024-09-13T17:58:19.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.000+0000 7fafe88d0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe40b1810 con 0x7fafe40b2b40 2024-09-13T17:58:19.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.000+0000 7fafe259c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fafe4159310 0x7fafe40ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.000+0000 7fafe259c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fafe4159310 0x7fafe40ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43776/0 (socket says 172.21.15.111:43776) 2024-09-13T17:58:19.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.000+0000 7fafe259c640 1 -- 172.21.15.111:0/592352542 learned_addr learned my addr 172.21.15.111:0/592352542 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:19.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.000+0000 7fafe259c640 1 -- 172.21.15.111:0/592352542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafe40b2b40 msgr2=0x7fafe40ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:19.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.000+0000 7fafe259c640 1 --2- 172.21.15.111:0/592352542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafe40b2b40 0x7fafe40ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.000+0000 7fafe259c640 1 -- 172.21.15.111:0/592352542 >> v2:172.21.15.188:3300/0 conn(0x7fafe40ab680 msgr2=0x7fafe40ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:19.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.000+0000 7fafe259c640 1 --2- 172.21.15.111:0/592352542 >> v2:172.21.15.188:3300/0 conn(0x7fafe40ab680 0x7fafe40ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.000+0000 7fafe259c640 1 -- 172.21.15.111:0/592352542 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe406d9e0 con 0x7fafe4159310 2024-09-13T17:58:19.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.001+0000 7fafe259c640 1 --2- 172.21.15.111:0/592352542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafe4159310 0x7fafe40ad400 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fafe40afdf0 tx=0x7fafd402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905250c2d72180f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:19.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.001+0000 7fafe159a640 1 -- 172.21.15.111:0/592352542 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4032440 con 0x7fafe4159310 2024-09-13T17:58:19.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.001+0000 7fafe159a640 1 -- 172.21.15.111:0/592352542 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd40325a0 con 0x7fafe4159310 2024-09-13T17:58:19.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.001+0000 7fafe159a640 1 -- 172.21.15.111:0/592352542 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4032930 con 0x7fafe4159310 2024-09-13T17:58:19.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.002+0000 7fafe88d0640 1 -- 172.21.15.111:0/592352542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafe4159310 msgr2=0x7fafe40ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.002+0000 7fafe88d0640 1 --2- 172.21.15.111:0/592352542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafe4159310 0x7fafe40ad400 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fafe40afdf0 tx=0x7fafd402a160 comp rx=0 tx=0).stop 2024-09-13T17:58:19.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.002+0000 7fafe88d0640 1 -- 172.21.15.111:0/592352542 shutdown_connections 2024-09-13T17:58:19.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.002+0000 7fafe88d0640 1 --2- 172.21.15.111:0/592352542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafe40b2b40 0x7fafe40ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.002+0000 7fafe88d0640 1 --2- 172.21.15.111:0/592352542 >> v2:172.21.15.188:3300/0 conn(0x7fafe40ab680 0x7fafe40ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.002+0000 7fafe88d0640 1 --2- 172.21.15.111:0/592352542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafe4159310 0x7fafe40ad400 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.002+0000 7fafe88d0640 1 -- 172.21.15.111:0/592352542 >> 172.21.15.111:0/592352542 conn(0x7fafe406a0b0 msgr2=0x7fafe406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:19.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.002+0000 7fafe88d0640 1 -- 172.21.15.111:0/592352542 shutdown_connections 2024-09-13T17:58:19.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.002+0000 7fafe88d0640 1 -- 172.21.15.111:0/592352542 wait complete. 2024-09-13T17:58:19.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.003+0000 7fafe88d0640 1 Processor -- start 2024-09-13T17:58:19.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.003+0000 7fafe88d0640 1 -- start start 2024-09-13T17:58:19.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.004+0000 7fafe88d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafe40ab680 0x7fafe4157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.004+0000 7fafe88d0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafe40b2b40 0x7fafe40c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.004+0000 7fafe88d0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafe4159310 0x7fafe40cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.004+0000 7fafe88d0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe40c75e0 con 0x7fafe40b2b40 2024-09-13T17:58:19.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.004+0000 7fafe88d0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe41f4380 con 0x7fafe4159310 2024-09-13T17:58:19.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.004+0000 7fafe88d0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe40c78a0 con 0x7fafe40ab680 2024-09-13T17:58:19.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.004+0000 7fafe1d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafe40b2b40 0x7fafe40c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.004+0000 7fafe1d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafe40b2b40 0x7fafe40c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43786/0 (socket says 172.21.15.111:43786) 2024-09-13T17:58:19.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.004+0000 7fafe1d9b640 1 -- 172.21.15.111:0/3806377073 learned_addr learned my addr 172.21.15.111:0/3806377073 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:19.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.004+0000 7fafe2d9d640 1 --2- 172.21.15.111:0/3806377073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafe4159310 0x7fafe40cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.004+0000 7fafe259c640 1 --2- 172.21.15.111:0/3806377073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafe40ab680 0x7fafe4157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.004+0000 7fafe1d9b640 1 -- 172.21.15.111:0/3806377073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafe40ab680 msgr2=0x7fafe4157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.004+0000 7fafe1d9b640 1 --2- 172.21.15.111:0/3806377073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafe40ab680 0x7fafe4157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.004+0000 7fafe1d9b640 1 -- 172.21.15.111:0/3806377073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafe4159310 msgr2=0x7fafe40cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.004+0000 7fafe1d9b640 1 --2- 172.21.15.111:0/3806377073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafe4159310 0x7fafe40cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.004+0000 7fafe1d9b640 1 -- 172.21.15.111:0/3806377073 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd40094d0 con 0x7fafe40b2b40 2024-09-13T17:58:19.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.005+0000 7fafe1d9b640 1 --2- 172.21.15.111:0/3806377073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafe40b2b40 0x7fafe40c80a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fafd800b4a0 tx=0x7fafd800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:19.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.005+0000 7fafd2ffd640 1 -- 172.21.15.111:0/3806377073 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd8010610 con 0x7fafe40b2b40 2024-09-13T17:58:19.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.005+0000 7fafd2ffd640 1 -- 172.21.15.111:0/3806377073 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd8010770 con 0x7fafe40b2b40 2024-09-13T17:58:19.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.005+0000 7fafe88d0640 1 -- 172.21.15.111:0/3806377073 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe40c7b80 con 0x7fafe40b2b40 2024-09-13T17:58:19.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.005+0000 7fafd2ffd640 1 -- 172.21.15.111:0/3806377073 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd8010b20 con 0x7fafe40b2b40 2024-09-13T17:58:19.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.005+0000 7fafe88d0640 1 -- 172.21.15.111:0/3806377073 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe40c85e0 con 0x7fafe40b2b40 2024-09-13T17:58:19.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.006+0000 7fafe88d0640 1 -- 172.21.15.111:0/3806377073 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe40c8860 con 0x7fafe40b2b40 2024-09-13T17:58:19.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.006+0000 7fafd2ffd640 1 -- 172.21.15.111:0/3806377073 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fafd8010c80 con 0x7fafe40b2b40 2024-09-13T17:58:19.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.007+0000 7fafd2ffd640 1 --2- 172.21.15.111:0/3806377073 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fafc406ac30 0x7fafc406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.007+0000 7fafe259c640 1 --2- 172.21.15.111:0/3806377073 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fafc406ac30 0x7fafc406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.007+0000 7fafd2ffd640 1 -- 172.21.15.111:0/3806377073 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fafd80891e0 con 0x7fafe40b2b40 2024-09-13T17:58:19.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.007+0000 7fafe259c640 1 --2- 172.21.15.111:0/3806377073 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fafc406ac30 0x7fafc406d0f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fafd4034000 tx=0x7fafd402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:19.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.011+0000 7fafd2ffd640 1 -- 172.21.15.111:0/3806377073 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fafd8054140 con 0x7fafe40b2b40 2024-09-13T17:58:19.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.170+0000 7fafe88d0640 1 -- 172.21.15.111:0/3806377073 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fafe40c7d70 con 0x7fafe40b2b40 2024-09-13T17:58:19.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.170+0000 7fafd2ffd640 1 -- 172.21.15.111:0/3806377073 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fafd8057760 con 0x7fafe40b2b40 2024-09-13T17:58:19.172 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:58:19.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.172+0000 7fafe88d0640 1 -- 172.21.15.111:0/3806377073 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fafc406ac30 msgr2=0x7fafc406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.172+0000 7fafe88d0640 1 --2- 172.21.15.111:0/3806377073 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fafc406ac30 0x7fafc406d0f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fafd4034000 tx=0x7fafd402e000 comp rx=0 tx=0).stop 2024-09-13T17:58:19.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.172+0000 7fafe88d0640 1 -- 172.21.15.111:0/3806377073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafe40b2b40 msgr2=0x7fafe40c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.172+0000 7fafe88d0640 1 --2- 172.21.15.111:0/3806377073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafe40b2b40 0x7fafe40c80a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fafd800b4a0 tx=0x7fafd800b960 comp rx=0 tx=0).stop 2024-09-13T17:58:19.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.172+0000 7fafe88d0640 1 -- 172.21.15.111:0/3806377073 shutdown_connections 2024-09-13T17:58:19.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.172+0000 7fafe88d0640 1 --2- 172.21.15.111:0/3806377073 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fafc406ac30 0x7fafc406d0f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.172+0000 7fafe88d0640 1 --2- 172.21.15.111:0/3806377073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafe4159310 0x7fafe40cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.172+0000 7fafe88d0640 1 --2- 172.21.15.111:0/3806377073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafe40b2b40 0x7fafe40c80a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.172+0000 7fafe88d0640 1 --2- 172.21.15.111:0/3806377073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafe40ab680 0x7fafe4157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.172+0000 7fafe88d0640 1 -- 172.21.15.111:0/3806377073 >> 172.21.15.111:0/3806377073 conn(0x7fafe406a0b0 msgr2=0x7fafe40ae660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:19.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.172+0000 7fafe88d0640 1 -- 172.21.15.111:0/3806377073 shutdown_connections 2024-09-13T17:58:19.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.172+0000 7fafe88d0640 1 -- 172.21.15.111:0/3806377073 wait complete. 2024-09-13T17:58:19.184 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:58:19.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.283+0000 7feba6a6c640 1 Processor -- start 2024-09-13T17:58:19.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.283+0000 7feba6a6c640 1 -- start start 2024-09-13T17:58:19.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feba6a6c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba0159310 0x7feba00ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feba6a6c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feba00ce4f0 0x7feba00ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feba6a6c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba00ccbe0 con 0x7feba00b2b40 2024-09-13T17:58:19.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feba6a6c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feba00ad940 con 0x7feba00ce4f0 2024-09-13T17:58:19.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feba6a6c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba00cee80 con 0x7feba0159310 2024-09-13T17:58:19.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb97fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7feba00b2b40 0x7feba00ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33586/0 (socket says 172.21.15.111:33586) 2024-09-13T17:58:19.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb97fff640 1 -- 172.21.15.111:0/1345098797 learned_addr learned my addr 172.21.15.111:0/1345098797 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:19.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb977fe640 1 -- 172.21.15.111:0/1345098797 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995447191 0 0) 0x7feba00ccbe0 con 0x7feba00b2b40 2024-09-13T17:58:19.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb977fe640 1 -- 172.21.15.111:0/1345098797 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb88003520 con 0x7feba00b2b40 2024-09-13T17:58:19.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feba5009640 1 --2- 172.21.15.111:0/1345098797 >> v2:172.21.15.188:3300/0 conn(0x7feba00ce4f0 0x7feba00ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feba4808640 1 --2- 172.21.15.111:0/1345098797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba0159310 0x7feba00ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb977fe640 1 -- 172.21.15.111:0/1345098797 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 80946206 0 0) 0x7feb88003520 con 0x7feba00b2b40 2024-09-13T17:58:19.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb977fe640 1 -- 172.21.15.111:0/1345098797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba0159310 msgr2=0x7feba00ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:19.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb977fe640 1 --2- 172.21.15.111:0/1345098797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba0159310 0x7feba00ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb977fe640 1 -- 172.21.15.111:0/1345098797 >> v2:172.21.15.188:3300/0 conn(0x7feba00ce4f0 msgr2=0x7feba00ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:19.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb977fe640 1 --2- 172.21.15.111:0/1345098797 >> v2:172.21.15.188:3300/0 conn(0x7feba00ce4f0 0x7feba00ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb977fe640 1 -- 172.21.15.111:0/1345098797 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba00b1810 con 0x7feba00b2b40 2024-09-13T17:58:19.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb977fe640 1 -- 172.21.15.111:0/1345098797 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7feb90001670 con 0x7feba00b2b40 2024-09-13T17:58:19.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb977fe640 1 -- 172.21.15.111:0/1345098797 >> v1:172.21.15.111:6789/0 conn(0x7feba00b2b40 legacy=0x7feba00ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb977fe640 1 --2- 172.21.15.111:0/1345098797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb88003970 0x7feb88003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb977fe640 1 --2- 172.21.15.111:0/1345098797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba0159310 0x7feb880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb977fe640 1 --2- 172.21.15.111:0/1345098797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb88004d00 0x7feb88005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb977fe640 1 -- 172.21.15.111:0/1345098797 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba00cee80 con 0x7feb88004d00 2024-09-13T17:58:19.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb977fe640 1 -- 172.21.15.111:0/1345098797 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8800ba60 con 0x7feb88003970 2024-09-13T17:58:19.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feb977fe640 1 -- 172.21.15.111:0/1345098797 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8800de30 con 0x7feba0159310 2024-09-13T17:58:19.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feba5009640 1 --2- 172.21.15.111:0/1345098797 >> v2:172.21.15.188:3300/0 conn(0x7feba00ce4f0 0x7feba00ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:19.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feba4808640 1 --2- 172.21.15.111:0/1345098797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb88003970 0x7feb88003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.284+0000 7feba5009640 1 --2- 172.21.15.111:0/1345098797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb88004d00 0x7feb88005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.285+0000 7feb97fff640 1 --2- 172.21.15.111:0/1345098797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba0159310 0x7feb880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.285+0000 7feba5009640 1 -- 172.21.15.111:0/1345098797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba0159310 msgr2=0x7feb880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:19.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.285+0000 7feba5009640 1 --2- 172.21.15.111:0/1345098797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba0159310 0x7feb880047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.285+0000 7feba5009640 1 -- 172.21.15.111:0/1345098797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb88003970 msgr2=0x7feb88003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:19.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.285+0000 7feba5009640 1 --2- 172.21.15.111:0/1345098797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb88003970 0x7feb88003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.285+0000 7feba5009640 1 -- 172.21.15.111:0/1345098797 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb88003520 con 0x7feb88004d00 2024-09-13T17:58:19.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.285+0000 7feba4808640 1 --2- 172.21.15.111:0/1345098797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb88003970 0x7feb88003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:19.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.286+0000 7feba5009640 1 --2- 172.21.15.111:0/1345098797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb88004d00 0x7feb88005180 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7feb980073e0 tx=0x7feb880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcf066c022277ae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:19.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.286+0000 7feb977fe640 1 -- 172.21.15.111:0/1345098797 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9802c680 con 0x7feb88004d00 2024-09-13T17:58:19.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.286+0000 7feb977fe640 1 -- 172.21.15.111:0/1345098797 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9802c7e0 con 0x7feb88004d00 2024-09-13T17:58:19.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.286+0000 7feb977fe640 1 -- 172.21.15.111:0/1345098797 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9802cb50 con 0x7feb88004d00 2024-09-13T17:58:19.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.286+0000 7feba6a6c640 1 -- 172.21.15.111:0/1345098797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb88004d00 msgr2=0x7feb88005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.286+0000 7feba6a6c640 1 --2- 172.21.15.111:0/1345098797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb88004d00 0x7feb88005180 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7feb980073e0 tx=0x7feb880057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.286+0000 7feba5009640 1 -- 172.21.15.111:0/1345098797 reap_dead start 2024-09-13T17:58:19.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.286+0000 7feba6a6c640 1 -- 172.21.15.111:0/1345098797 shutdown_connections 2024-09-13T17:58:19.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.286+0000 7feba6a6c640 1 -- 172.21.15.111:0/1345098797 >> 172.21.15.111:0/1345098797 conn(0x7feba006a0b0 msgr2=0x7feba006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:19.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.286+0000 7feba6a6c640 1 -- 172.21.15.111:0/1345098797 shutdown_connections 2024-09-13T17:58:19.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.286+0000 7feba6a6c640 1 -- 172.21.15.111:0/1345098797 wait complete. 2024-09-13T17:58:19.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.287+0000 7feba6a6c640 1 Processor -- start 2024-09-13T17:58:19.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.288+0000 7feba6a6c640 1 -- start start 2024-09-13T17:58:19.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.288+0000 7feba6a6c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb88004d00 0x7feba00c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.288+0000 7feba6a6c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feba00c46d0 0x7feba00c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.288+0000 7feba6a6c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feba0151e00 0x7feba0152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.288+0000 7feba6a6c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba01527c0 con 0x7feba0151e00 2024-09-13T17:58:19.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.288+0000 7feba6a6c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba01e0f40 con 0x7feba00c46d0 2024-09-13T17:58:19.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.288+0000 7feba6a6c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba01e1220 con 0x7feb88004d00 2024-09-13T17:58:19.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.288+0000 7feba5009640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feba0151e00 0x7feba0152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.289+0000 7feba5009640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feba0151e00 0x7feba0152280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43806/0 (socket says 172.21.15.111:43806) 2024-09-13T17:58:19.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.289+0000 7feba5009640 1 -- 172.21.15.111:0/2390814983 learned_addr learned my addr 172.21.15.111:0/2390814983 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:19.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.289+0000 7feb97fff640 1 --2- 172.21.15.111:0/2390814983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feba00c46d0 0x7feba00c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.289+0000 7feba4808640 1 --2- 172.21.15.111:0/2390814983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb88004d00 0x7feba00c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.289+0000 7feba5009640 1 -- 172.21.15.111:0/2390814983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb88004d00 msgr2=0x7feba00c4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.289+0000 7feba5009640 1 --2- 172.21.15.111:0/2390814983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb88004d00 0x7feba00c4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.289+0000 7feba5009640 1 -- 172.21.15.111:0/2390814983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feba00c46d0 msgr2=0x7feba00c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.289+0000 7feba5009640 1 --2- 172.21.15.111:0/2390814983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feba00c46d0 0x7feba00c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.289+0000 7feba5009640 1 -- 172.21.15.111:0/2390814983 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba01e1500 con 0x7feba0151e00 2024-09-13T17:58:19.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.289+0000 7feba5009640 1 --2- 172.21.15.111:0/2390814983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feba0151e00 0x7feba0152280 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7feb98007500 tx=0x7feb98004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:19.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.289+0000 7feb957fa640 1 -- 172.21.15.111:0/2390814983 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9802d870 con 0x7feba0151e00 2024-09-13T17:58:19.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.289+0000 7feb957fa640 1 -- 172.21.15.111:0/2390814983 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9802d9d0 con 0x7feba0151e00 2024-09-13T17:58:19.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.289+0000 7feba6a6c640 1 -- 172.21.15.111:0/2390814983 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba01e17e0 con 0x7feba0151e00 2024-09-13T17:58:19.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.289+0000 7feb957fa640 1 -- 172.21.15.111:0/2390814983 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9802dd80 con 0x7feba0151e00 2024-09-13T17:58:19.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.289+0000 7feba6a6c640 1 -- 172.21.15.111:0/2390814983 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba0152ba0 con 0x7feba0151e00 2024-09-13T17:58:19.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.290+0000 7feba6a6c640 1 -- 172.21.15.111:0/2390814983 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba01e1d00 con 0x7feba0151e00 2024-09-13T17:58:19.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.291+0000 7feb957fa640 1 -- 172.21.15.111:0/2390814983 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7feb98041420 con 0x7feba0151e00 2024-09-13T17:58:19.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.291+0000 7feb957fa640 1 --2- 172.21.15.111:0/2390814983 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7feb8c06ac00 0x7feb8c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.291+0000 7feb957fa640 1 -- 172.21.15.111:0/2390814983 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7feb9803d070 con 0x7feba0151e00 2024-09-13T17:58:19.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.291+0000 7feba4808640 1 --2- 172.21.15.111:0/2390814983 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7feb8c06ac00 0x7feb8c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.291+0000 7feba4808640 1 --2- 172.21.15.111:0/2390814983 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7feb8c06ac00 0x7feb8c06d0c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00b5d0 tx=0x7feb9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:19.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.295+0000 7feb957fa640 1 -- 172.21.15.111:0/2390814983 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7feba01e1d00 con 0x7feba0151e00 2024-09-13T17:58:19.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.456+0000 7feba6a6c640 1 -- 172.21.15.111:0/2390814983 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7feba01e1d00 con 0x7feba0151e00 2024-09-13T17:58:19.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.457+0000 7feb957fa640 1 -- 172.21.15.111:0/2390814983 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7feba01e1d00 con 0x7feba0151e00 2024-09-13T17:58:19.459 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:19.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.459+0000 7feba6a6c640 1 -- 172.21.15.111:0/2390814983 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7feb8c06ac00 msgr2=0x7feb8c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.459+0000 7feba6a6c640 1 --2- 172.21.15.111:0/2390814983 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7feb8c06ac00 0x7feb8c06d0c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00b5d0 tx=0x7feb9c008040 comp rx=0 tx=0).stop 2024-09-13T17:58:19.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.459+0000 7feba6a6c640 1 -- 172.21.15.111:0/2390814983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feba0151e00 msgr2=0x7feba0152280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.459+0000 7feba6a6c640 1 --2- 172.21.15.111:0/2390814983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feba0151e00 0x7feba0152280 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7feb98007500 tx=0x7feb98004ab0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.459+0000 7feba6a6c640 1 -- 172.21.15.111:0/2390814983 shutdown_connections 2024-09-13T17:58:19.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.459+0000 7feba6a6c640 1 --2- 172.21.15.111:0/2390814983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feba0151e00 0x7feba0152280 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.459+0000 7feba6a6c640 1 --2- 172.21.15.111:0/2390814983 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7feb8c06ac00 0x7feb8c06d0c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.459+0000 7feba6a6c640 1 --2- 172.21.15.111:0/2390814983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feba00c46d0 0x7feba00c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.459+0000 7feba6a6c640 1 --2- 172.21.15.111:0/2390814983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb88004d00 0x7feba00c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.459+0000 7feba6a6c640 1 -- 172.21.15.111:0/2390814983 >> 172.21.15.111:0/2390814983 conn(0x7feba006a0b0 msgr2=0x7feba00708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:19.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.459+0000 7feba6a6c640 1 -- 172.21.15.111:0/2390814983 shutdown_connections 2024-09-13T17:58:19.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.459+0000 7feba6a6c640 1 -- 172.21.15.111:0/2390814983 wait complete. 2024-09-13T17:58:19.472 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:58:19.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.568+0000 7fd65ded7640 1 Processor -- start 2024-09-13T17:58:19.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.568+0000 7fd65ded7640 1 -- start start 2024-09-13T17:58:19.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.569+0000 7fd65ded7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd65815c6e0 0x7fd6580b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.569+0000 7fd65ded7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd6580d07b0 0x7fd6580b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.569+0000 7fd65ded7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6580d1840 con 0x7fd6580ccb20 2024-09-13T17:58:19.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.569+0000 7fd65ded7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd65806bd20 con 0x7fd65815c6e0 2024-09-13T17:58:19.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.569+0000 7fd65ded7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6580d0c30 con 0x7fd6580d07b0 2024-09-13T17:58:19.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.569+0000 7fd6577fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd65815c6e0 0x7fd6580b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.569+0000 7fd6577fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd65815c6e0 0x7fd6580b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43812/0 (socket says 172.21.15.111:43812) 2024-09-13T17:58:19.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.569+0000 7fd6577fe640 1 -- 172.21.15.111:0/3309031456 learned_addr learned my addr 172.21.15.111:0/3309031456 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:19.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.570+0000 7fd656ffd640 1 --2- 172.21.15.111:0/3309031456 >> v2:172.21.15.188:3300/0 conn(0x7fd6580d07b0 0x7fd6580b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.570+0000 7fd6577fe640 1 -- 172.21.15.111:0/3309031456 >> v2:172.21.15.188:3300/0 conn(0x7fd6580d07b0 msgr2=0x7fd6580b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:19.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.570+0000 7fd6577fe640 1 --2- 172.21.15.111:0/3309031456 >> v2:172.21.15.188:3300/0 conn(0x7fd6580d07b0 0x7fd6580b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.570+0000 7fd6577fe640 1 -- 172.21.15.111:0/3309031456 >> v1:172.21.15.111:6789/0 conn(0x7fd6580ccb20 legacy=0x7fd658160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.570+0000 7fd6577fe640 1 -- 172.21.15.111:0/3309031456 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6580d0f10 con 0x7fd65815c6e0 2024-09-13T17:58:19.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.570+0000 7fd656ffd640 1 --2- 172.21.15.111:0/3309031456 >> v2:172.21.15.188:3300/0 conn(0x7fd6580d07b0 0x7fd6580b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:19.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.570+0000 7fd6577fe640 1 --2- 172.21.15.111:0/3309031456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd65815c6e0 0x7fd6580b3f50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd6580b1d80 tx=0x7fd64802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fe07a2c159ed7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:19.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.570+0000 7fd6567fc640 1 -- 172.21.15.111:0/3309031456 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648032440 con 0x7fd65815c6e0 2024-09-13T17:58:19.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.570+0000 7fd6567fc640 1 -- 172.21.15.111:0/3309031456 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6480325a0 con 0x7fd65815c6e0 2024-09-13T17:58:19.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.570+0000 7fd6567fc640 1 -- 172.21.15.111:0/3309031456 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648032930 con 0x7fd65815c6e0 2024-09-13T17:58:19.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.571+0000 7fd65ded7640 1 -- 172.21.15.111:0/3309031456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd65815c6e0 msgr2=0x7fd6580b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.571+0000 7fd65ded7640 1 --2- 172.21.15.111:0/3309031456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd65815c6e0 0x7fd6580b3f50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd6580b1d80 tx=0x7fd64802a1e0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.571+0000 7fd65ded7640 1 -- 172.21.15.111:0/3309031456 shutdown_connections 2024-09-13T17:58:19.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.571+0000 7fd65ded7640 1 --2- 172.21.15.111:0/3309031456 >> v2:172.21.15.188:3300/0 conn(0x7fd6580d07b0 0x7fd6580b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.571+0000 7fd65ded7640 1 --2- 172.21.15.111:0/3309031456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd65815c6e0 0x7fd6580b3f50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.571+0000 7fd65ded7640 1 -- 172.21.15.111:0/3309031456 >> 172.21.15.111:0/3309031456 conn(0x7fd658089f60 msgr2=0x7fd6580d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:19.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.571+0000 7fd65ded7640 1 -- 172.21.15.111:0/3309031456 shutdown_connections 2024-09-13T17:58:19.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.571+0000 7fd65ded7640 1 -- 172.21.15.111:0/3309031456 wait complete. 2024-09-13T17:58:19.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.572+0000 7fd65ded7640 1 Processor -- start 2024-09-13T17:58:19.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.572+0000 7fd65ded7640 1 -- start start 2024-09-13T17:58:19.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.573+0000 7fd65ded7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6580ccb20 0x7fd6580cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.573+0000 7fd65ded7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6580d07b0 0x7fd6581545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.573+0000 7fd65ded7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd65815c6e0 0x7fd658154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.573+0000 7fd65ded7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6580b8350 con 0x7fd6580d07b0 2024-09-13T17:58:19.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.573+0000 7fd65ded7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6580b7740 con 0x7fd6580ccb20 2024-09-13T17:58:19.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.573+0000 7fd65ded7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6580b7a20 con 0x7fd65815c6e0 2024-09-13T17:58:19.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.573+0000 7fd656ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6580d07b0 0x7fd6581545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.573+0000 7fd656ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6580d07b0 0x7fd6581545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43820/0 (socket says 172.21.15.111:43820) 2024-09-13T17:58:19.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.573+0000 7fd656ffd640 1 -- 172.21.15.111:0/1950616728 learned_addr learned my addr 172.21.15.111:0/1950616728 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:19.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.573+0000 7fd6577fe640 1 --2- 172.21.15.111:0/1950616728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6580ccb20 0x7fd6580cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.573+0000 7fd657fff640 1 --2- 172.21.15.111:0/1950616728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd65815c6e0 0x7fd658154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.574+0000 7fd656ffd640 1 -- 172.21.15.111:0/1950616728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd65815c6e0 msgr2=0x7fd658154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.574+0000 7fd656ffd640 1 --2- 172.21.15.111:0/1950616728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd65815c6e0 0x7fd658154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.574+0000 7fd656ffd640 1 -- 172.21.15.111:0/1950616728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6580ccb20 msgr2=0x7fd6580cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.574+0000 7fd656ffd640 1 --2- 172.21.15.111:0/1950616728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6580ccb20 0x7fd6580cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.574+0000 7fd656ffd640 1 -- 172.21.15.111:0/1950616728 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6480094d0 con 0x7fd6580d07b0 2024-09-13T17:58:19.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.574+0000 7fd656ffd640 1 --2- 172.21.15.111:0/1950616728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6580d07b0 0x7fd6581545f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00c7f0 tx=0x7fd64c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:19.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.574+0000 7fd63ffff640 1 -- 172.21.15.111:0/1950616728 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c007820 con 0x7fd6580d07b0 2024-09-13T17:58:19.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.574+0000 7fd63ffff640 1 -- 172.21.15.111:0/1950616728 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c007980 con 0x7fd6580d07b0 2024-09-13T17:58:19.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.574+0000 7fd65ded7640 1 -- 172.21.15.111:0/1950616728 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6580b7d00 con 0x7fd6580d07b0 2024-09-13T17:58:19.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.574+0000 7fd63ffff640 1 -- 172.21.15.111:0/1950616728 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c007cf0 con 0x7fd6580d07b0 2024-09-13T17:58:19.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.574+0000 7fd65ded7640 1 -- 172.21.15.111:0/1950616728 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6580cc5f0 con 0x7fd6580d07b0 2024-09-13T17:58:19.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.575+0000 7fd65ded7640 1 -- 172.21.15.111:0/1950616728 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6580b4c80 con 0x7fd6580d07b0 2024-09-13T17:58:19.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.575+0000 7fd63ffff640 1 -- 172.21.15.111:0/1950616728 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd64c0113f0 con 0x7fd6580d07b0 2024-09-13T17:58:19.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.576+0000 7fd63ffff640 1 --2- 172.21.15.111:0/1950616728 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd64406ac00 0x7fd64406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.576+0000 7fd63ffff640 1 -- 172.21.15.111:0/1950616728 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd64c0898f0 con 0x7fd6580d07b0 2024-09-13T17:58:19.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.576+0000 7fd6577fe640 1 --2- 172.21.15.111:0/1950616728 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd64406ac00 0x7fd64406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.576+0000 7fd6577fe640 1 --2- 172.21.15.111:0/1950616728 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd64406ac00 0x7fd64406d0c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd648034000 tx=0x7fd64802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:19.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.580+0000 7fd63ffff640 1 -- 172.21.15.111:0/1950616728 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fd64c054850 con 0x7fd6580d07b0 2024-09-13T17:58:19.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.751+0000 7fd65ded7640 1 -- 172.21.15.111:0/1950616728 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd658158f50 con 0x7fd6580d07b0 2024-09-13T17:58:19.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.751+0000 7fd63ffff640 1 -- 172.21.15.111:0/1950616728 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd64c057e70 con 0x7fd6580d07b0 2024-09-13T17:58:19.753 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2024-09-13T17:58:19.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.753+0000 7fd65ded7640 1 -- 172.21.15.111:0/1950616728 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd64406ac00 msgr2=0x7fd64406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.753+0000 7fd65ded7640 1 --2- 172.21.15.111:0/1950616728 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd64406ac00 0x7fd64406d0c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd648034000 tx=0x7fd64802e000 comp rx=0 tx=0).stop 2024-09-13T17:58:19.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.753+0000 7fd65ded7640 1 -- 172.21.15.111:0/1950616728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6580d07b0 msgr2=0x7fd6581545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.753+0000 7fd65ded7640 1 --2- 172.21.15.111:0/1950616728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6580d07b0 0x7fd6581545f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00c7f0 tx=0x7fd64c00ccb0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.753+0000 7fd65ded7640 1 -- 172.21.15.111:0/1950616728 shutdown_connections 2024-09-13T17:58:19.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.753+0000 7fd65ded7640 1 --2- 172.21.15.111:0/1950616728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd65815c6e0 0x7fd658154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.753+0000 7fd65ded7640 1 --2- 172.21.15.111:0/1950616728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6580d07b0 0x7fd6581545f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.753+0000 7fd65ded7640 1 --2- 172.21.15.111:0/1950616728 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd64406ac00 0x7fd64406d0c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.753+0000 7fd65ded7640 1 --2- 172.21.15.111:0/1950616728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6580ccb20 0x7fd6580cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.753+0000 7fd65ded7640 1 -- 172.21.15.111:0/1950616728 >> 172.21.15.111:0/1950616728 conn(0x7fd658089f60 msgr2=0x7fd65815ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:19.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.753+0000 7fd65ded7640 1 -- 172.21.15.111:0/1950616728 shutdown_connections 2024-09-13T17:58:19.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.753+0000 7fd65ded7640 1 -- 172.21.15.111:0/1950616728 wait complete. 2024-09-13T17:58:19.766 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:58:19.766 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:58:19.786 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:19.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.881+0000 7f5db09fe640 1 Processor -- start 2024-09-13T17:58:19.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.881+0000 7f5db09fe640 1 -- start start 2024-09-13T17:58:19.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.882+0000 7f5db09fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dac071590 0x7f5dac0cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.882+0000 7f5db09fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5dac0cd690 0x7f5dac15a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.882+0000 7f5db09fe640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dac0cccc0 con 0x7f5dac0b1350 2024-09-13T17:58:19.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.882+0000 7f5db09fe640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dac0ce5a0 con 0x7f5dac0cd690 2024-09-13T17:58:19.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.882+0000 7f5db09fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac15b590 con 0x7f5dac071590 2024-09-13T17:58:19.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.882+0000 7f5da3fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dac071590 0x7f5dac0cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.882+0000 7f5dab59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5dac0cd690 0x7f5dac15a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.882+0000 7f5da3fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dac071590 0x7f5dac0cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:33274/0 (socket says 172.21.15.111:33274) 2024-09-13T17:58:19.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.882+0000 7f5daad9d640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f5dac0b1350 0x7f5dac0b1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:35394/0 (socket says 172.21.15.111:35394) 2024-09-13T17:58:19.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.882+0000 7f5da3fff640 1 -- 172.21.15.111:0/827944109 learned_addr learned my addr 172.21.15.111:0/827944109 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:19.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.882+0000 7f5dab59e640 1 --2- 172.21.15.111:0/827944109 >> v2:172.21.15.188:3300/0 conn(0x7f5dac0cd690 0x7f5dac15a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:55990/0 (socket says 172.21.15.111:55990) 2024-09-13T17:58:19.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.882+0000 7f5dab59e640 1 -- 172.21.15.111:0/827944109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dac071590 msgr2=0x7f5dac0cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:19.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.882+0000 7f5dab59e640 1 --2- 172.21.15.111:0/827944109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dac071590 0x7f5dac0cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.882+0000 7f5dab59e640 1 -- 172.21.15.111:0/827944109 >> v1:172.21.15.188:6789/0 conn(0x7f5dac0b1350 legacy=0x7f5dac0b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.882+0000 7f5dab59e640 1 -- 172.21.15.111:0/827944109 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dac15afd0 con 0x7f5dac0cd690 2024-09-13T17:58:19.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.882+0000 7f5da3fff640 1 --2- 172.21.15.111:0/827944109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dac071590 0x7f5dac0cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:19.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.883+0000 7f5dab59e640 1 --2- 172.21.15.111:0/827944109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dac0cd690 0x7f5dac15a380 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009d90 tx=0x7f5da402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb2ee6df1bb5e84d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:19.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.883+0000 7f5daa59c640 1 -- 172.21.15.111:0/827944109 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4031600 con 0x7f5dac0cd690 2024-09-13T17:58:19.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.883+0000 7f5daa59c640 1 -- 172.21.15.111:0/827944109 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da402d050 con 0x7f5dac0cd690 2024-09-13T17:58:19.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.883+0000 7f5daa59c640 1 -- 172.21.15.111:0/827944109 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4031980 con 0x7f5dac0cd690 2024-09-13T17:58:19.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.884+0000 7f5db09fe640 1 -- 172.21.15.111:0/827944109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dac0cd690 msgr2=0x7f5dac15a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.884+0000 7f5db09fe640 1 --2- 172.21.15.111:0/827944109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dac0cd690 0x7f5dac15a380 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009d90 tx=0x7f5da402a5d0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.884+0000 7f5db09fe640 1 -- 172.21.15.111:0/827944109 shutdown_connections 2024-09-13T17:58:19.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.884+0000 7f5db09fe640 1 --2- 172.21.15.111:0/827944109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dac0cd690 0x7f5dac15a380 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.884+0000 7f5db09fe640 1 --2- 172.21.15.111:0/827944109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dac071590 0x7f5dac0cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.884+0000 7f5db09fe640 1 -- 172.21.15.111:0/827944109 >> 172.21.15.111:0/827944109 conn(0x7f5dac0b48c0 msgr2=0x7f5dac0b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:19.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.884+0000 7f5db09fe640 1 -- 172.21.15.111:0/827944109 shutdown_connections 2024-09-13T17:58:19.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.884+0000 7f5db09fe640 1 -- 172.21.15.111:0/827944109 wait complete. 2024-09-13T17:58:19.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.885+0000 7f5db09fe640 1 Processor -- start 2024-09-13T17:58:19.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.885+0000 7f5db09fe640 1 -- start start 2024-09-13T17:58:19.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.886+0000 7f5db09fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dac071590 0x7f5dac0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.886+0000 7f5db09fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dac0b1350 0x7f5dac0c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.886+0000 7f5db09fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dac0cd690 0x7f5dac14da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.886+0000 7f5db09fe640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac0c76d0 con 0x7f5dac0b1350 2024-09-13T17:58:19.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.886+0000 7f5db09fe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac14df90 con 0x7f5dac0cd690 2024-09-13T17:58:19.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.886+0000 7f5db09fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac0c78c0 con 0x7f5dac071590 2024-09-13T17:58:19.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.886+0000 7f5da3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dac0b1350 0x7f5dac0c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.886+0000 7f5da3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dac0b1350 0x7f5dac0c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43824/0 (socket says 172.21.15.111:43824) 2024-09-13T17:58:19.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.886+0000 7f5da3fff640 1 -- 172.21.15.111:0/2973198654 learned_addr learned my addr 172.21.15.111:0/2973198654 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:19.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.886+0000 7f5daad9d640 1 --2- 172.21.15.111:0/2973198654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dac071590 0x7f5dac0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.886+0000 7f5dab59e640 1 --2- 172.21.15.111:0/2973198654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dac0cd690 0x7f5dac14da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.886+0000 7f5da3fff640 1 -- 172.21.15.111:0/2973198654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dac071590 msgr2=0x7f5dac0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.886+0000 7f5da3fff640 1 --2- 172.21.15.111:0/2973198654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dac071590 0x7f5dac0b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.886+0000 7f5da3fff640 1 -- 172.21.15.111:0/2973198654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dac0cd690 msgr2=0x7f5dac14da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:19.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.886+0000 7f5da3fff640 1 --2- 172.21.15.111:0/2973198654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dac0cd690 0x7f5dac14da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:19.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.886+0000 7f5da3fff640 1 -- 172.21.15.111:0/2973198654 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d940094d0 con 0x7f5dac0b1350 2024-09-13T17:58:19.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.886+0000 7f5dab59e640 1 --2- 172.21.15.111:0/2973198654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dac0cd690 0x7f5dac14da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:19.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.887+0000 7f5da3fff640 1 --2- 172.21.15.111:0/2973198654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dac0b1350 0x7f5dac0c7190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400e9b0 tx=0x7f5d9400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:19.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.887+0000 7f5da37fe640 1 -- 172.21.15.111:0/2973198654 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9400ca00 con 0x7f5dac0b1350 2024-09-13T17:58:19.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.887+0000 7f5da37fe640 1 -- 172.21.15.111:0/2973198654 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9400cb60 con 0x7f5dac0b1350 2024-09-13T17:58:19.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.887+0000 7f5db09fe640 1 -- 172.21.15.111:0/2973198654 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da4009a70 con 0x7f5dac0b1350 2024-09-13T17:58:19.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.887+0000 7f5da37fe640 1 -- 172.21.15.111:0/2973198654 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d94013470 con 0x7f5dac0b1350 2024-09-13T17:58:19.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.887+0000 7f5db09fe640 1 -- 172.21.15.111:0/2973198654 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dac0c7e80 con 0x7f5dac0b1350 2024-09-13T17:58:19.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.888+0000 7f5db09fe640 1 -- 172.21.15.111:0/2973198654 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dac0cd150 con 0x7f5dac0b1350 2024-09-13T17:58:19.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.888+0000 7f5da37fe640 1 -- 172.21.15.111:0/2973198654 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5d940135d0 con 0x7f5dac0b1350 2024-09-13T17:58:19.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.889+0000 7f5da37fe640 1 --2- 172.21.15.111:0/2973198654 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5d8c06ac00 0x7f5d8c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:19.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.889+0000 7f5da37fe640 1 -- 172.21.15.111:0/2973198654 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5d94089970 con 0x7f5dac0b1350 2024-09-13T17:58:19.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.889+0000 7f5daad9d640 1 --2- 172.21.15.111:0/2973198654 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5d8c06ac00 0x7f5d8c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:19.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.889+0000 7f5daad9d640 1 --2- 172.21.15.111:0/2973198654 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5d8c06ac00 0x7f5d8c06d0c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c000f60 tx=0x7f5d9c007c80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:19.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:19.893+0000 7f5da37fe640 1 -- 172.21.15.111:0/2973198654 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f5d94054820 con 0x7f5dac0b1350 2024-09-13T17:58:20.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.055+0000 7f5db09fe640 1 -- 172.21.15.111:0/2973198654 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dac067250 con 0x7f5dac0b1350 2024-09-13T17:58:20.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.056+0000 7f5da37fe640 1 -- 172.21.15.111:0/2973198654 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18794 (secure 0 0 0) 0x7f5d94057e40 con 0x7f5dac0b1350 2024-09-13T17:58:20.058 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:20.058 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":26,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:07.749113+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:20.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.058+0000 7f5db09fe640 1 -- 172.21.15.111:0/2973198654 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5d8c06ac00 msgr2=0x7f5d8c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.058+0000 7f5db09fe640 1 --2- 172.21.15.111:0/2973198654 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5d8c06ac00 0x7f5d8c06d0c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c000f60 tx=0x7f5d9c007c80 comp rx=0 tx=0).stop 2024-09-13T17:58:20.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.058+0000 7f5db09fe640 1 -- 172.21.15.111:0/2973198654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dac0b1350 msgr2=0x7f5dac0c7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.058+0000 7f5db09fe640 1 --2- 172.21.15.111:0/2973198654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dac0b1350 0x7f5dac0c7190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400e9b0 tx=0x7f5d9400ee70 comp rx=0 tx=0).stop 2024-09-13T17:58:20.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.058+0000 7f5db09fe640 1 -- 172.21.15.111:0/2973198654 shutdown_connections 2024-09-13T17:58:20.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.058+0000 7f5db09fe640 1 --2- 172.21.15.111:0/2973198654 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5d8c06ac00 0x7f5d8c06d0c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.058+0000 7f5db09fe640 1 --2- 172.21.15.111:0/2973198654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dac0cd690 0x7f5dac14da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.058+0000 7f5db09fe640 1 --2- 172.21.15.111:0/2973198654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dac0b1350 0x7f5dac0c7190 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.058+0000 7f5db09fe640 1 --2- 172.21.15.111:0/2973198654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dac071590 0x7f5dac0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.058+0000 7f5db09fe640 1 -- 172.21.15.111:0/2973198654 >> 172.21.15.111:0/2973198654 conn(0x7f5dac0b48c0 msgr2=0x7f5dac0b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:20.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.058+0000 7f5db09fe640 1 -- 172.21.15.111:0/2973198654 shutdown_connections 2024-09-13T17:58:20.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.058+0000 7f5db09fe640 1 -- 172.21.15.111:0/2973198654 wait complete. 2024-09-13T17:58:20.071 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:58:20.072 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:58:20.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.170+0000 7f463dfdb640 1 Processor -- start 2024-09-13T17:58:20.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.170+0000 7f463dfdb640 1 -- start start 2024-09-13T17:58:20.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.171+0000 7f463dfdb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f46380d2760 0x7f46380cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.171+0000 7f463dfdb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46380cde20 0x7f46380d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.171+0000 7f463dfdb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46380cd450 con 0x7f463815c500 2024-09-13T17:58:20.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.171+0000 7f463dfdb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46380d0cf0 con 0x7f46380d2760 2024-09-13T17:58:20.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.171+0000 7f463dfdb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46380d0fd0 con 0x7f46380cde20 2024-09-13T17:58:20.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.171+0000 7f46377fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f463815c500 0x7f46380d2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33614/0 (socket says 172.21.15.111:33614) 2024-09-13T17:58:20.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.171+0000 7f46377fe640 1 -- 172.21.15.111:0/4226533672 learned_addr learned my addr 172.21.15.111:0/4226533672 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:20.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f4637fff640 1 --2- 172.21.15.111:0/4226533672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46380cde20 0x7f46380d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f46367fc640 1 -- 172.21.15.111:0/4226533672 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544258948 0 0) 0x7f46380cd450 con 0x7f463815c500 2024-09-13T17:58:20.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f46367fc640 1 -- 172.21.15.111:0/4226533672 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4620003520 con 0x7f463815c500 2024-09-13T17:58:20.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f4636ffd640 1 --2- 172.21.15.111:0/4226533672 >> v2:172.21.15.188:3300/0 conn(0x7f46380d2760 0x7f46380cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f46367fc640 1 -- 172.21.15.111:0/4226533672 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1848747934 0 0) 0x7f4620003520 con 0x7f463815c500 2024-09-13T17:58:20.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f46367fc640 1 -- 172.21.15.111:0/4226533672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46380cde20 msgr2=0x7f46380d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:20.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f46367fc640 1 --2- 172.21.15.111:0/4226533672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46380cde20 0x7f46380d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f46367fc640 1 -- 172.21.15.111:0/4226533672 >> v2:172.21.15.188:3300/0 conn(0x7f46380d2760 msgr2=0x7f46380cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:20.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f46367fc640 1 --2- 172.21.15.111:0/4226533672 >> v2:172.21.15.188:3300/0 conn(0x7f46380d2760 0x7f46380cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f46367fc640 1 -- 172.21.15.111:0/4226533672 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46380cd0e0 con 0x7f463815c500 2024-09-13T17:58:20.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f46367fc640 1 -- 172.21.15.111:0/4226533672 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f46280019f0 con 0x7f463815c500 2024-09-13T17:58:20.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f46367fc640 1 -- 172.21.15.111:0/4226533672 >> v1:172.21.15.111:6789/0 conn(0x7f463815c500 legacy=0x7f46380d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f46367fc640 1 --2- 172.21.15.111:0/4226533672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46380cde20 0x7f4620003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f46367fc640 1 --2- 172.21.15.111:0/4226533672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4620003ef0 0x7f46200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f46367fc640 1 --2- 172.21.15.111:0/4226533672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4620004d00 0x7f4620005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f46367fc640 1 -- 172.21.15.111:0/4226533672 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46380d0cf0 con 0x7f4620004d00 2024-09-13T17:58:20.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f46367fc640 1 -- 172.21.15.111:0/4226533672 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46380d0fd0 con 0x7f4620003ef0 2024-09-13T17:58:20.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f46367fc640 1 -- 172.21.15.111:0/4226533672 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462000b980 con 0x7f46380cde20 2024-09-13T17:58:20.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f4637fff640 1 --2- 172.21.15.111:0/4226533672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4620004d00 0x7f4620005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f46377fe640 1 --2- 172.21.15.111:0/4226533672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46380cde20 0x7f4620003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f4636ffd640 1 --2- 172.21.15.111:0/4226533672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4620003ef0 0x7f46200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f4637fff640 1 -- 172.21.15.111:0/4226533672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46380cde20 msgr2=0x7f4620003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:20.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f4637fff640 1 --2- 172.21.15.111:0/4226533672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46380cde20 0x7f4620003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f4637fff640 1 -- 172.21.15.111:0/4226533672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4620003ef0 msgr2=0x7f46200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:20.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f4637fff640 1 --2- 172.21.15.111:0/4226533672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4620003ef0 0x7f46200047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f4637fff640 1 -- 172.21.15.111:0/4226533672 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4620003520 con 0x7f4620004d00 2024-09-13T17:58:20.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.172+0000 7f4636ffd640 1 --2- 172.21.15.111:0/4226533672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4620003ef0 0x7f46200047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:20.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.173+0000 7f4637fff640 1 --2- 172.21.15.111:0/4226533672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4620004d00 0x7f4620005180 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4630009ac0 tx=0x7f46200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ebba98dea40db8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:20.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.173+0000 7f46367fc640 1 -- 172.21.15.111:0/4226533672 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630031400 con 0x7f4620004d00 2024-09-13T17:58:20.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.173+0000 7f46367fc640 1 -- 172.21.15.111:0/4226533672 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463002d050 con 0x7f4620004d00 2024-09-13T17:58:20.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.173+0000 7f46367fc640 1 -- 172.21.15.111:0/4226533672 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630031780 con 0x7f4620004d00 2024-09-13T17:58:20.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.174+0000 7f463dfdb640 1 -- 172.21.15.111:0/4226533672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4620004d00 msgr2=0x7f4620005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.174+0000 7f463dfdb640 1 --2- 172.21.15.111:0/4226533672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4620004d00 0x7f4620005180 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4630009ac0 tx=0x7f46200057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.174+0000 7f4637fff640 1 -- 172.21.15.111:0/4226533672 reap_dead start 2024-09-13T17:58:20.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.174+0000 7f463dfdb640 1 -- 172.21.15.111:0/4226533672 shutdown_connections 2024-09-13T17:58:20.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.174+0000 7f463dfdb640 1 -- 172.21.15.111:0/4226533672 >> 172.21.15.111:0/4226533672 conn(0x7f4638089f60 msgr2=0x7f463809b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:20.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.174+0000 7f463dfdb640 1 -- 172.21.15.111:0/4226533672 shutdown_connections 2024-09-13T17:58:20.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.174+0000 7f463dfdb640 1 -- 172.21.15.111:0/4226533672 wait complete. 2024-09-13T17:58:20.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.175+0000 7f463dfdb640 1 Processor -- start 2024-09-13T17:58:20.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.175+0000 7f463dfdb640 1 -- start start 2024-09-13T17:58:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f463dfdb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46380c88e0 0x7f46380cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f463dfdb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f463814f120 0x7f463815dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f463dfdb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463815e0e0 0x7f463814b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f463dfdb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46380c9aa0 con 0x7f463814f120 2024-09-13T17:58:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f463dfdb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46380c9500 con 0x7f463815e0e0 2024-09-13T17:58:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f463dfdb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46380c97c0 con 0x7f46380c88e0 2024-09-13T17:58:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f4636ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f463814f120 0x7f463815dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f4636ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f463814f120 0x7f463815dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43848/0 (socket says 172.21.15.111:43848) 2024-09-13T17:58:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f4636ffd640 1 -- 172.21.15.111:0/1429444699 learned_addr learned my addr 172.21.15.111:0/1429444699 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f4637fff640 1 --2- 172.21.15.111:0/1429444699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463815e0e0 0x7f463814b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f46377fe640 1 --2- 172.21.15.111:0/1429444699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46380c88e0 0x7f46380cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f4636ffd640 1 -- 172.21.15.111:0/1429444699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46380c88e0 msgr2=0x7f46380cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f4636ffd640 1 --2- 172.21.15.111:0/1429444699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46380c88e0 0x7f46380cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f4636ffd640 1 -- 172.21.15.111:0/1429444699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463815e0e0 msgr2=0x7f463814b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f4636ffd640 1 --2- 172.21.15.111:0/1429444699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463815e0e0 0x7f463814b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f4636ffd640 1 -- 172.21.15.111:0/1429444699 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46380cbc10 con 0x7f463814f120 2024-09-13T17:58:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f4637fff640 1 --2- 172.21.15.111:0/1429444699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463815e0e0 0x7f463814b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f4636ffd640 1 --2- 172.21.15.111:0/1429444699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f463814f120 0x7f463815dba0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f462c0115d0 tx=0x7f462c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.176+0000 7f461ffff640 1 -- 172.21.15.111:0/1429444699 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c00f8f0 con 0x7f463814f120 2024-09-13T17:58:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.177+0000 7f461ffff640 1 -- 172.21.15.111:0/1429444699 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c00fa50 con 0x7f463814f120 2024-09-13T17:58:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.177+0000 7f461ffff640 1 -- 172.21.15.111:0/1429444699 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c00fde0 con 0x7f463814f120 2024-09-13T17:58:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.177+0000 7f463dfdb640 1 -- 172.21.15.111:0/1429444699 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46380cbef0 con 0x7f463814f120 2024-09-13T17:58:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.177+0000 7f463dfdb640 1 -- 172.21.15.111:0/1429444699 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46380c8cc0 con 0x7f463814f120 2024-09-13T17:58:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.177+0000 7f463dfdb640 1 -- 172.21.15.111:0/1429444699 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46380cd8e0 con 0x7f463814f120 2024-09-13T17:58:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.178+0000 7f461ffff640 1 -- 172.21.15.111:0/1429444699 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f462c015460 con 0x7f463814f120 2024-09-13T17:58:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.179+0000 7f461ffff640 1 --2- 172.21.15.111:0/1429444699 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f462406ac30 0x7f462406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.179+0000 7f46377fe640 1 --2- 172.21.15.111:0/1429444699 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f462406ac30 0x7f462406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.179+0000 7f461ffff640 1 -- 172.21.15.111:0/1429444699 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f462c089ed0 con 0x7f463814f120 2024-09-13T17:58:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.179+0000 7f46377fe640 1 --2- 172.21.15.111:0/1429444699 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f462406ac30 0x7f462406d0f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4628004bc0 tx=0x7f4628006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.183+0000 7f461ffff640 1 -- 172.21.15.111:0/1429444699 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f462c054d80 con 0x7f463814f120 2024-09-13T17:58:20.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.343+0000 7f463dfdb640 1 -- 172.21.15.111:0/1429444699 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f463806c2e0 con 0x7f463814f120 2024-09-13T17:58:20.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.344+0000 7f461ffff640 1 -- 172.21.15.111:0/1429444699 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f462c0583a0 con 0x7f463814f120 2024-09-13T17:58:20.346 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:58:20.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.346+0000 7f463dfdb640 1 -- 172.21.15.111:0/1429444699 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f462406ac30 msgr2=0x7f462406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.346+0000 7f463dfdb640 1 --2- 172.21.15.111:0/1429444699 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f462406ac30 0x7f462406d0f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4628004bc0 tx=0x7f4628006040 comp rx=0 tx=0).stop 2024-09-13T17:58:20.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.346+0000 7f463dfdb640 1 -- 172.21.15.111:0/1429444699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f463814f120 msgr2=0x7f463815dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.346+0000 7f463dfdb640 1 --2- 172.21.15.111:0/1429444699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f463814f120 0x7f463815dba0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f462c0115d0 tx=0x7f462c011a90 comp rx=0 tx=0).stop 2024-09-13T17:58:20.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.346+0000 7f463dfdb640 1 -- 172.21.15.111:0/1429444699 shutdown_connections 2024-09-13T17:58:20.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.346+0000 7f463dfdb640 1 --2- 172.21.15.111:0/1429444699 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f462406ac30 0x7f462406d0f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.346+0000 7f463dfdb640 1 --2- 172.21.15.111:0/1429444699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463815e0e0 0x7f463814b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.346+0000 7f463dfdb640 1 --2- 172.21.15.111:0/1429444699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f463814f120 0x7f463815dba0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.346+0000 7f463dfdb640 1 --2- 172.21.15.111:0/1429444699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46380c88e0 0x7f46380cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.346+0000 7f463dfdb640 1 -- 172.21.15.111:0/1429444699 >> 172.21.15.111:0/1429444699 conn(0x7f4638089f60 msgr2=0x7f463809b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:20.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.346+0000 7f463dfdb640 1 -- 172.21.15.111:0/1429444699 shutdown_connections 2024-09-13T17:58:20.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.346+0000 7f463dfdb640 1 -- 172.21.15.111:0/1429444699 wait complete. 2024-09-13T17:58:20.359 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:58:20.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.457+0000 7f7f9ec6b640 1 Processor -- start 2024-09-13T17:58:20.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.457+0000 7f7f9ec6b640 1 -- start start 2024-09-13T17:58:20.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.458+0000 7f7f9ec6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9815c500 0x7f7f980d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.458+0000 7f7f9ec6b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7f980cdff0 0x7f7f980d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.458+0000 7f7f9ec6b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f980cd450 con 0x7f7f980d2590 2024-09-13T17:58:20.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.458+0000 7f7f9ec6b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f980d29f0 con 0x7f7f980cdff0 2024-09-13T17:58:20.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.458+0000 7f7f9ec6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f980d0cf0 con 0x7f7f9815c500 2024-09-13T17:58:20.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.458+0000 7f7f9d208640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7f980cdff0 0x7f7f980d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.458+0000 7f7f9d208640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7f980cdff0 0x7f7f980d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43852/0 (socket says 172.21.15.111:43852) 2024-09-13T17:58:20.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.458+0000 7f7f9d208640 1 -- 172.21.15.111:0/520937513 learned_addr learned my addr 172.21.15.111:0/520937513 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:20.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.459+0000 7f7f9ca07640 1 --2- 172.21.15.111:0/520937513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9815c500 0x7f7f980d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.459+0000 7f7f9d208640 1 -- 172.21.15.111:0/520937513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9815c500 msgr2=0x7f7f980d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:20.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.459+0000 7f7f9d208640 1 --2- 172.21.15.111:0/520937513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9815c500 0x7f7f980d2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.459+0000 7f7f9d208640 1 -- 172.21.15.111:0/520937513 >> v1:172.21.15.188:6789/0 conn(0x7f7f980d2590 legacy=0x7f7f980cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:20.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.459+0000 7f7f9d208640 1 -- 172.21.15.111:0/520937513 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f980d0fd0 con 0x7f7f980cdff0 2024-09-13T17:58:20.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.459+0000 7f7f9ca07640 1 --2- 172.21.15.111:0/520937513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9815c500 0x7f7f980d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:20.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.459+0000 7f7f9d208640 1 --2- 172.21.15.111:0/520937513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f980cdff0 0x7f7f980d0000 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009e50 tx=0x7f7f9002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22cf1802095a6aa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:20.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.459+0000 7f7f8f7fe640 1 -- 172.21.15.111:0/520937513 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90031600 con 0x7f7f980cdff0 2024-09-13T17:58:20.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.460+0000 7f7f8f7fe640 1 -- 172.21.15.111:0/520937513 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9002d050 con 0x7f7f980cdff0 2024-09-13T17:58:20.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.460+0000 7f7f8f7fe640 1 -- 172.21.15.111:0/520937513 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90031980 con 0x7f7f980cdff0 2024-09-13T17:58:20.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.460+0000 7f7f9ec6b640 1 -- 172.21.15.111:0/520937513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f980cdff0 msgr2=0x7f7f980d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.460+0000 7f7f9ec6b640 1 --2- 172.21.15.111:0/520937513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f980cdff0 0x7f7f980d0000 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009e50 tx=0x7f7f9002a630 comp rx=0 tx=0).stop 2024-09-13T17:58:20.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.460+0000 7f7f9ec6b640 1 -- 172.21.15.111:0/520937513 shutdown_connections 2024-09-13T17:58:20.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.460+0000 7f7f9ec6b640 1 --2- 172.21.15.111:0/520937513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f980cdff0 0x7f7f980d0000 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.460+0000 7f7f9ec6b640 1 --2- 172.21.15.111:0/520937513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9815c500 0x7f7f980d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.460+0000 7f7f9ec6b640 1 -- 172.21.15.111:0/520937513 >> 172.21.15.111:0/520937513 conn(0x7f7f98089f60 msgr2=0x7f7f9809b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:20.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.460+0000 7f7f9ec6b640 1 -- 172.21.15.111:0/520937513 shutdown_connections 2024-09-13T17:58:20.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.460+0000 7f7f9ec6b640 1 -- 172.21.15.111:0/520937513 wait complete. 2024-09-13T17:58:20.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.461+0000 7f7f9ec6b640 1 Processor -- start 2024-09-13T17:58:20.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.461+0000 7f7f9ec6b640 1 -- start start 2024-09-13T17:58:20.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.462+0000 7f7f9ec6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f980cdff0 0x7f7f980b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.462+0000 7f7f9ec6b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f980d2590 0x7f7f98159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.462+0000 7f7f9ec6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9815c500 0x7f7f980c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.462+0000 7f7f9ec6b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f98159a60 con 0x7f7f980cdff0 2024-09-13T17:58:20.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.462+0000 7f7f9ec6b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f980b2020 con 0x7f7f980d2590 2024-09-13T17:58:20.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.462+0000 7f7f9ec6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f980b2300 con 0x7f7f9815c500 2024-09-13T17:58:20.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.462+0000 7f7f9ca07640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f980cdff0 0x7f7f980b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.462+0000 7f7f9ca07640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f980cdff0 0x7f7f980b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43868/0 (socket says 172.21.15.111:43868) 2024-09-13T17:58:20.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.462+0000 7f7f9ca07640 1 -- 172.21.15.111:0/2591228227 learned_addr learned my addr 172.21.15.111:0/2591228227 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:20.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.462+0000 7f7f8ffff640 1 --2- 172.21.15.111:0/2591228227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f980d2590 0x7f7f98159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.462+0000 7f7f9d208640 1 --2- 172.21.15.111:0/2591228227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9815c500 0x7f7f980c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.463+0000 7f7f9ca07640 1 -- 172.21.15.111:0/2591228227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9815c500 msgr2=0x7f7f980c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.463+0000 7f7f9ca07640 1 --2- 172.21.15.111:0/2591228227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9815c500 0x7f7f980c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.463+0000 7f7f9ca07640 1 -- 172.21.15.111:0/2591228227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f980d2590 msgr2=0x7f7f98159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.463+0000 7f7f9ca07640 1 --2- 172.21.15.111:0/2591228227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f980d2590 0x7f7f98159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.463+0000 7f7f9ca07640 1 -- 172.21.15.111:0/2591228227 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f90009b30 con 0x7f7f980cdff0 2024-09-13T17:58:20.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.463+0000 7f7f9ca07640 1 --2- 172.21.15.111:0/2591228227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f980cdff0 0x7f7f980b1a40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400d7d0 tx=0x7f7f9400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:20.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.463+0000 7f7f8d7fa640 1 -- 172.21.15.111:0/2591228227 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94004030 con 0x7f7f980cdff0 2024-09-13T17:58:20.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.463+0000 7f7f8d7fa640 1 -- 172.21.15.111:0/2591228227 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f94004190 con 0x7f7f980cdff0 2024-09-13T17:58:20.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.463+0000 7f7f9ec6b640 1 -- 172.21.15.111:0/2591228227 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f980b25e0 con 0x7f7f980cdff0 2024-09-13T17:58:20.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.463+0000 7f7f8d7fa640 1 -- 172.21.15.111:0/2591228227 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94004520 con 0x7f7f980cdff0 2024-09-13T17:58:20.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.463+0000 7f7f9ec6b640 1 -- 172.21.15.111:0/2591228227 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f980b2ba0 con 0x7f7f980cdff0 2024-09-13T17:58:20.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.464+0000 7f7f9ec6b640 1 -- 172.21.15.111:0/2591228227 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f980d0970 con 0x7f7f980cdff0 2024-09-13T17:58:20.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.464+0000 7f7f8d7fa640 1 -- 172.21.15.111:0/2591228227 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7f94009640 con 0x7f7f980cdff0 2024-09-13T17:58:20.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.465+0000 7f7f8d7fa640 1 --2- 172.21.15.111:0/2591228227 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7f8406ac30 0x7f7f8406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.465+0000 7f7f8d7fa640 1 -- 172.21.15.111:0/2591228227 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7f94089820 con 0x7f7f980cdff0 2024-09-13T17:58:20.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.465+0000 7f7f8ffff640 1 --2- 172.21.15.111:0/2591228227 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7f8406ac30 0x7f7f8406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.465+0000 7f7f8ffff640 1 --2- 172.21.15.111:0/2591228227 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7f8406ac30 0x7f7f8406d0f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f9815f740 tx=0x7f7f88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:20.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.469+0000 7f7f8d7fa640 1 -- 172.21.15.111:0/2591228227 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f7f94054640 con 0x7f7f980cdff0 2024-09-13T17:58:20.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.629+0000 7f7f9ec6b640 1 -- 172.21.15.111:0/2591228227 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7f9806c2e0 con 0x7f7f980cdff0 2024-09-13T17:58:20.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.629+0000 7f7f8d7fa640 1 -- 172.21.15.111:0/2591228227 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7f94057c60 con 0x7f7f980cdff0 2024-09-13T17:58:20.631 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:20.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.631+0000 7f7f9ec6b640 1 -- 172.21.15.111:0/2591228227 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7f8406ac30 msgr2=0x7f7f8406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.631+0000 7f7f9ec6b640 1 --2- 172.21.15.111:0/2591228227 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7f8406ac30 0x7f7f8406d0f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f9815f740 tx=0x7f7f88009040 comp rx=0 tx=0).stop 2024-09-13T17:58:20.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.631+0000 7f7f9ec6b640 1 -- 172.21.15.111:0/2591228227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f980cdff0 msgr2=0x7f7f980b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.631+0000 7f7f9ec6b640 1 --2- 172.21.15.111:0/2591228227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f980cdff0 0x7f7f980b1a40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400d7d0 tx=0x7f7f9400dc90 comp rx=0 tx=0).stop 2024-09-13T17:58:20.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.631+0000 7f7f9ec6b640 1 -- 172.21.15.111:0/2591228227 shutdown_connections 2024-09-13T17:58:20.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.631+0000 7f7f9ec6b640 1 --2- 172.21.15.111:0/2591228227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9815c500 0x7f7f980c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.631+0000 7f7f9ec6b640 1 --2- 172.21.15.111:0/2591228227 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7f8406ac30 0x7f7f8406d0f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.631+0000 7f7f9ec6b640 1 --2- 172.21.15.111:0/2591228227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f980d2590 0x7f7f98159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.631+0000 7f7f9ec6b640 1 --2- 172.21.15.111:0/2591228227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f980cdff0 0x7f7f980b1a40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.631+0000 7f7f9ec6b640 1 -- 172.21.15.111:0/2591228227 >> 172.21.15.111:0/2591228227 conn(0x7f7f98089f60 msgr2=0x7f7f980c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:20.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.631+0000 7f7f9ec6b640 1 -- 172.21.15.111:0/2591228227 shutdown_connections 2024-09-13T17:58:20.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.631+0000 7f7f9ec6b640 1 -- 172.21.15.111:0/2591228227 wait complete. 2024-09-13T17:58:20.644 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:58:20.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.741+0000 7fa48d9b5640 1 Processor -- start 2024-09-13T17:58:20.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.741+0000 7fa48d9b5640 1 -- start start 2024-09-13T17:58:20.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.742+0000 7fa48d9b5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa4881652d0 0x7fa4881676c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.742+0000 7fa48d9b5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa48806b6c0 con 0x7fa48815fbc0 2024-09-13T17:58:20.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.742+0000 7fa48d9b5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4880ce500 con 0x7fa4880d22d0 2024-09-13T17:58:20.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.742+0000 7fa48d9b5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4880cdc60 con 0x7fa4881652d0 2024-09-13T17:58:20.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.742+0000 7fa486ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa48815fbc0 0x7fa4880cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33628/0 (socket says 172.21.15.111:33628) 2024-09-13T17:58:20.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.742+0000 7fa486ffd640 1 -- 172.21.15.111:0/4189957243 learned_addr learned my addr 172.21.15.111:0/4189957243 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:20.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa4867fc640 1 -- 172.21.15.111:0/4189957243 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144212960 0 0) 0x7fa48806b6c0 con 0x7fa48815fbc0 2024-09-13T17:58:20.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa4877fe640 1 --2- 172.21.15.111:0/4189957243 >> v2:172.21.15.188:3300/0 conn(0x7fa4881652d0 0x7fa4881676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa4867fc640 1 -- 172.21.15.111:0/4189957243 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa474003520 con 0x7fa48815fbc0 2024-09-13T17:58:20.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa4867fc640 1 -- 172.21.15.111:0/4189957243 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1825010962 0 0) 0x7fa474003520 con 0x7fa48815fbc0 2024-09-13T17:58:20.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa4867fc640 1 -- 172.21.15.111:0/4189957243 >> v2:172.21.15.188:3300/0 conn(0x7fa4881652d0 msgr2=0x7fa4881676c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:20.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa4867fc640 1 --2- 172.21.15.111:0/4189957243 >> v2:172.21.15.188:3300/0 conn(0x7fa4881652d0 0x7fa4881676c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa4867fc640 1 -- 172.21.15.111:0/4189957243 >> v1:172.21.15.188:6789/0 conn(0x7fa4880d22d0 legacy=0x7fa4880d2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa4867fc640 1 -- 172.21.15.111:0/4189957243 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4880cdf40 con 0x7fa48815fbc0 2024-09-13T17:58:20.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa4867fc640 1 -- 172.21.15.111:0/4189957243 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7fa478001670 con 0x7fa48815fbc0 2024-09-13T17:58:20.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa4867fc640 1 -- 172.21.15.111:0/4189957243 >> v1:172.21.15.111:6789/0 conn(0x7fa48815fbc0 legacy=0x7fa4880cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa4867fc640 1 --2- 172.21.15.111:0/4189957243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa474003970 0x7fa474003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa4867fc640 1 --2- 172.21.15.111:0/4189957243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa474004370 0x7fa474004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa4867fc640 1 --2- 172.21.15.111:0/4189957243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa474005040 0x7fa47400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa4867fc640 1 -- 172.21.15.111:0/4189957243 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4880cdc60 con 0x7fa474004370 2024-09-13T17:58:20.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa4867fc640 1 -- 172.21.15.111:0/4189957243 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47400bfa0 con 0x7fa474005040 2024-09-13T17:58:20.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa4867fc640 1 -- 172.21.15.111:0/4189957243 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47400c160 con 0x7fa474003970 2024-09-13T17:58:20.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa47e7fc640 1 --2- 172.21.15.111:0/4189957243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa474004370 0x7fa474004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa486ffd640 1 --2- 172.21.15.111:0/4189957243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa474003970 0x7fa474003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa4877fe640 1 --2- 172.21.15.111:0/4189957243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa474005040 0x7fa47400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa47e7fc640 1 -- 172.21.15.111:0/4189957243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa474003970 msgr2=0x7fa474003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:20.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa47e7fc640 1 --2- 172.21.15.111:0/4189957243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa474003970 0x7fa474003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa47e7fc640 1 -- 172.21.15.111:0/4189957243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa474005040 msgr2=0x7fa47400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:20.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa47e7fc640 1 --2- 172.21.15.111:0/4189957243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa474005040 0x7fa47400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.743+0000 7fa47e7fc640 1 -- 172.21.15.111:0/4189957243 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa474003520 con 0x7fa474004370 2024-09-13T17:58:20.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.744+0000 7fa4877fe640 1 -- 172.21.15.111:0/4189957243 reap_dead start 2024-09-13T17:58:20.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.744+0000 7fa47e7fc640 1 --2- 172.21.15.111:0/4189957243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa474004370 0x7fa474004840 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa4700018b0 tx=0x7fa4740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f02524826c743eb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:20.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.744+0000 7fa4867fc640 1 -- 172.21.15.111:0/4189957243 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47002abc0 con 0x7fa474004370 2024-09-13T17:58:20.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.744+0000 7fa4867fc640 1 -- 172.21.15.111:0/4189957243 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47002ad40 con 0x7fa474004370 2024-09-13T17:58:20.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.744+0000 7fa4867fc640 1 -- 172.21.15.111:0/4189957243 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa470032680 con 0x7fa474004370 2024-09-13T17:58:20.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.745+0000 7fa48d9b5640 1 -- 172.21.15.111:0/4189957243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa474004370 msgr2=0x7fa474004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.745+0000 7fa48d9b5640 1 --2- 172.21.15.111:0/4189957243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa474004370 0x7fa474004840 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa4700018b0 tx=0x7fa4740057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.745+0000 7fa48d9b5640 1 -- 172.21.15.111:0/4189957243 shutdown_connections 2024-09-13T17:58:20.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.745+0000 7fa48d9b5640 1 --2- 172.21.15.111:0/4189957243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa474004370 0x7fa474004840 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.745+0000 7fa48d9b5640 1 -- 172.21.15.111:0/4189957243 >> 172.21.15.111:0/4189957243 conn(0x7fa48809b830 msgr2=0x7fa488089f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:20.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.745+0000 7fa48d9b5640 1 -- 172.21.15.111:0/4189957243 shutdown_connections 2024-09-13T17:58:20.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.745+0000 7fa48d9b5640 1 -- 172.21.15.111:0/4189957243 wait complete. 2024-09-13T17:58:20.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.746+0000 7fa48d9b5640 1 Processor -- start 2024-09-13T17:58:20.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.746+0000 7fa48d9b5640 1 -- start start 2024-09-13T17:58:20.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.747+0000 7fa48d9b5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa474004370 0x7fa48814f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.747+0000 7fa48d9b5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4880b70b0 0x7fa4880b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.747+0000 7fa48d9b5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4880b7510 0x7fa4880b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.747+0000 7fa48d9b5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4880b8390 con 0x7fa4880b70b0 2024-09-13T17:58:20.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.747+0000 7fa48d9b5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4880b4460 con 0x7fa4880b7510 2024-09-13T17:58:20.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.747+0000 7fa48d9b5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4880b4740 con 0x7fa474004370 2024-09-13T17:58:20.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.747+0000 7fa47e7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4880b70b0 0x7fa4880b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.747+0000 7fa47e7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4880b70b0 0x7fa4880b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43880/0 (socket says 172.21.15.111:43880) 2024-09-13T17:58:20.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.747+0000 7fa47e7fc640 1 -- 172.21.15.111:0/2655815455 learned_addr learned my addr 172.21.15.111:0/2655815455 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:20.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.747+0000 7fa486ffd640 1 --2- 172.21.15.111:0/2655815455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa474004370 0x7fa48814f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.747+0000 7fa47e7fc640 1 -- 172.21.15.111:0/2655815455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa474004370 msgr2=0x7fa48814f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.747+0000 7fa47e7fc640 1 --2- 172.21.15.111:0/2655815455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa474004370 0x7fa48814f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.747+0000 7fa47e7fc640 1 -- 172.21.15.111:0/2655815455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4880b7510 msgr2=0x7fa4880b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.747+0000 7fa47e7fc640 1 --2- 172.21.15.111:0/2655815455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4880b7510 0x7fa4880b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.747+0000 7fa47e7fc640 1 -- 172.21.15.111:0/2655815455 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4880b4a20 con 0x7fa4880b70b0 2024-09-13T17:58:20.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.748+0000 7fa47e7fc640 1 --2- 172.21.15.111:0/2655815455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4880b70b0 0x7fa4880b2050 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa47002a5a0 tx=0x7fa470001280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:20.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.748+0000 7fa484ff9640 1 -- 172.21.15.111:0/2655815455 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa470007450 con 0x7fa4880b70b0 2024-09-13T17:58:20.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.748+0000 7fa484ff9640 1 -- 172.21.15.111:0/2655815455 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4700075d0 con 0x7fa4880b70b0 2024-09-13T17:58:20.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.748+0000 7fa48d9b5640 1 -- 172.21.15.111:0/2655815455 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4880b5390 con 0x7fa4880b70b0 2024-09-13T17:58:20.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.748+0000 7fa484ff9640 1 -- 172.21.15.111:0/2655815455 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47003e690 con 0x7fa4880b70b0 2024-09-13T17:58:20.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.748+0000 7fa48d9b5640 1 -- 172.21.15.111:0/2655815455 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4880b4080 con 0x7fa4880b70b0 2024-09-13T17:58:20.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.749+0000 7fa48d9b5640 1 -- 172.21.15.111:0/2655815455 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4880af090 con 0x7fa4880b70b0 2024-09-13T17:58:20.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.749+0000 7fa484ff9640 1 -- 172.21.15.111:0/2655815455 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa47003e7f0 con 0x7fa4880b70b0 2024-09-13T17:58:20.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.750+0000 7fa484ff9640 1 --2- 172.21.15.111:0/2655815455 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa46806ac00 0x7fa46806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:20.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.750+0000 7fa486ffd640 1 --2- 172.21.15.111:0/2655815455 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa46806ac00 0x7fa46806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:20.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.750+0000 7fa484ff9640 1 -- 172.21.15.111:0/2655815455 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa4700a9fc0 con 0x7fa4880b70b0 2024-09-13T17:58:20.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.750+0000 7fa486ffd640 1 --2- 172.21.15.111:0/2655815455 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa46806ac00 0x7fa46806d0c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa478001050 tx=0x7fa478009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:20.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.754+0000 7fa484ff9640 1 -- 172.21.15.111:0/2655815455 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fa4700dd9c0 con 0x7fa4880b70b0 2024-09-13T17:58:20.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.917+0000 7fa48d9b5640 1 -- 172.21.15.111:0/2655815455 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa4880d2e10 con 0x7fa4880b70b0 2024-09-13T17:58:20.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.917+0000 7fa484ff9640 1 -- 172.21.15.111:0/2655815455 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4700370a0 con 0x7fa4880b70b0 2024-09-13T17:58:20.919 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:58:20.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.919+0000 7fa48d9b5640 1 -- 172.21.15.111:0/2655815455 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa46806ac00 msgr2=0x7fa46806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.919+0000 7fa48d9b5640 1 --2- 172.21.15.111:0/2655815455 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa46806ac00 0x7fa46806d0c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa478001050 tx=0x7fa478009040 comp rx=0 tx=0).stop 2024-09-13T17:58:20.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.919+0000 7fa48d9b5640 1 -- 172.21.15.111:0/2655815455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4880b70b0 msgr2=0x7fa4880b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:20.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.919+0000 7fa48d9b5640 1 --2- 172.21.15.111:0/2655815455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4880b70b0 0x7fa4880b2050 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa47002a5a0 tx=0x7fa470001280 comp rx=0 tx=0).stop 2024-09-13T17:58:20.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.919+0000 7fa48d9b5640 1 -- 172.21.15.111:0/2655815455 shutdown_connections 2024-09-13T17:58:20.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.919+0000 7fa48d9b5640 1 --2- 172.21.15.111:0/2655815455 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa46806ac00 0x7fa46806d0c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.919+0000 7fa48d9b5640 1 --2- 172.21.15.111:0/2655815455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4880b7510 0x7fa4880b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.919+0000 7fa48d9b5640 1 --2- 172.21.15.111:0/2655815455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4880b70b0 0x7fa4880b2050 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.919+0000 7fa48d9b5640 1 --2- 172.21.15.111:0/2655815455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa474004370 0x7fa48814f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:20.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.919+0000 7fa48d9b5640 1 -- 172.21.15.111:0/2655815455 >> 172.21.15.111:0/2655815455 conn(0x7fa48809b830 msgr2=0x7fa4880c9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:20.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.919+0000 7fa48d9b5640 1 -- 172.21.15.111:0/2655815455 shutdown_connections 2024-09-13T17:58:20.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:20.919+0000 7fa48d9b5640 1 -- 172.21.15.111:0/2655815455 wait complete. 2024-09-13T17:58:20.932 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:58:21.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.030+0000 7fd34722b640 1 Processor -- start 2024-09-13T17:58:21.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.030+0000 7fd34722b640 1 -- start start 2024-09-13T17:58:21.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.031+0000 7fd34722b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd3400ad9b0 0x7fd34006d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.031+0000 7fd34722b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3400c1990 0x7fd3400cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.031+0000 7fd34722b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd34006dfc0 con 0x7fd3400b1d90 2024-09-13T17:58:21.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.031+0000 7fd34722b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd340160d60 con 0x7fd3400ad9b0 2024-09-13T17:58:21.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.031+0000 7fd34722b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34006cd40 con 0x7fd3400c1990 2024-09-13T17:58:21.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.031+0000 7fd344fc7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd3400ad9b0 0x7fd34006d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:21.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.031+0000 7fd344fc7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd3400ad9b0 0x7fd34006d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43892/0 (socket says 172.21.15.111:43892) 2024-09-13T17:58:21.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.031+0000 7fd344fc7640 1 -- 172.21.15.111:0/144352512 learned_addr learned my addr 172.21.15.111:0/144352512 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:21.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.031+0000 7fd344fc7640 1 -- 172.21.15.111:0/144352512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3400c1990 msgr2=0x7fd3400cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:21.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.031+0000 7fd344fc7640 1 --2- 172.21.15.111:0/144352512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3400c1990 0x7fd3400cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.031+0000 7fd344fc7640 1 -- 172.21.15.111:0/144352512 >> v1:172.21.15.188:6789/0 conn(0x7fd3400b1d90 legacy=0x7fd3400c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:21.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.031+0000 7fd344fc7640 1 -- 172.21.15.111:0/144352512 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34006d020 con 0x7fd3400ad9b0 2024-09-13T17:58:21.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.032+0000 7fd344fc7640 1 --2- 172.21.15.111:0/144352512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3400ad9b0 0x7fd34006d8c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0098a0 tx=0x7fd33c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9491fa5da8d6b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:21.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.032+0000 7fd3377fe640 1 -- 172.21.15.111:0/144352512 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c032440 con 0x7fd3400ad9b0 2024-09-13T17:58:21.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.032+0000 7fd3377fe640 1 -- 172.21.15.111:0/144352512 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33c0325a0 con 0x7fd3400ad9b0 2024-09-13T17:58:21.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.032+0000 7fd3377fe640 1 -- 172.21.15.111:0/144352512 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c032930 con 0x7fd3400ad9b0 2024-09-13T17:58:21.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.032+0000 7fd34722b640 1 -- 172.21.15.111:0/144352512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3400ad9b0 msgr2=0x7fd34006d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:21.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.032+0000 7fd34722b640 1 --2- 172.21.15.111:0/144352512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3400ad9b0 0x7fd34006d8c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0098a0 tx=0x7fd33c02a250 comp rx=0 tx=0).stop 2024-09-13T17:58:21.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.033+0000 7fd34722b640 1 -- 172.21.15.111:0/144352512 shutdown_connections 2024-09-13T17:58:21.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.033+0000 7fd34722b640 1 --2- 172.21.15.111:0/144352512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3400c1990 0x7fd3400cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.033+0000 7fd34722b640 1 --2- 172.21.15.111:0/144352512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3400ad9b0 0x7fd34006d8c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.033+0000 7fd34722b640 1 -- 172.21.15.111:0/144352512 >> 172.21.15.111:0/144352512 conn(0x7fd34006a0b0 msgr2=0x7fd34006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.033+0000 7fd34722b640 1 -- 172.21.15.111:0/144352512 shutdown_connections 2024-09-13T17:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.033+0000 7fd34722b640 1 -- 172.21.15.111:0/144352512 wait complete. 2024-09-13T17:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.034+0000 7fd34722b640 1 Processor -- start 2024-09-13T17:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.034+0000 7fd34722b640 1 -- start start 2024-09-13T17:58:21.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd34722b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3400ad9b0 0x7fd34015f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd34722b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3400b1d90 0x7fd3400c1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd34722b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3400c1990 0x7fd3401ec2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd34722b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400c3280 con 0x7fd3400ad9b0 2024-09-13T17:58:21.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd34722b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400c2670 con 0x7fd3400c1990 2024-09-13T17:58:21.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd34722b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400c2950 con 0x7fd3400b1d90 2024-09-13T17:58:21.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd344fc7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3400ad9b0 0x7fd34015f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:21.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd344fc7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3400ad9b0 0x7fd34015f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43896/0 (socket says 172.21.15.111:43896) 2024-09-13T17:58:21.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd344fc7640 1 -- 172.21.15.111:0/1157438928 learned_addr learned my addr 172.21.15.111:0/1157438928 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:21.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd337fff640 1 --2- 172.21.15.111:0/1157438928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3400b1d90 0x7fd3400c1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:21.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd3457c8640 1 --2- 172.21.15.111:0/1157438928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3400c1990 0x7fd3401ec2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:21.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd344fc7640 1 -- 172.21.15.111:0/1157438928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3400b1d90 msgr2=0x7fd3400c1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:21.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd344fc7640 1 --2- 172.21.15.111:0/1157438928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3400b1d90 0x7fd3400c1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd344fc7640 1 -- 172.21.15.111:0/1157438928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3400c1990 msgr2=0x7fd3401ec2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:21.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd344fc7640 1 --2- 172.21.15.111:0/1157438928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3400c1990 0x7fd3401ec2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd344fc7640 1 -- 172.21.15.111:0/1157438928 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33c0094d0 con 0x7fd3400ad9b0 2024-09-13T17:58:21.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd337fff640 1 --2- 172.21.15.111:0/1157438928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3400b1d90 0x7fd3400c1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:21.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd344fc7640 1 --2- 172.21.15.111:0/1157438928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3400ad9b0 0x7fd34015f1c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0098a0 tx=0x7fd33c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:21.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd3357fa640 1 -- 172.21.15.111:0/1157438928 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c032440 con 0x7fd3400ad9b0 2024-09-13T17:58:21.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd3357fa640 1 -- 172.21.15.111:0/1157438928 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33c030b80 con 0x7fd3400ad9b0 2024-09-13T17:58:21.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd3357fa640 1 -- 172.21.15.111:0/1157438928 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c004030 con 0x7fd3400ad9b0 2024-09-13T17:58:21.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd34722b640 1 -- 172.21.15.111:0/1157438928 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3400c2c30 con 0x7fd3400ad9b0 2024-09-13T17:58:21.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.035+0000 7fd34722b640 1 -- 172.21.15.111:0/1157438928 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd34015f700 con 0x7fd3400ad9b0 2024-09-13T17:58:21.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.036+0000 7fd34722b640 1 -- 172.21.15.111:0/1157438928 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3400c2a90 con 0x7fd3400ad9b0 2024-09-13T17:58:21.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.037+0000 7fd3357fa640 1 -- 172.21.15.111:0/1157438928 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd33c039040 con 0x7fd3400ad9b0 2024-09-13T17:58:21.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.037+0000 7fd3357fa640 1 --2- 172.21.15.111:0/1157438928 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd32c06ac00 0x7fd32c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.037+0000 7fd337fff640 1 --2- 172.21.15.111:0/1157438928 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd32c06ac00 0x7fd32c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:21.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.037+0000 7fd3357fa640 1 -- 172.21.15.111:0/1157438928 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd33c0a9d00 con 0x7fd3400ad9b0 2024-09-13T17:58:21.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.038+0000 7fd337fff640 1 --2- 172.21.15.111:0/1157438928 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd32c06ac00 0x7fd32c06d0c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd34009c340 tx=0x7fd330009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:21.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.042+0000 7fd3357fa640 1 -- 172.21.15.111:0/1157438928 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fd3400c2a90 con 0x7fd3400ad9b0 2024-09-13T17:58:21.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.202+0000 7fd34722b640 1 -- 172.21.15.111:0/1157438928 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd3400c2a90 con 0x7fd3400ad9b0 2024-09-13T17:58:21.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.202+0000 7fd3357fa640 1 -- 172.21.15.111:0/1157438928 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd3400c2a90 con 0x7fd3400ad9b0 2024-09-13T17:58:21.204 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:21.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.204+0000 7fd34722b640 1 -- 172.21.15.111:0/1157438928 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd32c06ac00 msgr2=0x7fd32c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:21.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.204+0000 7fd34722b640 1 --2- 172.21.15.111:0/1157438928 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd32c06ac00 0x7fd32c06d0c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd34009c340 tx=0x7fd330009040 comp rx=0 tx=0).stop 2024-09-13T17:58:21.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.204+0000 7fd34722b640 1 -- 172.21.15.111:0/1157438928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3400ad9b0 msgr2=0x7fd34015f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:21.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.204+0000 7fd34722b640 1 --2- 172.21.15.111:0/1157438928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3400ad9b0 0x7fd34015f1c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0098a0 tx=0x7fd33c0046c0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.204+0000 7fd34722b640 1 -- 172.21.15.111:0/1157438928 shutdown_connections 2024-09-13T17:58:21.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.204+0000 7fd34722b640 1 --2- 172.21.15.111:0/1157438928 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd32c06ac00 0x7fd32c06d0c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.204+0000 7fd34722b640 1 --2- 172.21.15.111:0/1157438928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3400c1990 0x7fd3401ec2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.204+0000 7fd34722b640 1 --2- 172.21.15.111:0/1157438928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3400b1d90 0x7fd3400c1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.204+0000 7fd34722b640 1 --2- 172.21.15.111:0/1157438928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3400ad9b0 0x7fd34015f1c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.204+0000 7fd34722b640 1 -- 172.21.15.111:0/1157438928 >> 172.21.15.111:0/1157438928 conn(0x7fd34006a0b0 msgr2=0x7fd3400ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:21.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.204+0000 7fd34722b640 1 -- 172.21.15.111:0/1157438928 shutdown_connections 2024-09-13T17:58:21.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.204+0000 7fd34722b640 1 -- 172.21.15.111:0/1157438928 wait complete. 2024-09-13T17:58:21.217 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:58:21.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.315+0000 7f7cdb4c9640 1 Processor -- start 2024-09-13T17:58:21.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.315+0000 7f7cdb4c9640 1 -- start start 2024-09-13T17:58:21.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.316+0000 7f7cdb4c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cd415c280 0x7f7cd406aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.316+0000 7f7cdb4c9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7cd415c660 0x7f7cd406b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.316+0000 7f7cdb4c9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7cd40abae0 0x7f7cd40abf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.316+0000 7f7cdb4c9640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cd40ac480 con 0x7f7cd415c660 2024-09-13T17:58:21.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.316+0000 7f7cdb4c9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cd4066090 con 0x7f7cd40abae0 2024-09-13T17:58:21.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.316+0000 7f7cdb4c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd40ac6f0 con 0x7f7cd415c280 2024-09-13T17:58:21.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.316+0000 7f7cd8a64640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7cd415c660 0x7f7cd406b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:21.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.316+0000 7f7cd8a64640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7cd415c660 0x7f7cd406b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43908/0 (socket says 172.21.15.111:43908) 2024-09-13T17:58:21.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.316+0000 7f7cd8a64640 1 -- 172.21.15.111:0/584531450 learned_addr learned my addr 172.21.15.111:0/584531450 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:21.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.316+0000 7f7cd9265640 1 --2- 172.21.15.111:0/584531450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cd415c280 0x7f7cd406aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:21.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.316+0000 7f7cd9a66640 1 --2- 172.21.15.111:0/584531450 >> v2:172.21.15.188:3300/0 conn(0x7f7cd40abae0 0x7f7cd40abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:21.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.316+0000 7f7cd8a64640 1 -- 172.21.15.111:0/584531450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cd415c280 msgr2=0x7f7cd406aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:21.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.316+0000 7f7cd8a64640 1 --2- 172.21.15.111:0/584531450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cd415c280 0x7f7cd406aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.316+0000 7f7cd8a64640 1 -- 172.21.15.111:0/584531450 >> v2:172.21.15.188:3300/0 conn(0x7f7cd40abae0 msgr2=0x7f7cd40abf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:21.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.316+0000 7f7cd8a64640 1 --2- 172.21.15.111:0/584531450 >> v2:172.21.15.188:3300/0 conn(0x7f7cd40abae0 0x7f7cd40abf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.316+0000 7f7cd8a64640 1 -- 172.21.15.111:0/584531450 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd40ac9d0 con 0x7f7cd415c660 2024-09-13T17:58:21.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.316+0000 7f7cd9265640 1 --2- 172.21.15.111:0/584531450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cd415c280 0x7f7cd406aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:21.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.317+0000 7f7cd8a64640 1 --2- 172.21.15.111:0/584531450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cd415c660 0x7f7cd406b420 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40b3f40 tx=0x7f7cc402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2805efa89d5000 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:21.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.317+0000 7f7ccbfff640 1 -- 172.21.15.111:0/584531450 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4032440 con 0x7f7cd415c660 2024-09-13T17:58:21.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.317+0000 7f7ccbfff640 1 -- 172.21.15.111:0/584531450 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc40325a0 con 0x7f7cd415c660 2024-09-13T17:58:21.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.317+0000 7f7ccbfff640 1 -- 172.21.15.111:0/584531450 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4032930 con 0x7f7cd415c660 2024-09-13T17:58:21.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.317+0000 7f7cdb4c9640 1 -- 172.21.15.111:0/584531450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cd415c660 msgr2=0x7f7cd406b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:21.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.317+0000 7f7cdb4c9640 1 --2- 172.21.15.111:0/584531450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cd415c660 0x7f7cd406b420 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40b3f40 tx=0x7f7cc402a1e0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.318+0000 7f7cdb4c9640 1 -- 172.21.15.111:0/584531450 shutdown_connections 2024-09-13T17:58:21.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.318+0000 7f7cdb4c9640 1 --2- 172.21.15.111:0/584531450 >> v2:172.21.15.188:3300/0 conn(0x7f7cd40abae0 0x7f7cd40abf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.318+0000 7f7cdb4c9640 1 --2- 172.21.15.111:0/584531450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cd415c660 0x7f7cd406b420 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.318+0000 7f7cdb4c9640 1 --2- 172.21.15.111:0/584531450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cd415c280 0x7f7cd406aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.318+0000 7f7cdb4c9640 1 -- 172.21.15.111:0/584531450 >> 172.21.15.111:0/584531450 conn(0x7f7cd40709f0 msgr2=0x7f7cd4070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:21.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.318+0000 7f7cdb4c9640 1 -- 172.21.15.111:0/584531450 shutdown_connections 2024-09-13T17:58:21.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.318+0000 7f7cdb4c9640 1 -- 172.21.15.111:0/584531450 wait complete. 2024-09-13T17:58:21.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.319+0000 7f7cdb4c9640 1 Processor -- start 2024-09-13T17:58:21.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.319+0000 7f7cdb4c9640 1 -- start start 2024-09-13T17:58:21.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cdb4c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd40abae0 0x7f7cd41f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cdb4c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cd415c280 0x7f7cd41516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cdb4c9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cd415c660 0x7f7cd41584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cdb4c9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd41f2670 con 0x7f7cd415c660 2024-09-13T17:58:21.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cdb4c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd4158a20 con 0x7f7cd40abae0 2024-09-13T17:58:21.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cdb4c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd4155090 con 0x7f7cd415c280 2024-09-13T17:58:21.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cd9a66640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cd415c660 0x7f7cd41584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:21.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cd9a66640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cd415c660 0x7f7cd41584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43920/0 (socket says 172.21.15.111:43920) 2024-09-13T17:58:21.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cd9a66640 1 -- 172.21.15.111:0/2360448126 learned_addr learned my addr 172.21.15.111:0/2360448126 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:21.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cd9265640 1 --2- 172.21.15.111:0/2360448126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd40abae0 0x7f7cd41f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:21.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cd8a64640 1 --2- 172.21.15.111:0/2360448126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cd415c280 0x7f7cd41516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:21.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cd9a66640 1 -- 172.21.15.111:0/2360448126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cd415c280 msgr2=0x7f7cd41516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:21.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cd9a66640 1 --2- 172.21.15.111:0/2360448126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cd415c280 0x7f7cd41516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cd9a66640 1 -- 172.21.15.111:0/2360448126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd40abae0 msgr2=0x7f7cd41f2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:21.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cd9a66640 1 --2- 172.21.15.111:0/2360448126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd40abae0 0x7f7cd41f2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cd9a66640 1 -- 172.21.15.111:0/2360448126 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc40094d0 con 0x7f7cd415c660 2024-09-13T17:58:21.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cd9a66640 1 --2- 172.21.15.111:0/2360448126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cd415c660 0x7f7cd41584e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00cd20 tx=0x7f7ccc00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:21.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cc9ffb640 1 -- 172.21.15.111:0/2360448126 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc003b90 con 0x7f7cd415c660 2024-09-13T17:58:21.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cc9ffb640 1 -- 172.21.15.111:0/2360448126 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ccc003cf0 con 0x7f7cd415c660 2024-09-13T17:58:21.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cdb4c9640 1 -- 172.21.15.111:0/2360448126 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd4155370 con 0x7f7cd415c660 2024-09-13T17:58:21.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.320+0000 7f7cc9ffb640 1 -- 172.21.15.111:0/2360448126 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc004240 con 0x7f7cd415c660 2024-09-13T17:58:21.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.321+0000 7f7cdb4c9640 1 -- 172.21.15.111:0/2360448126 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd41f1f70 con 0x7f7cd415c660 2024-09-13T17:58:21.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.322+0000 7f7cdb4c9640 1 -- 172.21.15.111:0/2360448126 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd41f2160 con 0x7f7cd415c660 2024-09-13T17:58:21.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.322+0000 7f7cc9ffb640 1 -- 172.21.15.111:0/2360448126 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7ccc0043a0 con 0x7f7cd415c660 2024-09-13T17:58:21.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.322+0000 7f7cc9ffb640 1 --2- 172.21.15.111:0/2360448126 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cb806ac00 0x7f7cb806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.323+0000 7f7cd9265640 1 --2- 172.21.15.111:0/2360448126 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cb806ac00 0x7f7cb806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:21.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.323+0000 7f7cc9ffb640 1 -- 172.21.15.111:0/2360448126 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7ccc012070 con 0x7f7cd415c660 2024-09-13T17:58:21.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.323+0000 7f7cd9265640 1 --2- 172.21.15.111:0/2360448126 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cb806ac00 0x7f7cb806d0c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000ab60 tx=0x7f7cd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:21.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.327+0000 7f7cc9ffb640 1 -- 172.21.15.111:0/2360448126 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f7cd41f2160 con 0x7f7cd415c660 2024-09-13T17:58:21.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.501+0000 7f7cdb4c9640 1 -- 172.21.15.111:0/2360448126 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7cd41f2820 con 0x7f7cd415c660 2024-09-13T17:58:21.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.502+0000 7f7cc9ffb640 1 -- 172.21.15.111:0/2360448126 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7cd41f2820 con 0x7f7cd415c660 2024-09-13T17:58:21.503 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2024-09-13T17:58:21.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.503+0000 7f7cdb4c9640 1 -- 172.21.15.111:0/2360448126 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cb806ac00 msgr2=0x7f7cb806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:21.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.503+0000 7f7cdb4c9640 1 --2- 172.21.15.111:0/2360448126 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cb806ac00 0x7f7cb806d0c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000ab60 tx=0x7f7cd0008040 comp rx=0 tx=0).stop 2024-09-13T17:58:21.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.503+0000 7f7cdb4c9640 1 -- 172.21.15.111:0/2360448126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cd415c660 msgr2=0x7f7cd41584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:21.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.503+0000 7f7cdb4c9640 1 --2- 172.21.15.111:0/2360448126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cd415c660 0x7f7cd41584e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00cd20 tx=0x7f7ccc00a660 comp rx=0 tx=0).stop 2024-09-13T17:58:21.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.503+0000 7f7cdb4c9640 1 -- 172.21.15.111:0/2360448126 shutdown_connections 2024-09-13T17:58:21.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.503+0000 7f7cdb4c9640 1 --2- 172.21.15.111:0/2360448126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cd415c660 0x7f7cd41584e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.503+0000 7f7cdb4c9640 1 --2- 172.21.15.111:0/2360448126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cd415c280 0x7f7cd41516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.503+0000 7f7cdb4c9640 1 --2- 172.21.15.111:0/2360448126 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f7cb806ac00 0x7f7cb806d0c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.503+0000 7f7cdb4c9640 1 --2- 172.21.15.111:0/2360448126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd40abae0 0x7f7cd41f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.503+0000 7f7cdb4c9640 1 -- 172.21.15.111:0/2360448126 >> 172.21.15.111:0/2360448126 conn(0x7f7cd40709f0 msgr2=0x7f7cd40ce340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:21.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.504+0000 7f7cdb4c9640 1 -- 172.21.15.111:0/2360448126 shutdown_connections 2024-09-13T17:58:21.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.504+0000 7f7cdb4c9640 1 -- 172.21.15.111:0/2360448126 wait complete. 2024-09-13T17:58:21.516 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi188.front.sepia.ceph.com... 2024-09-13T17:58:21.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:21.516 DEBUG:teuthology.orchestra.run.smithi188:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-13T17:58:21.547 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:58:21.547 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-09-13T17:58:21.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:21.547 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-13T17:58:21.678 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:58:21.678 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:58:21.694 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:58:21.694 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:58:21.751 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:21.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.887+0000 7fb3069db640 1 Processor -- start 2024-09-13T17:58:21.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.887+0000 7fb3069db640 1 -- start start 2024-09-13T17:58:21.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.887+0000 7fb3069db640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb3000d07b0 0x7fb30006ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.887+0000 7fb3069db640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3000b4660 0x7fb30006b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.887+0000 7fb3069db640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3000d1840 con 0x7fb30015c6e0 2024-09-13T17:58:21.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.887+0000 7fb3069db640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3000d0c30 con 0x7fb3000d07b0 2024-09-13T17:58:21.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.887+0000 7fb3069db640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3000d0f10 con 0x7fb3000b4660 2024-09-13T17:58:21.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.888+0000 7fb2f7fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb3000d07b0 0x7fb30006ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:21.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.888+0000 7fb2fffff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb30015c6e0 0x7fb3000b3f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33634/0 (socket says 172.21.15.111:33634) 2024-09-13T17:58:21.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.888+0000 7fb2fffff640 1 -- 172.21.15.111:0/2982699412 learned_addr learned my addr 172.21.15.111:0/2982699412 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:21.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.888+0000 7fb2f7fff640 1 -- 172.21.15.111:0/2982699412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3000b4660 msgr2=0x7fb30006b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:21.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.888+0000 7fb304f78640 1 --2- 172.21.15.111:0/2982699412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3000b4660 0x7fb30006b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:21.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.888+0000 7fb2f7fff640 1 --2- 172.21.15.111:0/2982699412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3000b4660 0x7fb30006b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.888+0000 7fb2f7fff640 1 -- 172.21.15.111:0/2982699412 >> v1:172.21.15.111:6789/0 conn(0x7fb30015c6e0 legacy=0x7fb3000b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:21.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.888+0000 7fb2ff7fe640 1 -- 172.21.15.111:0/2982699412 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414336409 0 0) 0x7fb3000d1840 con 0x7fb30015c6e0 2024-09-13T17:58:21.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.888+0000 7fb2f7fff640 1 -- 172.21.15.111:0/2982699412 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3000d11f0 con 0x7fb3000d07b0 2024-09-13T17:58:21.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.888+0000 7fb304f78640 1 --2- 172.21.15.111:0/2982699412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3000b4660 0x7fb30006b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:21.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.889+0000 7fb2f7fff640 1 --2- 172.21.15.111:0/2982699412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3000d07b0 0x7fb30006ad10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80097a0 tx=0x7fb2e802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb554a654bbd45c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:21.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.889+0000 7fb2ff7fe640 1 -- 172.21.15.111:0/2982699412 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e802ae50 con 0x7fb3000d07b0 2024-09-13T17:58:21.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.889+0000 7fb2ff7fe640 1 -- 172.21.15.111:0/2982699412 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e8032440 con 0x7fb3000d07b0 2024-09-13T17:58:21.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.889+0000 7fb2ff7fe640 1 -- 172.21.15.111:0/2982699412 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e80327d0 con 0x7fb3000d07b0 2024-09-13T17:58:21.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.889+0000 7fb3069db640 1 -- 172.21.15.111:0/2982699412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3000d07b0 msgr2=0x7fb30006ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:21.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.889+0000 7fb3069db640 1 --2- 172.21.15.111:0/2982699412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3000d07b0 0x7fb30006ad10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80097a0 tx=0x7fb2e802a100 comp rx=0 tx=0).stop 2024-09-13T17:58:21.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.889+0000 7fb3069db640 1 -- 172.21.15.111:0/2982699412 shutdown_connections 2024-09-13T17:58:21.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.889+0000 7fb3069db640 1 --2- 172.21.15.111:0/2982699412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3000b4660 0x7fb30006b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.889+0000 7fb3069db640 1 --2- 172.21.15.111:0/2982699412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3000d07b0 0x7fb30006ad10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.889+0000 7fb3069db640 1 -- 172.21.15.111:0/2982699412 >> 172.21.15.111:0/2982699412 conn(0x7fb300089f60 msgr2=0x7fb3000d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:21.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.889+0000 7fb3069db640 1 -- 172.21.15.111:0/2982699412 shutdown_connections 2024-09-13T17:58:21.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.889+0000 7fb3069db640 1 -- 172.21.15.111:0/2982699412 wait complete. 2024-09-13T17:58:21.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.891+0000 7fb3069db640 1 Processor -- start 2024-09-13T17:58:21.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.891+0000 7fb3069db640 1 -- start start 2024-09-13T17:58:21.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.891+0000 7fb3069db640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3000b4660 0x7fb3001516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.891+0000 7fb3069db640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3000d07b0 0x7fb3000c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.891+0000 7fb3069db640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30015c6e0 0x7fb3001533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.891+0000 7fb3069db640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3000cac20 con 0x7fb3000d07b0 2024-09-13T17:58:21.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.891+0000 7fb3069db640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3000c9fa0 con 0x7fb30015c6e0 2024-09-13T17:58:21.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.891+0000 7fb3069db640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3000ca280 con 0x7fb3000b4660 2024-09-13T17:58:21.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.891+0000 7fb2f7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3000d07b0 0x7fb3000c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.891+0000 7fb2f7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3000d07b0 0x7fb3000c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43926/0 (socket says 172.21.15.111:43926) 2024-09-13T17:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.891+0000 7fb2f7fff640 1 -- 172.21.15.111:0/1988481603 learned_addr learned my addr 172.21.15.111:0/1988481603 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.892+0000 7fb2f7fff640 1 -- 172.21.15.111:0/1988481603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3000b4660 msgr2=0x7fb3001516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.892+0000 7fb2f7fff640 1 --2- 172.21.15.111:0/1988481603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3000b4660 0x7fb3001516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.892+0000 7fb2f7fff640 1 -- 172.21.15.111:0/1988481603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30015c6e0 msgr2=0x7fb3001533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:21.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.892+0000 7fb2f7fff640 1 --2- 172.21.15.111:0/1988481603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30015c6e0 0x7fb3001533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:21.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.892+0000 7fb2f7fff640 1 -- 172.21.15.111:0/1988481603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e8009400 con 0x7fb3000d07b0 2024-09-13T17:58:21.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.892+0000 7fb2f7fff640 1 --2- 172.21.15.111:0/1988481603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3000d07b0 0x7fb3000c95e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8000c00 tx=0x7fb2e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:21.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.892+0000 7fb2fd7fa640 1 -- 172.21.15.111:0/1988481603 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e803d910 con 0x7fb3000d07b0 2024-09-13T17:58:21.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.892+0000 7fb2fd7fa640 1 -- 172.21.15.111:0/1988481603 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e803da70 con 0x7fb3000d07b0 2024-09-13T17:58:21.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.892+0000 7fb3069db640 1 -- 172.21.15.111:0/1988481603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3000ca560 con 0x7fb3000d07b0 2024-09-13T17:58:21.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.892+0000 7fb2fd7fa640 1 -- 172.21.15.111:0/1988481603 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e803de20 con 0x7fb3000d07b0 2024-09-13T17:58:21.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.892+0000 7fb3069db640 1 -- 172.21.15.111:0/1988481603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb300151c00 con 0x7fb3000d07b0 2024-09-13T17:58:21.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.893+0000 7fb3069db640 1 -- 172.21.15.111:0/1988481603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3000cad60 con 0x7fb3000d07b0 2024-09-13T17:58:21.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.894+0000 7fb2fd7fa640 1 -- 172.21.15.111:0/1988481603 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb2e8039040 con 0x7fb3000d07b0 2024-09-13T17:58:21.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.894+0000 7fb2fd7fa640 1 --2- 172.21.15.111:0/1988481603 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb2e006ac00 0x7fb2e006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:21.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.894+0000 7fb2fd7fa640 1 -- 172.21.15.111:0/1988481603 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb2e8032e40 con 0x7fb3000d07b0 2024-09-13T17:58:21.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.894+0000 7fb2fffff640 1 --2- 172.21.15.111:0/1988481603 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb2e006ac00 0x7fb2e006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:21.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.894+0000 7fb2fffff640 1 --2- 172.21.15.111:0/1988481603 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb2e006ac00 0x7fb2e006d0c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0007fd0 tx=0x7fb2f0007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:21.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:21.898+0000 7fb2fd7fa640 1 -- 172.21.15.111:0/1988481603 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fb3000cad60 con 0x7fb3000d07b0 2024-09-13T17:58:22.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.062+0000 7fb3069db640 1 -- 172.21.15.111:0/1988481603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3000cad60 con 0x7fb3000d07b0 2024-09-13T17:58:22.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.063+0000 7fb2fd7fa640 1 -- 172.21.15.111:0/1988481603 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18794 (secure 0 0 0) 0x7fb3000cad60 con 0x7fb3000d07b0 2024-09-13T17:58:22.066 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:22.066 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":26,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:07.749113+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:22.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.065+0000 7fb3069db640 1 -- 172.21.15.111:0/1988481603 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb2e006ac00 msgr2=0x7fb2e006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.065+0000 7fb3069db640 1 --2- 172.21.15.111:0/1988481603 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb2e006ac00 0x7fb2e006d0c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0007fd0 tx=0x7fb2f0007960 comp rx=0 tx=0).stop 2024-09-13T17:58:22.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.065+0000 7fb3069db640 1 -- 172.21.15.111:0/1988481603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3000d07b0 msgr2=0x7fb3000c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.065+0000 7fb3069db640 1 --2- 172.21.15.111:0/1988481603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3000d07b0 0x7fb3000c95e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8000c00 tx=0x7fb2e80046c0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.065+0000 7fb3069db640 1 -- 172.21.15.111:0/1988481603 shutdown_connections 2024-09-13T17:58:22.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.065+0000 7fb3069db640 1 --2- 172.21.15.111:0/1988481603 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb2e006ac00 0x7fb2e006d0c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.065+0000 7fb3069db640 1 --2- 172.21.15.111:0/1988481603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30015c6e0 0x7fb3001533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.065+0000 7fb3069db640 1 --2- 172.21.15.111:0/1988481603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3000d07b0 0x7fb3000c95e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.065+0000 7fb3069db640 1 --2- 172.21.15.111:0/1988481603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3000b4660 0x7fb3001516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.065+0000 7fb3069db640 1 -- 172.21.15.111:0/1988481603 >> 172.21.15.111:0/1988481603 conn(0x7fb300089f60 msgr2=0x7fb3000d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:22.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.065+0000 7fb3069db640 1 -- 172.21.15.111:0/1988481603 shutdown_connections 2024-09-13T17:58:22.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.065+0000 7fb3069db640 1 -- 172.21.15.111:0/1988481603 wait complete. 2024-09-13T17:58:22.078 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:58:22.078 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:58:22.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.177+0000 7f482ee3a640 1 Processor -- start 2024-09-13T17:58:22.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.178+0000 7f482ee3a640 1 -- start start 2024-09-13T17:58:22.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.178+0000 7f482ee3a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f482815c500 0x7f48280d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:22.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.178+0000 7f482ee3a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48280d2590 0x7f48280cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:22.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.178+0000 7f482ee3a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48280cd450 con 0x7f48280cde20 2024-09-13T17:58:22.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.178+0000 7f482ee3a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48280d29f0 con 0x7f482815c500 2024-09-13T17:58:22.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.178+0000 7f482ee3a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48280d0cf0 con 0x7f48280d2590 2024-09-13T17:58:22.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.179+0000 7f482cbd6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f482815c500 0x7f48280d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:22.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.179+0000 7f4827fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48280d2590 0x7f48280cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:22.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.179+0000 7f482d3d7640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f48280cde20 0x7f48280d0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:35414/0 (socket says 172.21.15.111:35414) 2024-09-13T17:58:22.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.179+0000 7f482cbd6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f482815c500 0x7f48280d2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:56100/0 (socket says 172.21.15.111:56100) 2024-09-13T17:58:22.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.179+0000 7f482d3d7640 1 -- 172.21.15.111:0/390689475 learned_addr learned my addr 172.21.15.111:0/390689475 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:22.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.179+0000 7f482cbd6640 1 -- 172.21.15.111:0/390689475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48280d2590 msgr2=0x7f48280cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:22.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.179+0000 7f482cbd6640 1 --2- 172.21.15.111:0/390689475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48280d2590 0x7f48280cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.179+0000 7f482cbd6640 1 -- 172.21.15.111:0/390689475 >> v1:172.21.15.188:6789/0 conn(0x7f48280cde20 legacy=0x7f48280d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.179+0000 7f482cbd6640 1 -- 172.21.15.111:0/390689475 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48280d0fd0 con 0x7f482815c500 2024-09-13T17:58:22.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.180+0000 7f482cbd6640 1 --2- 172.21.15.111:0/390689475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482815c500 0x7f48280d2050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f48200097a0 tx=0x7f482002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb9676855827f469 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:22.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.180+0000 7f48277fe640 1 -- 172.21.15.111:0/390689475 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4820032440 con 0x7f482815c500 2024-09-13T17:58:22.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.180+0000 7f48277fe640 1 -- 172.21.15.111:0/390689475 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48200325a0 con 0x7f482815c500 2024-09-13T17:58:22.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.180+0000 7f48277fe640 1 -- 172.21.15.111:0/390689475 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4820032930 con 0x7f482815c500 2024-09-13T17:58:22.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.181+0000 7f482ee3a640 1 -- 172.21.15.111:0/390689475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482815c500 msgr2=0x7f48280d2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.181+0000 7f482ee3a640 1 --2- 172.21.15.111:0/390689475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482815c500 0x7f48280d2050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f48200097a0 tx=0x7f482002a180 comp rx=0 tx=0).stop 2024-09-13T17:58:22.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.181+0000 7f482ee3a640 1 -- 172.21.15.111:0/390689475 shutdown_connections 2024-09-13T17:58:22.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.181+0000 7f482ee3a640 1 --2- 172.21.15.111:0/390689475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48280d2590 0x7f48280cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.181+0000 7f482ee3a640 1 --2- 172.21.15.111:0/390689475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482815c500 0x7f48280d2050 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.181+0000 7f482ee3a640 1 -- 172.21.15.111:0/390689475 >> 172.21.15.111:0/390689475 conn(0x7f4828089f60 msgr2=0x7f482809b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:22.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.181+0000 7f482ee3a640 1 -- 172.21.15.111:0/390689475 shutdown_connections 2024-09-13T17:58:22.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.181+0000 7f482ee3a640 1 -- 172.21.15.111:0/390689475 wait complete. 2024-09-13T17:58:22.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.182+0000 7f482ee3a640 1 Processor -- start 2024-09-13T17:58:22.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.182+0000 7f482ee3a640 1 -- start start 2024-09-13T17:58:22.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f482ee3a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48280cde20 0x7f48280b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:22.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f482ee3a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48280d2590 0x7f4828151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:22.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f482ee3a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f482815c500 0x7f48280cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:22.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f482ee3a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48280b6c00 con 0x7f482815c500 2024-09-13T17:58:22.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f482ee3a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48280b7af0 con 0x7f48280d2590 2024-09-13T17:58:22.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f482ee3a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48280b6230 con 0x7f48280cde20 2024-09-13T17:58:22.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f482d3d7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f482815c500 0x7f48280cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:22.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f482d3d7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f482815c500 0x7f48280cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43934/0 (socket says 172.21.15.111:43934) 2024-09-13T17:58:22.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f482d3d7640 1 -- 172.21.15.111:0/1841589147 learned_addr learned my addr 172.21.15.111:0/1841589147 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:22.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f4827fff640 1 --2- 172.21.15.111:0/1841589147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48280d2590 0x7f4828151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:22.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f482cbd6640 1 --2- 172.21.15.111:0/1841589147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48280cde20 0x7f48280b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:22.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f482d3d7640 1 -- 172.21.15.111:0/1841589147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48280cde20 msgr2=0x7f48280b66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f482d3d7640 1 --2- 172.21.15.111:0/1841589147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48280cde20 0x7f48280b66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f482d3d7640 1 -- 172.21.15.111:0/1841589147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48280d2590 msgr2=0x7f4828151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f482d3d7640 1 --2- 172.21.15.111:0/1841589147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48280d2590 0x7f4828151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f482d3d7640 1 -- 172.21.15.111:0/1841589147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4820009400 con 0x7f482815c500 2024-09-13T17:58:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f482cbd6640 1 --2- 172.21.15.111:0/1841589147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48280cde20 0x7f48280b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f4827fff640 1 --2- 172.21.15.111:0/1841589147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48280d2590 0x7f4828151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.183+0000 7f482d3d7640 1 --2- 172.21.15.111:0/1841589147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f482815c500 0x7f48280cb450 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f481800ca80 tx=0x7f481800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.184+0000 7f48257fa640 1 -- 172.21.15.111:0/1841589147 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481800ac30 con 0x7f482815c500 2024-09-13T17:58:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.184+0000 7f48257fa640 1 -- 172.21.15.111:0/1841589147 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481800ad90 con 0x7f482815c500 2024-09-13T17:58:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.184+0000 7f482ee3a640 1 -- 172.21.15.111:0/1841589147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48280c97b0 con 0x7f482815c500 2024-09-13T17:58:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.184+0000 7f48257fa640 1 -- 172.21.15.111:0/1841589147 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818012600 con 0x7f482815c500 2024-09-13T17:58:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.184+0000 7f482ee3a640 1 -- 172.21.15.111:0/1841589147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48280b7d40 con 0x7f482815c500 2024-09-13T17:58:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.185+0000 7f482ee3a640 1 -- 172.21.15.111:0/1841589147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48280b6370 con 0x7f482815c500 2024-09-13T17:58:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.185+0000 7f48257fa640 1 -- 172.21.15.111:0/1841589147 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4818007070 con 0x7f482815c500 2024-09-13T17:58:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.185+0000 7f48257fa640 1 --2- 172.21.15.111:0/1841589147 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f480806ac30 0x7f480806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.185+0000 7f48257fa640 1 -- 172.21.15.111:0/1841589147 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4818089450 con 0x7f482815c500 2024-09-13T17:58:22.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.186+0000 7f482cbd6640 1 --2- 172.21.15.111:0/1841589147 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f480806ac30 0x7f480806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:22.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.186+0000 7f482cbd6640 1 --2- 172.21.15.111:0/1841589147 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f480806ac30 0x7f480806d0f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4820034000 tx=0x7f482002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:22.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.190+0000 7f48257fa640 1 -- 172.21.15.111:0/1841589147 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f48180543b0 con 0x7f482815c500 2024-09-13T17:58:22.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.352+0000 7f482ee3a640 1 -- 172.21.15.111:0/1841589147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f482806c2e0 con 0x7f482815c500 2024-09-13T17:58:22.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.352+0000 7f48257fa640 1 -- 172.21.15.111:0/1841589147 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48180579d0 con 0x7f482815c500 2024-09-13T17:58:22.354 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:58:22.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.354+0000 7f482ee3a640 1 -- 172.21.15.111:0/1841589147 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f480806ac30 msgr2=0x7f480806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.354+0000 7f482ee3a640 1 --2- 172.21.15.111:0/1841589147 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f480806ac30 0x7f480806d0f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4820034000 tx=0x7f482002e000 comp rx=0 tx=0).stop 2024-09-13T17:58:22.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.354+0000 7f482ee3a640 1 -- 172.21.15.111:0/1841589147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f482815c500 msgr2=0x7f48280cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.354+0000 7f482ee3a640 1 --2- 172.21.15.111:0/1841589147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f482815c500 0x7f48280cb450 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f481800ca80 tx=0x7f481800cf40 comp rx=0 tx=0).stop 2024-09-13T17:58:22.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.354+0000 7f482ee3a640 1 -- 172.21.15.111:0/1841589147 shutdown_connections 2024-09-13T17:58:22.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.354+0000 7f482ee3a640 1 --2- 172.21.15.111:0/1841589147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f482815c500 0x7f48280cb450 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.354+0000 7f482ee3a640 1 --2- 172.21.15.111:0/1841589147 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f480806ac30 0x7f480806d0f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.354+0000 7f482ee3a640 1 --2- 172.21.15.111:0/1841589147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48280d2590 0x7f4828151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.354+0000 7f482ee3a640 1 --2- 172.21.15.111:0/1841589147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48280cde20 0x7f48280b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.354+0000 7f482ee3a640 1 -- 172.21.15.111:0/1841589147 >> 172.21.15.111:0/1841589147 conn(0x7f4828089f60 msgr2=0x7f48280b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:22.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.354+0000 7f482ee3a640 1 -- 172.21.15.111:0/1841589147 shutdown_connections 2024-09-13T17:58:22.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.354+0000 7f482ee3a640 1 -- 172.21.15.111:0/1841589147 wait complete. 2024-09-13T17:58:22.366 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:58:22.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.464+0000 7fab87aa3640 1 Processor -- start 2024-09-13T17:58:22.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.465+0000 7fab87aa3640 1 -- start start 2024-09-13T17:58:22.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.465+0000 7fab87aa3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fab800d2760 0x7fab800cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:22.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.465+0000 7fab87aa3640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab800cd450 con 0x7fab800cde20 2024-09-13T17:58:22.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.465+0000 7fab87aa3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab800d0cf0 con 0x7fab8015c500 2024-09-13T17:58:22.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.465+0000 7fab87aa3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab800d0fd0 con 0x7fab800d2760 2024-09-13T17:58:22.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.466+0000 7fab86040640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fab800cde20 0x7fab800d0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:55888/0 (socket says 172.21.15.111:55888) 2024-09-13T17:58:22.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.466+0000 7fab86040640 1 -- 172.21.15.111:0/3475868743 learned_addr learned my addr 172.21.15.111:0/3475868743 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:22.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.466+0000 7fab8503e640 1 --2- 172.21.15.111:0/3475868743 >> v2:172.21.15.188:3300/0 conn(0x7fab800d2760 0x7fab800cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:22.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.466+0000 7fab8483d640 1 -- 172.21.15.111:0/3475868743 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456314309 0 0) 0x7fab800cd450 con 0x7fab800cde20 2024-09-13T17:58:22.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.466+0000 7fab8483d640 1 -- 172.21.15.111:0/3475868743 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab6c003520 con 0x7fab800cde20 2024-09-13T17:58:22.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.466+0000 7fab8483d640 1 -- 172.21.15.111:0/3475868743 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719132578 0 0) 0x7fab800d0cf0 con 0x7fab8015c500 2024-09-13T17:58:22.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.466+0000 7fab8483d640 1 -- 172.21.15.111:0/3475868743 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab800cd450 con 0x7fab8015c500 2024-09-13T17:58:22.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.466+0000 7fab8503e640 1 -- 172.21.15.111:0/3475868743 >> v1:172.21.15.188:6789/0 conn(0x7fab8015c500 legacy=0x7fab800d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.466+0000 7fab8503e640 1 -- 172.21.15.111:0/3475868743 >> v1:172.21.15.111:6789/0 conn(0x7fab800cde20 legacy=0x7fab800d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.466+0000 7fab8503e640 1 -- 172.21.15.111:0/3475868743 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab800cd0e0 con 0x7fab800d2760 2024-09-13T17:58:22.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.467+0000 7fab8503e640 1 --2- 172.21.15.111:0/3475868743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab800d2760 0x7fab800cd8e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fab700097a0 tx=0x7fab7002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bac78ad019c2984 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:22.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.467+0000 7fab8483d640 1 -- 172.21.15.111:0/3475868743 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab70032440 con 0x7fab800d2760 2024-09-13T17:58:22.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.467+0000 7fab8483d640 1 -- 172.21.15.111:0/3475868743 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab700325a0 con 0x7fab800d2760 2024-09-13T17:58:22.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.467+0000 7fab8483d640 1 -- 172.21.15.111:0/3475868743 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab70032910 con 0x7fab800d2760 2024-09-13T17:58:22.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.467+0000 7fab87aa3640 1 -- 172.21.15.111:0/3475868743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab800d2760 msgr2=0x7fab800cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.467+0000 7fab87aa3640 1 --2- 172.21.15.111:0/3475868743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab800d2760 0x7fab800cd8e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fab700097a0 tx=0x7fab7002a230 comp rx=0 tx=0).stop 2024-09-13T17:58:22.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.468+0000 7fab87aa3640 1 -- 172.21.15.111:0/3475868743 shutdown_connections 2024-09-13T17:58:22.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.468+0000 7fab87aa3640 1 --2- 172.21.15.111:0/3475868743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab800d2760 0x7fab800cd8e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.468+0000 7fab87aa3640 1 -- 172.21.15.111:0/3475868743 >> 172.21.15.111:0/3475868743 conn(0x7fab80089f60 msgr2=0x7fab8009b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:22.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.468+0000 7fab87aa3640 1 -- 172.21.15.111:0/3475868743 shutdown_connections 2024-09-13T17:58:22.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.468+0000 7fab87aa3640 1 -- 172.21.15.111:0/3475868743 wait complete. 2024-09-13T17:58:22.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.469+0000 7fab87aa3640 1 Processor -- start 2024-09-13T17:58:22.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.469+0000 7fab87aa3640 1 -- start start 2024-09-13T17:58:22.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.469+0000 7fab87aa3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab800cde20 0x7fab8006b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:22.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.469+0000 7fab87aa3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab800d2760 0x7fab800c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:22.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab87aa3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab8015c500 0x7fab8014f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:22.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab87aa3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab800c9f30 con 0x7fab800cde20 2024-09-13T17:58:22.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab87aa3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab800ca210 con 0x7fab8015c500 2024-09-13T17:58:22.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab87aa3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab800ca4f0 con 0x7fab800d2760 2024-09-13T17:58:22.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab8583f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab800cde20 0x7fab8006b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:22.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab8583f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab800cde20 0x7fab8006b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33794/0 (socket says 172.21.15.111:33794) 2024-09-13T17:58:22.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab8583f640 1 -- 172.21.15.111:0/3476380420 learned_addr learned my addr 172.21.15.111:0/3476380420 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:22.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab86040640 1 --2- 172.21.15.111:0/3476380420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab8015c500 0x7fab8014f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:22.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab8503e640 1 --2- 172.21.15.111:0/3476380420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab800d2760 0x7fab800c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:22.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab8583f640 1 -- 172.21.15.111:0/3476380420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab800d2760 msgr2=0x7fab800c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab8583f640 1 --2- 172.21.15.111:0/3476380420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab800d2760 0x7fab800c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab8583f640 1 -- 172.21.15.111:0/3476380420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab8015c500 msgr2=0x7fab8014f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab8583f640 1 --2- 172.21.15.111:0/3476380420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab8015c500 0x7fab8014f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab8583f640 1 -- 172.21.15.111:0/3476380420 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab70009400 con 0x7fab800cde20 2024-09-13T17:58:22.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab8583f640 1 --2- 172.21.15.111:0/3476380420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab800cde20 0x7fab8006b5f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00c430 tx=0x7fab7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:22.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab767fc640 1 -- 172.21.15.111:0/3476380420 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c0052d0 con 0x7fab800cde20 2024-09-13T17:58:22.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab767fc640 1 -- 172.21.15.111:0/3476380420 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c005450 con 0x7fab800cde20 2024-09-13T17:58:22.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab87aa3640 1 -- 172.21.15.111:0/3476380420 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab800ca7d0 con 0x7fab800cde20 2024-09-13T17:58:22.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab767fc640 1 -- 172.21.15.111:0/3476380420 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c003830 con 0x7fab800cde20 2024-09-13T17:58:22.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.470+0000 7fab87aa3640 1 -- 172.21.15.111:0/3476380420 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8015a0a0 con 0x7fab800cde20 2024-09-13T17:58:22.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.471+0000 7fab87aa3640 1 -- 172.21.15.111:0/3476380420 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab800cacf0 con 0x7fab800cde20 2024-09-13T17:58:22.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.472+0000 7fab767fc640 1 -- 172.21.15.111:0/3476380420 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fab7c003990 con 0x7fab800cde20 2024-09-13T17:58:22.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.472+0000 7fab767fc640 1 --2- 172.21.15.111:0/3476380420 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fab6406ac00 0x7fab6406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:22.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.472+0000 7fab8503e640 1 --2- 172.21.15.111:0/3476380420 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fab6406ac00 0x7fab6406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:22.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.472+0000 7fab767fc640 1 -- 172.21.15.111:0/3476380420 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fab7c088ba0 con 0x7fab800cde20 2024-09-13T17:58:22.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.473+0000 7fab8503e640 1 --2- 172.21.15.111:0/3476380420 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fab6406ac00 0x7fab6406d0c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fab70034000 tx=0x7fab7002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:22.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.477+0000 7fab767fc640 1 -- 172.21.15.111:0/3476380420 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fab800cacf0 con 0x7fab800cde20 2024-09-13T17:58:22.636 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.634+0000 7fab87aa3640 1 -- 172.21.15.111:0/3476380420 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab800cacf0 con 0x7fab800cde20 2024-09-13T17:58:22.636 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.634+0000 7fab767fc640 1 -- 172.21.15.111:0/3476380420 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab7c017020 con 0x7fab800cde20 2024-09-13T17:58:22.636 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:22.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.636+0000 7fab87aa3640 1 -- 172.21.15.111:0/3476380420 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fab6406ac00 msgr2=0x7fab6406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.636+0000 7fab87aa3640 1 --2- 172.21.15.111:0/3476380420 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fab6406ac00 0x7fab6406d0c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fab70034000 tx=0x7fab7002e000 comp rx=0 tx=0).stop 2024-09-13T17:58:22.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.636+0000 7fab87aa3640 1 -- 172.21.15.111:0/3476380420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab800cde20 msgr2=0x7fab8006b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.636+0000 7fab87aa3640 1 --2- 172.21.15.111:0/3476380420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab800cde20 0x7fab8006b5f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00c430 tx=0x7fab7c00c8f0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.636+0000 7fab87aa3640 1 -- 172.21.15.111:0/3476380420 shutdown_connections 2024-09-13T17:58:22.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.636+0000 7fab87aa3640 1 --2- 172.21.15.111:0/3476380420 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fab6406ac00 0x7fab6406d0c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.636+0000 7fab87aa3640 1 --2- 172.21.15.111:0/3476380420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab8015c500 0x7fab8014f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.636+0000 7fab87aa3640 1 --2- 172.21.15.111:0/3476380420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab800d2760 0x7fab800c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.636+0000 7fab87aa3640 1 --2- 172.21.15.111:0/3476380420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab800cde20 0x7fab8006b5f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.636+0000 7fab87aa3640 1 -- 172.21.15.111:0/3476380420 >> 172.21.15.111:0/3476380420 conn(0x7fab80089f60 msgr2=0x7fab800cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:22.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.636+0000 7fab87aa3640 1 -- 172.21.15.111:0/3476380420 shutdown_connections 2024-09-13T17:58:22.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.636+0000 7fab87aa3640 1 -- 172.21.15.111:0/3476380420 wait complete. 2024-09-13T17:58:22.649 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:58:22.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.748+0000 7f0e547a3640 1 Processor -- start 2024-09-13T17:58:22.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.748+0000 7f0e547a3640 1 -- start start 2024-09-13T17:58:22.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.749+0000 7f0e547a3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e4c0b2b40 0x7f0e4c0ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:22.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.749+0000 7f0e547a3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0e4c0ce4f0 0x7f0e4c0ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:22.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.749+0000 7f0e547a3640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4c0ccbe0 con 0x7f0e4c159310 2024-09-13T17:58:22.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.749+0000 7f0e547a3640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4c0ad940 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.749+0000 7f0e547a3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c0ceea0 con 0x7f0e4c0b2b40 2024-09-13T17:58:22.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.749+0000 7f0e52d40640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0e4c0ce4f0 0x7f0e4c0ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:22.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.749+0000 7f0e5253f640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f0e4c159310 0x7f0e4c0ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:55900/0 (socket says 172.21.15.111:55900) 2024-09-13T17:58:22.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.749+0000 7f0e5253f640 1 -- 172.21.15.111:0/567709537 learned_addr learned my addr 172.21.15.111:0/567709537 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:22.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.749+0000 7f0e51d3e640 1 --2- 172.21.15.111:0/567709537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e4c0b2b40 0x7f0e4c0ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:22.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.749+0000 7f0e52d40640 1 -- 172.21.15.111:0/567709537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e4c0b2b40 msgr2=0x7f0e4c0ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:22.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.749+0000 7f0e52d40640 1 --2- 172.21.15.111:0/567709537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e4c0b2b40 0x7f0e4c0ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.749+0000 7f0e52d40640 1 -- 172.21.15.111:0/567709537 >> v1:172.21.15.111:6789/0 conn(0x7f0e4c159310 legacy=0x7f0e4c0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.749+0000 7f0e52d40640 1 -- 172.21.15.111:0/567709537 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4c0b1810 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.749+0000 7f0e5153d640 1 -- 172.21.15.111:0/567709537 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218794067 0 0) 0x7f0e4c0ccbe0 con 0x7f0e4c159310 2024-09-13T17:58:22.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.749+0000 7f0e51d3e640 1 --2- 172.21.15.111:0/567709537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e4c0b2b40 0x7f0e4c0ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:22.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.750+0000 7f0e52d40640 1 --2- 172.21.15.111:0/567709537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4c0ce4f0 0x7f0e4c0ce960 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009dc0 tx=0x7f0e4402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d738df41d0b2bc55 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:22.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.750+0000 7f0e5153d640 1 -- 172.21.15.111:0/567709537 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e44007740 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.750+0000 7f0e5153d640 1 -- 172.21.15.111:0/567709537 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e4402d050 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.750+0000 7f0e5153d640 1 -- 172.21.15.111:0/567709537 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e44007ac0 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.750+0000 7f0e547a3640 1 -- 172.21.15.111:0/567709537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4c0ce4f0 msgr2=0x7f0e4c0ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.750+0000 7f0e547a3640 1 --2- 172.21.15.111:0/567709537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4c0ce4f0 0x7f0e4c0ce960 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009dc0 tx=0x7f0e4402a560 comp rx=0 tx=0).stop 2024-09-13T17:58:22.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.751+0000 7f0e547a3640 1 -- 172.21.15.111:0/567709537 shutdown_connections 2024-09-13T17:58:22.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.751+0000 7f0e547a3640 1 --2- 172.21.15.111:0/567709537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4c0ce4f0 0x7f0e4c0ce960 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.751+0000 7f0e547a3640 1 --2- 172.21.15.111:0/567709537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e4c0b2b40 0x7f0e4c0ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.751+0000 7f0e547a3640 1 -- 172.21.15.111:0/567709537 >> 172.21.15.111:0/567709537 conn(0x7f0e4c06a0b0 msgr2=0x7f0e4c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:22.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.751+0000 7f0e547a3640 1 -- 172.21.15.111:0/567709537 shutdown_connections 2024-09-13T17:58:22.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.751+0000 7f0e547a3640 1 -- 172.21.15.111:0/567709537 wait complete. 2024-09-13T17:58:22.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.752+0000 7f0e547a3640 1 Processor -- start 2024-09-13T17:58:22.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.752+0000 7f0e547a3640 1 -- start start 2024-09-13T17:58:22.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e547a3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e4c0b2b40 0x7f0e4c0c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:22.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e547a3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4c0ce4f0 0x7f0e4c1e0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:22.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e547a3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e4c159310 0x7f0e4c1e0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:22.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e547a3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c1e3730 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e547a3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c1e3a10 con 0x7f0e4c0b2b40 2024-09-13T17:58:22.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e547a3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c1e3cf0 con 0x7f0e4c159310 2024-09-13T17:58:22.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e51d3e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4c0ce4f0 0x7f0e4c1e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:22.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e51d3e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4c0ce4f0 0x7f0e4c1e0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33814/0 (socket says 172.21.15.111:33814) 2024-09-13T17:58:22.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e51d3e640 1 -- 172.21.15.111:0/133197496 learned_addr learned my addr 172.21.15.111:0/133197496 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:22.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e5253f640 1 --2- 172.21.15.111:0/133197496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e4c0b2b40 0x7f0e4c0c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:22.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e52d40640 1 --2- 172.21.15.111:0/133197496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e4c159310 0x7f0e4c1e0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:22.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e51d3e640 1 -- 172.21.15.111:0/133197496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e4c159310 msgr2=0x7f0e4c1e0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e51d3e640 1 --2- 172.21.15.111:0/133197496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e4c159310 0x7f0e4c1e0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e51d3e640 1 -- 172.21.15.111:0/133197496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e4c0b2b40 msgr2=0x7f0e4c0c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e51d3e640 1 --2- 172.21.15.111:0/133197496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e4c0b2b40 0x7f0e4c0c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e51d3e640 1 -- 172.21.15.111:0/133197496 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e44009a70 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e51d3e640 1 --2- 172.21.15.111:0/133197496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4c0ce4f0 0x7f0e4c1e0660 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800e730 tx=0x7f0e4800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:22.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e3effd640 1 -- 172.21.15.111:0/133197496 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48004230 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e3effd640 1 -- 172.21.15.111:0/133197496 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e48004390 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e3effd640 1 -- 172.21.15.111:0/133197496 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4800c8f0 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e547a3640 1 -- 172.21.15.111:0/133197496 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4c1e3fd0 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.753+0000 7f0e547a3640 1 -- 172.21.15.111:0/133197496 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4c1e47d0 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.754+0000 7f0e547a3640 1 -- 172.21.15.111:0/133197496 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e4c06dfa0 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.755+0000 7f0e3effd640 1 -- 172.21.15.111:0/133197496 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0e4800ca50 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.755+0000 7f0e3effd640 1 --2- 172.21.15.111:0/133197496 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0e3006ac00 0x7f0e3006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:22.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.755+0000 7f0e3effd640 1 -- 172.21.15.111:0/133197496 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0e480892b0 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.755+0000 7f0e5253f640 1 --2- 172.21.15.111:0/133197496 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0e3006ac00 0x7f0e3006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:22.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.756+0000 7f0e5253f640 1 --2- 172.21.15.111:0/133197496 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0e3006ac00 0x7f0e3006d0c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0e40000f60 tx=0x7f0e40007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:22.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.759+0000 7f0e3effd640 1 -- 172.21.15.111:0/133197496 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f0e48054160 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.925 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.923+0000 7f0e547a3640 1 -- 172.21.15.111:0/133197496 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0e4c1e3510 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.925 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.923+0000 7f0e3effd640 1 -- 172.21.15.111:0/133197496 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0e48057780 con 0x7f0e4c0ce4f0 2024-09-13T17:58:22.926 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:58:22.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.925+0000 7f0e547a3640 1 -- 172.21.15.111:0/133197496 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0e3006ac00 msgr2=0x7f0e3006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.925+0000 7f0e547a3640 1 --2- 172.21.15.111:0/133197496 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0e3006ac00 0x7f0e3006d0c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0e40000f60 tx=0x7f0e40007c00 comp rx=0 tx=0).stop 2024-09-13T17:58:22.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.925+0000 7f0e547a3640 1 -- 172.21.15.111:0/133197496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4c0ce4f0 msgr2=0x7f0e4c1e0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:22.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.925+0000 7f0e547a3640 1 --2- 172.21.15.111:0/133197496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4c0ce4f0 0x7f0e4c1e0660 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800e730 tx=0x7f0e4800ebf0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.925+0000 7f0e547a3640 1 -- 172.21.15.111:0/133197496 shutdown_connections 2024-09-13T17:58:22.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.925+0000 7f0e547a3640 1 --2- 172.21.15.111:0/133197496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e4c159310 0x7f0e4c1e0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.925+0000 7f0e547a3640 1 --2- 172.21.15.111:0/133197496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4c0ce4f0 0x7f0e4c1e0660 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.925+0000 7f0e547a3640 1 --2- 172.21.15.111:0/133197496 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0e3006ac00 0x7f0e3006d0c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.925+0000 7f0e547a3640 1 --2- 172.21.15.111:0/133197496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e4c0b2b40 0x7f0e4c0c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:22.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.925+0000 7f0e547a3640 1 -- 172.21.15.111:0/133197496 >> 172.21.15.111:0/133197496 conn(0x7f0e4c06a0b0 msgr2=0x7f0e4c070140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:22.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.926+0000 7f0e547a3640 1 -- 172.21.15.111:0/133197496 shutdown_connections 2024-09-13T17:58:22.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:22.926+0000 7f0e547a3640 1 -- 172.21.15.111:0/133197496 wait complete. 2024-09-13T17:58:22.938 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:58:23.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.037+0000 7fc599e3a640 1 Processor -- start 2024-09-13T17:58:23.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.037+0000 7fc599e3a640 1 -- start start 2024-09-13T17:58:23.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.037+0000 7fc599e3a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc59415c500 0x7fc5940d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.037+0000 7fc599e3a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5940cd450 con 0x7fc5940cdff0 2024-09-13T17:58:23.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.037+0000 7fc599e3a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5940d29f0 con 0x7fc5940d2590 2024-09-13T17:58:23.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.037+0000 7fc599e3a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940d0cf0 con 0x7fc59415c500 2024-09-13T17:58:23.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc593fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc5940cdff0 0x7fc5940d0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:55906/0 (socket says 172.21.15.111:55906) 2024-09-13T17:58:23.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc593fff640 1 -- 172.21.15.111:0/3817827233 learned_addr learned my addr 172.21.15.111:0/3817827233 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:23.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5937fe640 1 --2- 172.21.15.111:0/3817827233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc59415c500 0x7fc5940d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5927fc640 1 -- 172.21.15.111:0/3817827233 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330633619 0 0) 0x7fc5940cd450 con 0x7fc5940cdff0 2024-09-13T17:58:23.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5927fc640 1 -- 172.21.15.111:0/3817827233 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc57c003520 con 0x7fc5940cdff0 2024-09-13T17:58:23.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5927fc640 1 -- 172.21.15.111:0/3817827233 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741955194 0 0) 0x7fc5940d29f0 con 0x7fc5940d2590 2024-09-13T17:58:23.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5927fc640 1 -- 172.21.15.111:0/3817827233 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5940cd450 con 0x7fc5940d2590 2024-09-13T17:58:23.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5927fc640 1 -- 172.21.15.111:0/3817827233 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2454537374 0 0) 0x7fc57c003520 con 0x7fc5940cdff0 2024-09-13T17:58:23.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5927fc640 1 -- 172.21.15.111:0/3817827233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc59415c500 msgr2=0x7fc5940d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:23.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5927fc640 1 --2- 172.21.15.111:0/3817827233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc59415c500 0x7fc5940d2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5927fc640 1 -- 172.21.15.111:0/3817827233 >> v1:172.21.15.188:6789/0 conn(0x7fc5940d2590 legacy=0x7fc5940cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5927fc640 1 -- 172.21.15.111:0/3817827233 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5940d0fd0 con 0x7fc5940cdff0 2024-09-13T17:58:23.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5927fc640 1 -- 172.21.15.111:0/3817827233 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7fc58c001670 con 0x7fc5940cdff0 2024-09-13T17:58:23.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5927fc640 1 -- 172.21.15.111:0/3817827233 >> v1:172.21.15.111:6789/0 conn(0x7fc5940cdff0 legacy=0x7fc5940d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5927fc640 1 --2- 172.21.15.111:0/3817827233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc59415c500 0x7fc57c003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5927fc640 1 --2- 172.21.15.111:0/3817827233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc57c004090 0x7fc57c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5927fc640 1 --2- 172.21.15.111:0/3817827233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc57c004d00 0x7fc57c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5927fc640 1 -- 172.21.15.111:0/3817827233 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940d0cf0 con 0x7fc57c004090 2024-09-13T17:58:23.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5927fc640 1 -- 172.21.15.111:0/3817827233 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c00bb10 con 0x7fc57c004d00 2024-09-13T17:58:23.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5927fc640 1 -- 172.21.15.111:0/3817827233 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c00bca0 con 0x7fc59415c500 2024-09-13T17:58:23.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc592ffd640 1 --2- 172.21.15.111:0/3817827233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc57c004090 0x7fc57c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.038+0000 7fc5937fe640 1 --2- 172.21.15.111:0/3817827233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc59415c500 0x7fc57c003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.039+0000 7fc593fff640 1 --2- 172.21.15.111:0/3817827233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc57c004d00 0x7fc57c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.039+0000 7fc592ffd640 1 -- 172.21.15.111:0/3817827233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc59415c500 msgr2=0x7fc57c003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:23.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.039+0000 7fc592ffd640 1 --2- 172.21.15.111:0/3817827233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc59415c500 0x7fc57c003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.039+0000 7fc592ffd640 1 -- 172.21.15.111:0/3817827233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc57c004d00 msgr2=0x7fc57c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:23.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.039+0000 7fc592ffd640 1 --2- 172.21.15.111:0/3817827233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc57c004d00 0x7fc57c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.039+0000 7fc592ffd640 1 -- 172.21.15.111:0/3817827233 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57c003520 con 0x7fc57c004090 2024-09-13T17:58:23.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.039+0000 7fc592ffd640 1 --2- 172.21.15.111:0/3817827233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc57c004090 0x7fc57c0047c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc5880018a0 tx=0x7fc57c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db33827cb8b13ae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:23.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.039+0000 7fc5927fc640 1 -- 172.21.15.111:0/3817827233 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5880052d0 con 0x7fc57c004090 2024-09-13T17:58:23.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.039+0000 7fc5927fc640 1 -- 172.21.15.111:0/3817827233 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc588005430 con 0x7fc57c004090 2024-09-13T17:58:23.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.039+0000 7fc5927fc640 1 -- 172.21.15.111:0/3817827233 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58802ac60 con 0x7fc57c004090 2024-09-13T17:58:23.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.040+0000 7fc599e3a640 1 -- 172.21.15.111:0/3817827233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc57c004090 msgr2=0x7fc57c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.040+0000 7fc599e3a640 1 --2- 172.21.15.111:0/3817827233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc57c004090 0x7fc57c0047c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc5880018a0 tx=0x7fc57c0057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.040+0000 7fc593fff640 1 -- 172.21.15.111:0/3817827233 reap_dead start 2024-09-13T17:58:23.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.040+0000 7fc599e3a640 1 -- 172.21.15.111:0/3817827233 shutdown_connections 2024-09-13T17:58:23.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.040+0000 7fc599e3a640 1 -- 172.21.15.111:0/3817827233 >> 172.21.15.111:0/3817827233 conn(0x7fc594089f60 msgr2=0x7fc59409b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:23.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.040+0000 7fc599e3a640 1 -- 172.21.15.111:0/3817827233 shutdown_connections 2024-09-13T17:58:23.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.040+0000 7fc599e3a640 1 -- 172.21.15.111:0/3817827233 wait complete. 2024-09-13T17:58:23.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.041+0000 7fc599e3a640 1 Processor -- start 2024-09-13T17:58:23.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.041+0000 7fc599e3a640 1 -- start start 2024-09-13T17:58:23.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.042+0000 7fc599e3a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc57c004090 0x7fc5941469b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.042+0000 7fc599e3a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc594150930 0x7fc5940b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.042+0000 7fc599e3a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5940b7770 0x7fc5940b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.042+0000 7fc599e3a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc594146ef0 con 0x7fc5940b7770 2024-09-13T17:58:23.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.042+0000 7fc599e3a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940b7c80 con 0x7fc57c004090 2024-09-13T17:58:23.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.042+0000 7fc599e3a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940b7f60 con 0x7fc594150930 2024-09-13T17:58:23.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.042+0000 7fc593fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5940b7770 0x7fc5940b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.042+0000 7fc593fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5940b7770 0x7fc5940b5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33818/0 (socket says 172.21.15.111:33818) 2024-09-13T17:58:23.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.042+0000 7fc593fff640 1 -- 172.21.15.111:0/400935062 learned_addr learned my addr 172.21.15.111:0/400935062 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:23.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.042+0000 7fc5937fe640 1 --2- 172.21.15.111:0/400935062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc57c004090 0x7fc5941469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.042+0000 7fc592ffd640 1 --2- 172.21.15.111:0/400935062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc594150930 0x7fc5940b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.042+0000 7fc593fff640 1 -- 172.21.15.111:0/400935062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc594150930 msgr2=0x7fc5940b7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.042+0000 7fc593fff640 1 --2- 172.21.15.111:0/400935062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc594150930 0x7fc5940b7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.042+0000 7fc593fff640 1 -- 172.21.15.111:0/400935062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc57c004090 msgr2=0x7fc5941469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.042+0000 7fc593fff640 1 --2- 172.21.15.111:0/400935062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc57c004090 0x7fc5941469b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.042+0000 7fc593fff640 1 -- 172.21.15.111:0/400935062 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59415cb00 con 0x7fc5940b7770 2024-09-13T17:58:23.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.043+0000 7fc592ffd640 1 --2- 172.21.15.111:0/400935062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc594150930 0x7fc5940b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:23.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.043+0000 7fc593fff640 1 --2- 172.21.15.111:0/400935062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5940b7770 0x7fc5940b5260 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc58c000c00 tx=0x7fc58c00e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:23.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.043+0000 7fc57bfff640 1 -- 172.21.15.111:0/400935062 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c010510 con 0x7fc5940b7770 2024-09-13T17:58:23.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.043+0000 7fc57bfff640 1 -- 172.21.15.111:0/400935062 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58c010670 con 0x7fc5940b7770 2024-09-13T17:58:23.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.043+0000 7fc599e3a640 1 -- 172.21.15.111:0/400935062 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc59415cde0 con 0x7fc5940b7770 2024-09-13T17:58:23.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.043+0000 7fc57bfff640 1 -- 172.21.15.111:0/400935062 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c010a20 con 0x7fc5940b7770 2024-09-13T17:58:23.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.043+0000 7fc599e3a640 1 -- 172.21.15.111:0/400935062 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5940b57a0 con 0x7fc5940b7770 2024-09-13T17:58:23.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.044+0000 7fc599e3a640 1 -- 172.21.15.111:0/400935062 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5940cd8e0 con 0x7fc5940b7770 2024-09-13T17:58:23.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.044+0000 7fc57bfff640 1 -- 172.21.15.111:0/400935062 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc58c010b80 con 0x7fc5940b7770 2024-09-13T17:58:23.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.045+0000 7fc57bfff640 1 --2- 172.21.15.111:0/400935062 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc58006ac30 0x7fc58006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.045+0000 7fc57bfff640 1 -- 172.21.15.111:0/400935062 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc58c088e00 con 0x7fc5940b7770 2024-09-13T17:58:23.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.045+0000 7fc5937fe640 1 --2- 172.21.15.111:0/400935062 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc58006ac30 0x7fc58006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.045+0000 7fc5937fe640 1 --2- 172.21.15.111:0/400935062 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc58006ac30 0x7fc58006d0f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc584014430 tx=0x7fc58400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:23.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.049+0000 7fc57bfff640 1 -- 172.21.15.111:0/400935062 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc58c053cd0 con 0x7fc5940b7770 2024-09-13T17:58:23.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.207+0000 7fc599e3a640 1 -- 172.21.15.111:0/400935062 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc59415c8e0 con 0x7fc5940b7770 2024-09-13T17:58:23.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.208+0000 7fc57bfff640 1 -- 172.21.15.111:0/400935062 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc58c019070 con 0x7fc5940b7770 2024-09-13T17:58:23.210 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:23.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.209+0000 7fc599e3a640 1 -- 172.21.15.111:0/400935062 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc58006ac30 msgr2=0x7fc58006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.209+0000 7fc599e3a640 1 --2- 172.21.15.111:0/400935062 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc58006ac30 0x7fc58006d0f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc584014430 tx=0x7fc58400b040 comp rx=0 tx=0).stop 2024-09-13T17:58:23.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.209+0000 7fc599e3a640 1 -- 172.21.15.111:0/400935062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5940b7770 msgr2=0x7fc5940b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.209+0000 7fc599e3a640 1 --2- 172.21.15.111:0/400935062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5940b7770 0x7fc5940b5260 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc58c000c00 tx=0x7fc58c00e910 comp rx=0 tx=0).stop 2024-09-13T17:58:23.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.209+0000 7fc599e3a640 1 -- 172.21.15.111:0/400935062 shutdown_connections 2024-09-13T17:58:23.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.209+0000 7fc599e3a640 1 --2- 172.21.15.111:0/400935062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5940b7770 0x7fc5940b5260 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.209+0000 7fc599e3a640 1 --2- 172.21.15.111:0/400935062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc594150930 0x7fc5940b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.209+0000 7fc599e3a640 1 --2- 172.21.15.111:0/400935062 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc58006ac30 0x7fc58006d0f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.209+0000 7fc599e3a640 1 --2- 172.21.15.111:0/400935062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc57c004090 0x7fc5941469b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.209+0000 7fc599e3a640 1 -- 172.21.15.111:0/400935062 >> 172.21.15.111:0/400935062 conn(0x7fc594089f60 msgr2=0x7fc59406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:23.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.210+0000 7fc599e3a640 1 -- 172.21.15.111:0/400935062 shutdown_connections 2024-09-13T17:58:23.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.210+0000 7fc599e3a640 1 -- 172.21.15.111:0/400935062 wait complete. 2024-09-13T17:58:23.222 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:58:23.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.319+0000 7f6cb71f4640 1 Processor -- start 2024-09-13T17:58:23.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.319+0000 7f6cb71f4640 1 -- start start 2024-09-13T17:58:23.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.320+0000 7f6cb71f4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6cb015c6e0 0x7f6cb00b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.320+0000 7f6cb71f4640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb00d1840 con 0x7f6cb00ccb20 2024-09-13T17:58:23.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.320+0000 7f6cb71f4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb006bd20 con 0x7f6cb00d07b0 2024-09-13T17:58:23.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.320+0000 7f6cb71f4640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb0160e50 con 0x7f6cb015c6e0 2024-09-13T17:58:23.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.320+0000 7f6cb4f90640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6cb015c6e0 0x7f6cb00b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.320+0000 7f6cb5791640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6cb00ccb20 0x7f6cb0160740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:55910/0 (socket says 172.21.15.111:55910) 2024-09-13T17:58:23.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.320+0000 7f6cb5791640 1 -- 172.21.15.111:0/3627186996 learned_addr learned my addr 172.21.15.111:0/3627186996 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:23.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.321+0000 7f6cb4f90640 1 -- 172.21.15.111:0/3627186996 >> v1:172.21.15.188:6789/0 conn(0x7f6cb00d07b0 legacy=0x7f6cb0160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:23.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.321+0000 7f6cb4f90640 1 -- 172.21.15.111:0/3627186996 >> v1:172.21.15.111:6789/0 conn(0x7f6cb00ccb20 legacy=0x7f6cb0160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.321+0000 7f6cb4f90640 1 -- 172.21.15.111:0/3627186996 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb00d0c30 con 0x7f6cb015c6e0 2024-09-13T17:58:23.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.321+0000 7f6ca77fe640 1 -- 172.21.15.111:0/3627186996 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844879500 0 0) 0x7f6cb00d1840 con 0x7f6cb00ccb20 2024-09-13T17:58:23.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.321+0000 7f6cb4f90640 1 --2- 172.21.15.111:0/3627186996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cb015c6e0 0x7f6cb00b3f50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0097a0 tx=0x7f6cac02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae678176612dcfd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:23.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.322+0000 7f6ca77fe640 1 -- 172.21.15.111:0/3627186996 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac02ae50 con 0x7f6cb015c6e0 2024-09-13T17:58:23.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.322+0000 7f6ca77fe640 1 -- 172.21.15.111:0/3627186996 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac032440 con 0x7f6cb015c6e0 2024-09-13T17:58:23.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.322+0000 7f6ca77fe640 1 -- 172.21.15.111:0/3627186996 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac0327d0 con 0x7f6cb015c6e0 2024-09-13T17:58:23.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.322+0000 7f6cb71f4640 1 -- 172.21.15.111:0/3627186996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cb015c6e0 msgr2=0x7f6cb00b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.322+0000 7f6cb71f4640 1 --2- 172.21.15.111:0/3627186996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cb015c6e0 0x7f6cb00b3f50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0097a0 tx=0x7f6cac02a140 comp rx=0 tx=0).stop 2024-09-13T17:58:23.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.322+0000 7f6cb71f4640 1 -- 172.21.15.111:0/3627186996 shutdown_connections 2024-09-13T17:58:23.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.322+0000 7f6cb71f4640 1 --2- 172.21.15.111:0/3627186996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cb015c6e0 0x7f6cb00b3f50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.322+0000 7f6cb71f4640 1 -- 172.21.15.111:0/3627186996 >> 172.21.15.111:0/3627186996 conn(0x7f6cb0089f60 msgr2=0x7f6cb00d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:23.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.322+0000 7f6cb71f4640 1 -- 172.21.15.111:0/3627186996 shutdown_connections 2024-09-13T17:58:23.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.322+0000 7f6cb71f4640 1 -- 172.21.15.111:0/3627186996 wait complete. 2024-09-13T17:58:23.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.323+0000 7f6cb71f4640 1 Processor -- start 2024-09-13T17:58:23.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.323+0000 7f6cb71f4640 1 -- start start 2024-09-13T17:58:23.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.324+0000 7f6cb71f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cb00ccb20 0x7f6cb0154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.324+0000 7f6cb71f4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cb00d07b0 0x7f6cb015eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.324+0000 7f6cb71f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cb015c6e0 0x7f6cb00b6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.324+0000 7f6cb71f4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb0155060 con 0x7f6cb00d07b0 2024-09-13T17:58:23.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.324+0000 7f6cb71f4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb00b5670 con 0x7f6cb00ccb20 2024-09-13T17:58:23.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.324+0000 7f6cb71f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb00b5950 con 0x7f6cb015c6e0 2024-09-13T17:58:23.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.324+0000 7f6ca7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cb00d07b0 0x7f6cb015eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.324+0000 7f6ca7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cb00d07b0 0x7f6cb015eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33836/0 (socket says 172.21.15.111:33836) 2024-09-13T17:58:23.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.324+0000 7f6ca7fff640 1 -- 172.21.15.111:0/2239950098 learned_addr learned my addr 172.21.15.111:0/2239950098 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:23.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.324+0000 7f6cb5791640 1 --2- 172.21.15.111:0/2239950098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cb015c6e0 0x7f6cb00b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.324+0000 7f6cb4f90640 1 --2- 172.21.15.111:0/2239950098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cb00ccb20 0x7f6cb0154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.325+0000 7f6ca7fff640 1 -- 172.21.15.111:0/2239950098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cb015c6e0 msgr2=0x7f6cb00b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.325+0000 7f6ca7fff640 1 --2- 172.21.15.111:0/2239950098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cb015c6e0 0x7f6cb00b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.325+0000 7f6ca7fff640 1 -- 172.21.15.111:0/2239950098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cb00ccb20 msgr2=0x7f6cb0154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.325+0000 7f6ca7fff640 1 --2- 172.21.15.111:0/2239950098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cb00ccb20 0x7f6cb0154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.325+0000 7f6ca7fff640 1 -- 172.21.15.111:0/2239950098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac009400 con 0x7f6cb00d07b0 2024-09-13T17:58:23.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.325+0000 7f6ca7fff640 1 --2- 172.21.15.111:0/2239950098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cb00d07b0 0x7f6cb015eb40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000c430 tx=0x7f6ca000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:23.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.325+0000 7f6ca57fa640 1 -- 172.21.15.111:0/2239950098 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0011790 con 0x7f6cb00d07b0 2024-09-13T17:58:23.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.325+0000 7f6ca57fa640 1 -- 172.21.15.111:0/2239950098 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca00118f0 con 0x7f6cb00d07b0 2024-09-13T17:58:23.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.325+0000 7f6cb71f4640 1 -- 172.21.15.111:0/2239950098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb00b5c30 con 0x7f6cb00d07b0 2024-09-13T17:58:23.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.325+0000 7f6ca57fa640 1 -- 172.21.15.111:0/2239950098 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0011ce0 con 0x7f6cb00d07b0 2024-09-13T17:58:23.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.325+0000 7f6cb71f4640 1 -- 172.21.15.111:0/2239950098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb00b61f0 con 0x7f6cb00d07b0 2024-09-13T17:58:23.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.326+0000 7f6cb71f4640 1 -- 172.21.15.111:0/2239950098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb00d1cb0 con 0x7f6cb00d07b0 2024-09-13T17:58:23.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.326+0000 7f6ca57fa640 1 -- 172.21.15.111:0/2239950098 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6ca001c460 con 0x7f6cb00d07b0 2024-09-13T17:58:23.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.327+0000 7f6ca57fa640 1 --2- 172.21.15.111:0/2239950098 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6c9c06ac00 0x7f6c9c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.327+0000 7f6cb4f90640 1 --2- 172.21.15.111:0/2239950098 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6c9c06ac00 0x7f6c9c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.327+0000 7f6ca57fa640 1 -- 172.21.15.111:0/2239950098 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6ca0089540 con 0x7f6cb00d07b0 2024-09-13T17:58:23.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.327+0000 7f6cb4f90640 1 --2- 172.21.15.111:0/2239950098 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6c9c06ac00 0x7f6c9c06d0c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6cac034000 tx=0x7f6cac02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:23.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.331+0000 7f6ca57fa640 1 -- 172.21.15.111:0/2239950098 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f6cb00d1cb0 con 0x7f6cb00d07b0 2024-09-13T17:58:23.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.503+0000 7f6cb71f4640 1 -- 172.21.15.111:0/2239950098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6cb00d1cb0 con 0x7f6cb00d07b0 2024-09-13T17:58:23.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.503+0000 7f6ca57fa640 1 -- 172.21.15.111:0/2239950098 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6cb00d1cb0 con 0x7f6cb00d07b0 2024-09-13T17:58:23.505 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2024-09-13T17:58:23.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.505+0000 7f6cb71f4640 1 -- 172.21.15.111:0/2239950098 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6c9c06ac00 msgr2=0x7f6c9c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.505+0000 7f6cb71f4640 1 --2- 172.21.15.111:0/2239950098 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6c9c06ac00 0x7f6c9c06d0c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6cac034000 tx=0x7f6cac02e000 comp rx=0 tx=0).stop 2024-09-13T17:58:23.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.505+0000 7f6cb71f4640 1 -- 172.21.15.111:0/2239950098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cb00d07b0 msgr2=0x7f6cb015eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.505+0000 7f6cb71f4640 1 --2- 172.21.15.111:0/2239950098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cb00d07b0 0x7f6cb015eb40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000c430 tx=0x7f6ca000c8f0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.505+0000 7f6cb71f4640 1 -- 172.21.15.111:0/2239950098 shutdown_connections 2024-09-13T17:58:23.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.505+0000 7f6cb71f4640 1 --2- 172.21.15.111:0/2239950098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cb015c6e0 0x7f6cb00b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.505+0000 7f6cb71f4640 1 --2- 172.21.15.111:0/2239950098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cb00d07b0 0x7f6cb015eb40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.505+0000 7f6cb71f4640 1 --2- 172.21.15.111:0/2239950098 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6c9c06ac00 0x7f6c9c06d0c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.505+0000 7f6cb71f4640 1 --2- 172.21.15.111:0/2239950098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cb00ccb20 0x7f6cb0154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.505+0000 7f6cb71f4640 1 -- 172.21.15.111:0/2239950098 >> 172.21.15.111:0/2239950098 conn(0x7f6cb0089f60 msgr2=0x7f6cb00b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:23.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.505+0000 7f6cb71f4640 1 -- 172.21.15.111:0/2239950098 shutdown_connections 2024-09-13T17:58:23.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.505+0000 7f6cb71f4640 1 -- 172.21.15.111:0/2239950098 wait complete. 2024-09-13T17:58:23.518 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:58:23.518 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:58:23.534 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:23.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.669+0000 7fde547c7640 1 Processor -- start 2024-09-13T17:58:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.669+0000 7fde547c7640 1 -- start start 2024-09-13T17:58:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.670+0000 7fde547c7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fde4c071d30 0x7fde4c070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.670+0000 7fde547c7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c072110 0x7fde4c0b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.670+0000 7fde547c7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fde4c0c8920 0x7fde4c0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.670+0000 7fde547c7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4c0c9310 con 0x7fde4c071d30 2024-09-13T17:58:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.670+0000 7fde547c7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4c0abdf0 con 0x7fde4c0c8920 2024-09-13T17:58:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.670+0000 7fde547c7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c0ab1e0 con 0x7fde4c072110 2024-09-13T17:58:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.670+0000 7fde52563640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fde4c071d30 0x7fde4c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.670+0000 7fde52563640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fde4c071d30 0x7fde4c070190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33838/0 (socket says 172.21.15.111:33838) 2024-09-13T17:58:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.670+0000 7fde52563640 1 -- 172.21.15.111:0/1981802484 learned_addr learned my addr 172.21.15.111:0/1981802484 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.670+0000 7fde52563640 1 -- 172.21.15.111:0/1981802484 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c072110 msgr2=0x7fde4c0b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.670+0000 7fde52563640 1 --2- 172.21.15.111:0/1981802484 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c072110 0x7fde4c0b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.670+0000 7fde52563640 1 -- 172.21.15.111:0/1981802484 >> v2:172.21.15.188:3300/0 conn(0x7fde4c0c8920 msgr2=0x7fde4c0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.670+0000 7fde52d64640 1 --2- 172.21.15.111:0/1981802484 >> v2:172.21.15.188:3300/0 conn(0x7fde4c0c8920 0x7fde4c0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.671+0000 7fde52563640 1 --2- 172.21.15.111:0/1981802484 >> v2:172.21.15.188:3300/0 conn(0x7fde4c0c8920 0x7fde4c0c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.671+0000 7fde52563640 1 -- 172.21.15.111:0/1981802484 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c0ab4c0 con 0x7fde4c071d30 2024-09-13T17:58:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.671+0000 7fde52d64640 1 --2- 172.21.15.111:0/1981802484 >> v2:172.21.15.188:3300/0 conn(0x7fde4c0c8920 0x7fde4c0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:23.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.671+0000 7fde52563640 1 --2- 172.21.15.111:0/1981802484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde4c071d30 0x7fde4c070190 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fde48009870 tx=0x7fde4802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6521b3737c52e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:23.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.671+0000 7fde51561640 1 -- 172.21.15.111:0/1981802484 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48032440 con 0x7fde4c071d30 2024-09-13T17:58:23.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.671+0000 7fde51561640 1 -- 172.21.15.111:0/1981802484 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde480325a0 con 0x7fde4c071d30 2024-09-13T17:58:23.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.671+0000 7fde51561640 1 -- 172.21.15.111:0/1981802484 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48032930 con 0x7fde4c071d30 2024-09-13T17:58:23.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.672+0000 7fde547c7640 1 -- 172.21.15.111:0/1981802484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde4c071d30 msgr2=0x7fde4c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.672+0000 7fde547c7640 1 --2- 172.21.15.111:0/1981802484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde4c071d30 0x7fde4c070190 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fde48009870 tx=0x7fde4802a230 comp rx=0 tx=0).stop 2024-09-13T17:58:23.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.672+0000 7fde547c7640 1 -- 172.21.15.111:0/1981802484 shutdown_connections 2024-09-13T17:58:23.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.672+0000 7fde547c7640 1 --2- 172.21.15.111:0/1981802484 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c072110 0x7fde4c0b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.672+0000 7fde547c7640 1 --2- 172.21.15.111:0/1981802484 >> v2:172.21.15.188:3300/0 conn(0x7fde4c0c8920 0x7fde4c0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.672+0000 7fde547c7640 1 --2- 172.21.15.111:0/1981802484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde4c071d30 0x7fde4c070190 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.672+0000 7fde547c7640 1 -- 172.21.15.111:0/1981802484 >> 172.21.15.111:0/1981802484 conn(0x7fde4c069ed0 msgr2=0x7fde4c06a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:23.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.672+0000 7fde547c7640 1 -- 172.21.15.111:0/1981802484 shutdown_connections 2024-09-13T17:58:23.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.672+0000 7fde547c7640 1 -- 172.21.15.111:0/1981802484 wait complete. 2024-09-13T17:58:23.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.673+0000 7fde547c7640 1 Processor -- start 2024-09-13T17:58:23.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.673+0000 7fde547c7640 1 -- start start 2024-09-13T17:58:23.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde547c7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c071d30 0x7fde4c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde547c7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde4c072110 0x7fde4c0c0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde547c7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde4c0c8920 0x7fde4c0c1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde547c7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c1537d0 con 0x7fde4c0c8920 2024-09-13T17:58:23.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde547c7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c0c16a0 con 0x7fde4c072110 2024-09-13T17:58:23.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde547c7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c1532e0 con 0x7fde4c071d30 2024-09-13T17:58:23.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde52d64640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde4c0c8920 0x7fde4c0c1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde52d64640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde4c0c8920 0x7fde4c0c1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33854/0 (socket says 172.21.15.111:33854) 2024-09-13T17:58:23.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde52d64640 1 -- 172.21.15.111:0/82706833 learned_addr learned my addr 172.21.15.111:0/82706833 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:23.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde52563640 1 --2- 172.21.15.111:0/82706833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c071d30 0x7fde4c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde51d62640 1 --2- 172.21.15.111:0/82706833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde4c072110 0x7fde4c0c0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde52d64640 1 -- 172.21.15.111:0/82706833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c071d30 msgr2=0x7fde4c06d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde52d64640 1 --2- 172.21.15.111:0/82706833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c071d30 0x7fde4c06d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde52d64640 1 -- 172.21.15.111:0/82706833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde4c072110 msgr2=0x7fde4c0c0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde52d64640 1 --2- 172.21.15.111:0/82706833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde4c072110 0x7fde4c0c0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde52d64640 1 -- 172.21.15.111:0/82706833 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde480094d0 con 0x7fde4c0c8920 2024-09-13T17:58:23.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde51d62640 1 --2- 172.21.15.111:0/82706833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde4c072110 0x7fde4c0c0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:23.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde52d64640 1 --2- 172.21.15.111:0/82706833 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde4c0c8920 0x7fde4c0c1160 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fde4400cd80 tx=0x7fde4400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:23.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde42ffd640 1 -- 172.21.15.111:0/82706833 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4400f680 con 0x7fde4c0c8920 2024-09-13T17:58:23.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.674+0000 7fde42ffd640 1 -- 172.21.15.111:0/82706833 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4400f800 con 0x7fde4c0c8920 2024-09-13T17:58:23.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.675+0000 7fde547c7640 1 -- 172.21.15.111:0/82706833 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4c0afc10 con 0x7fde4c0c8920 2024-09-13T17:58:23.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.675+0000 7fde42ffd640 1 -- 172.21.15.111:0/82706833 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4400fbb0 con 0x7fde4c0c8920 2024-09-13T17:58:23.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.675+0000 7fde547c7640 1 -- 172.21.15.111:0/82706833 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c06e0d0 con 0x7fde4c0c8920 2024-09-13T17:58:23.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.676+0000 7fde547c7640 1 -- 172.21.15.111:0/82706833 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4c0b2f00 con 0x7fde4c0c8920 2024-09-13T17:58:23.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.676+0000 7fde42ffd640 1 -- 172.21.15.111:0/82706833 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fde4400b040 con 0x7fde4c0c8920 2024-09-13T17:58:23.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.676+0000 7fde42ffd640 1 --2- 172.21.15.111:0/82706833 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fde3006ac00 0x7fde3006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.676+0000 7fde42ffd640 1 -- 172.21.15.111:0/82706833 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fde4408a0d0 con 0x7fde4c0c8920 2024-09-13T17:58:23.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.676+0000 7fde52563640 1 --2- 172.21.15.111:0/82706833 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fde3006ac00 0x7fde3006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.677+0000 7fde52563640 1 --2- 172.21.15.111:0/82706833 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fde3006ac00 0x7fde3006d0c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fde48034000 tx=0x7fde4802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:23.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.680+0000 7fde42ffd640 1 -- 172.21.15.111:0/82706833 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fde44054ef0 con 0x7fde4c0c8920 2024-09-13T17:58:23.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.845+0000 7fde547c7640 1 -- 172.21.15.111:0/82706833 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde4c0c6ae0 con 0x7fde4c0c8920 2024-09-13T17:58:23.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.846+0000 7fde42ffd640 1 -- 172.21.15.111:0/82706833 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18794 (secure 0 0 0) 0x7fde44058510 con 0x7fde4c0c8920 2024-09-13T17:58:23.848 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:23.848 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":26,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:07.749113+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:23.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.848+0000 7fde547c7640 1 -- 172.21.15.111:0/82706833 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fde3006ac00 msgr2=0x7fde3006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.848+0000 7fde547c7640 1 --2- 172.21.15.111:0/82706833 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fde3006ac00 0x7fde3006d0c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fde48034000 tx=0x7fde4802e000 comp rx=0 tx=0).stop 2024-09-13T17:58:23.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.848+0000 7fde547c7640 1 -- 172.21.15.111:0/82706833 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde4c0c8920 msgr2=0x7fde4c0c1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.848+0000 7fde547c7640 1 --2- 172.21.15.111:0/82706833 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde4c0c8920 0x7fde4c0c1160 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fde4400cd80 tx=0x7fde4400a650 comp rx=0 tx=0).stop 2024-09-13T17:58:23.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.848+0000 7fde547c7640 1 -- 172.21.15.111:0/82706833 shutdown_connections 2024-09-13T17:58:23.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.848+0000 7fde547c7640 1 --2- 172.21.15.111:0/82706833 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde4c0c8920 0x7fde4c0c1160 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.848+0000 7fde547c7640 1 --2- 172.21.15.111:0/82706833 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fde3006ac00 0x7fde3006d0c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.848+0000 7fde547c7640 1 --2- 172.21.15.111:0/82706833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde4c072110 0x7fde4c0c0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.848+0000 7fde547c7640 1 --2- 172.21.15.111:0/82706833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c071d30 0x7fde4c06d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.848+0000 7fde547c7640 1 -- 172.21.15.111:0/82706833 >> 172.21.15.111:0/82706833 conn(0x7fde4c069ed0 msgr2=0x7fde4c0ae230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:23.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.848+0000 7fde547c7640 1 -- 172.21.15.111:0/82706833 shutdown_connections 2024-09-13T17:58:23.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.848+0000 7fde547c7640 1 -- 172.21.15.111:0/82706833 wait complete. 2024-09-13T17:58:23.861 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:58:23.861 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:58:23.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.960+0000 7f394682b640 1 Processor -- start 2024-09-13T17:58:23.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.960+0000 7f394682b640 1 -- start start 2024-09-13T17:58:23.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.960+0000 7f394682b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f394015c500 0x7f39400d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.960+0000 7f394682b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39400d2590 0x7f39400cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.960+0000 7f394682b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f39400cde20 0x7f39400d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.960+0000 7f394682b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39400cd450 con 0x7f394015c500 2024-09-13T17:58:23.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.960+0000 7f394682b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39400d29f0 con 0x7f39400cde20 2024-09-13T17:58:23.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.960+0000 7f394682b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39400d0cf0 con 0x7f39400d2590 2024-09-13T17:58:23.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.961+0000 7f393ffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f394015c500 0x7f39400d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.961+0000 7f393ffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f394015c500 0x7f39400d2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33860/0 (socket says 172.21.15.111:33860) 2024-09-13T17:58:23.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.961+0000 7f393ffff640 1 -- 172.21.15.111:0/4132137682 learned_addr learned my addr 172.21.15.111:0/4132137682 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:23.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.961+0000 7f3937fff640 1 --2- 172.21.15.111:0/4132137682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39400d2590 0x7f39400cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.961+0000 7f3944dc8640 1 --2- 172.21.15.111:0/4132137682 >> v2:172.21.15.188:3300/0 conn(0x7f39400cde20 0x7f39400d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.961+0000 7f393ffff640 1 -- 172.21.15.111:0/4132137682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39400d2590 msgr2=0x7f39400cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:23.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.961+0000 7f393ffff640 1 --2- 172.21.15.111:0/4132137682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39400d2590 0x7f39400cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.961+0000 7f393ffff640 1 -- 172.21.15.111:0/4132137682 >> v2:172.21.15.188:3300/0 conn(0x7f39400cde20 msgr2=0x7f39400d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:23.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.961+0000 7f393ffff640 1 --2- 172.21.15.111:0/4132137682 >> v2:172.21.15.188:3300/0 conn(0x7f39400cde20 0x7f39400d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.961+0000 7f393ffff640 1 -- 172.21.15.111:0/4132137682 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39400d0fd0 con 0x7f394015c500 2024-09-13T17:58:23.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.962+0000 7f393ffff640 1 --2- 172.21.15.111:0/4132137682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f394015c500 0x7f39400d2050 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f39400b1e40 tx=0x7f393002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37812d2d25a2f46 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:23.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.962+0000 7f393f7fe640 1 -- 172.21.15.111:0/4132137682 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930032440 con 0x7f394015c500 2024-09-13T17:58:23.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.962+0000 7f393f7fe640 1 -- 172.21.15.111:0/4132137682 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39300325a0 con 0x7f394015c500 2024-09-13T17:58:23.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.962+0000 7f393f7fe640 1 -- 172.21.15.111:0/4132137682 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930032930 con 0x7f394015c500 2024-09-13T17:58:23.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.962+0000 7f394682b640 1 -- 172.21.15.111:0/4132137682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f394015c500 msgr2=0x7f39400d2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.962+0000 7f394682b640 1 --2- 172.21.15.111:0/4132137682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f394015c500 0x7f39400d2050 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f39400b1e40 tx=0x7f393002a1e0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.962+0000 7f394682b640 1 -- 172.21.15.111:0/4132137682 shutdown_connections 2024-09-13T17:58:23.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.962+0000 7f394682b640 1 --2- 172.21.15.111:0/4132137682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39400d2590 0x7f39400cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.962+0000 7f394682b640 1 --2- 172.21.15.111:0/4132137682 >> v2:172.21.15.188:3300/0 conn(0x7f39400cde20 0x7f39400d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.962+0000 7f394682b640 1 --2- 172.21.15.111:0/4132137682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f394015c500 0x7f39400d2050 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.962+0000 7f394682b640 1 -- 172.21.15.111:0/4132137682 >> 172.21.15.111:0/4132137682 conn(0x7f3940089f60 msgr2=0x7f394009b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:23.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.962+0000 7f394682b640 1 -- 172.21.15.111:0/4132137682 shutdown_connections 2024-09-13T17:58:23.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.962+0000 7f394682b640 1 -- 172.21.15.111:0/4132137682 wait complete. 2024-09-13T17:58:23.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.964+0000 7f394682b640 1 Processor -- start 2024-09-13T17:58:23.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.964+0000 7f394682b640 1 -- start start 2024-09-13T17:58:23.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.964+0000 7f394682b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39400cde20 0x7f39400b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.964+0000 7f394682b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39400d2590 0x7f39401f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.964+0000 7f394682b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f394015c500 0x7f394015a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.964+0000 7f394682b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39400b59a0 con 0x7f39400cde20 2024-09-13T17:58:23.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.964+0000 7f394682b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39401f2440 con 0x7f39400d2590 2024-09-13T17:58:23.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.964+0000 7f394682b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39400b53e0 con 0x7f394015c500 2024-09-13T17:58:23.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.964+0000 7f393ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39400cde20 0x7f39400b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.964+0000 7f393ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39400cde20 0x7f39400b7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33870/0 (socket says 172.21.15.111:33870) 2024-09-13T17:58:23.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.964+0000 7f393ffff640 1 -- 172.21.15.111:0/1013961465 learned_addr learned my addr 172.21.15.111:0/1013961465 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:23.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.965+0000 7f3944dc8640 1 --2- 172.21.15.111:0/1013961465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f394015c500 0x7f394015a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.965+0000 7f3937fff640 1 --2- 172.21.15.111:0/1013961465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39400d2590 0x7f39401f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.965+0000 7f393ffff640 1 -- 172.21.15.111:0/1013961465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f394015c500 msgr2=0x7f394015a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.965+0000 7f393ffff640 1 --2- 172.21.15.111:0/1013961465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f394015c500 0x7f394015a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.965+0000 7f393ffff640 1 -- 172.21.15.111:0/1013961465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39400d2590 msgr2=0x7f39401f1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:23.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.965+0000 7f393ffff640 1 --2- 172.21.15.111:0/1013961465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39400d2590 0x7f39401f1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:23.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.965+0000 7f393ffff640 1 -- 172.21.15.111:0/1013961465 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39300094d0 con 0x7f39400cde20 2024-09-13T17:58:23.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.965+0000 7f393ffff640 1 --2- 172.21.15.111:0/1013961465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39400cde20 0x7f39400b7d80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f393002a220 tx=0x7f39300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:23.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.965+0000 7f393d7fa640 1 -- 172.21.15.111:0/1013961465 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930032440 con 0x7f39400cde20 2024-09-13T17:58:23.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.965+0000 7f393d7fa640 1 -- 172.21.15.111:0/1013961465 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3930030b80 con 0x7f39400cde20 2024-09-13T17:58:23.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.965+0000 7f394682b640 1 -- 172.21.15.111:0/1013961465 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39400b56c0 con 0x7f39400cde20 2024-09-13T17:58:23.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.965+0000 7f393d7fa640 1 -- 172.21.15.111:0/1013961465 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393003d440 con 0x7f39400cde20 2024-09-13T17:58:23.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.965+0000 7f394682b640 1 -- 172.21.15.111:0/1013961465 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394015a550 con 0x7f39400cde20 2024-09-13T17:58:23.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.966+0000 7f394682b640 1 -- 172.21.15.111:0/1013961465 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39400cabd0 con 0x7f39400cde20 2024-09-13T17:58:23.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.966+0000 7f393d7fa640 1 -- 172.21.15.111:0/1013961465 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3930039040 con 0x7f39400cde20 2024-09-13T17:58:23.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.967+0000 7f393d7fa640 1 --2- 172.21.15.111:0/1013961465 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f392006ac00 0x7f392006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:23.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.967+0000 7f393d7fa640 1 -- 172.21.15.111:0/1013961465 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f39300aa2a0 con 0x7f39400cde20 2024-09-13T17:58:23.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.967+0000 7f3937fff640 1 --2- 172.21.15.111:0/1013961465 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f392006ac00 0x7f392006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:23.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.967+0000 7f3937fff640 1 --2- 172.21.15.111:0/1013961465 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f392006ac00 0x7f392006d0c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f394015f910 tx=0x7f3928008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:23.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:23.971+0000 7f393d7fa640 1 -- 172.21.15.111:0/1013961465 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f39300750c0 con 0x7f39400cde20 2024-09-13T17:58:24.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.130+0000 7f394682b640 1 -- 172.21.15.111:0/1013961465 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39400cd8e0 con 0x7f39400cde20 2024-09-13T17:58:24.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.130+0000 7f393d7fa640 1 -- 172.21.15.111:0/1013961465 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f39300786e0 con 0x7f39400cde20 2024-09-13T17:58:24.132 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:58:24.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.132+0000 7f394682b640 1 -- 172.21.15.111:0/1013961465 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f392006ac00 msgr2=0x7f392006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.132+0000 7f394682b640 1 --2- 172.21.15.111:0/1013961465 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f392006ac00 0x7f392006d0c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f394015f910 tx=0x7f3928008040 comp rx=0 tx=0).stop 2024-09-13T17:58:24.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.132+0000 7f394682b640 1 -- 172.21.15.111:0/1013961465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39400cde20 msgr2=0x7f39400b7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.132+0000 7f394682b640 1 --2- 172.21.15.111:0/1013961465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39400cde20 0x7f39400b7d80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f393002a220 tx=0x7f39300046c0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.132+0000 7f394682b640 1 -- 172.21.15.111:0/1013961465 shutdown_connections 2024-09-13T17:58:24.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.132+0000 7f394682b640 1 --2- 172.21.15.111:0/1013961465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f394015c500 0x7f394015a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.132+0000 7f394682b640 1 --2- 172.21.15.111:0/1013961465 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f392006ac00 0x7f392006d0c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.132+0000 7f394682b640 1 --2- 172.21.15.111:0/1013961465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39400d2590 0x7f39401f1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.132+0000 7f394682b640 1 --2- 172.21.15.111:0/1013961465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39400cde20 0x7f39400b7d80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.132+0000 7f394682b640 1 -- 172.21.15.111:0/1013961465 >> 172.21.15.111:0/1013961465 conn(0x7f3940089f60 msgr2=0x7f394015ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:24.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.132+0000 7f394682b640 1 -- 172.21.15.111:0/1013961465 shutdown_connections 2024-09-13T17:58:24.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.132+0000 7f394682b640 1 -- 172.21.15.111:0/1013961465 wait complete. 2024-09-13T17:58:24.145 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:58:24.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.244+0000 7f753b508640 1 Processor -- start 2024-09-13T17:58:24.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.244+0000 7f753b508640 1 -- start start 2024-09-13T17:58:24.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.245+0000 7f753b508640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75340b20a0 0x7f75340ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:24.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.245+0000 7f753b508640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f75340cea30 0x7f75340ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:24.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.245+0000 7f753b508640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f753415a790 con 0x7f753415a3b0 2024-09-13T17:58:24.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.245+0000 7f753b508640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75340ceec0 con 0x7f75340cea30 2024-09-13T17:58:24.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.245+0000 7f753b508640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75340b2680 con 0x7f75340b20a0 2024-09-13T17:58:24.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.245+0000 7f7539aa5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f75340cea30 0x7f75340ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:24.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.245+0000 7f7539aa5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f75340cea30 0x7f75340ab9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33884/0 (socket says 172.21.15.111:33884) 2024-09-13T17:58:24.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.245+0000 7f7539aa5640 1 -- 172.21.15.111:0/1818041401 learned_addr learned my addr 172.21.15.111:0/1818041401 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:24.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.245+0000 7f7538aa3640 1 --2- 172.21.15.111:0/1818041401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75340b20a0 0x7f75340ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:24.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.245+0000 7f7539aa5640 1 -- 172.21.15.111:0/1818041401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75340b20a0 msgr2=0x7f75340ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:24.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.245+0000 7f7539aa5640 1 --2- 172.21.15.111:0/1818041401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75340b20a0 0x7f75340ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.245+0000 7f7539aa5640 1 -- 172.21.15.111:0/1818041401 >> v1:172.21.15.188:6789/0 conn(0x7f753415a3b0 legacy=0x7f75340b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:24.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.245+0000 7f7539aa5640 1 -- 172.21.15.111:0/1818041401 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75340b2960 con 0x7f75340cea30 2024-09-13T17:58:24.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.245+0000 7f7538aa3640 1 --2- 172.21.15.111:0/1818041401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75340b20a0 0x7f75340ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:24.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.246+0000 7f7539aa5640 1 --2- 172.21.15.111:0/1818041401 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75340cea30 0x7f75340ab9e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f752c009e50 tx=0x7f752c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cfc87009a30591 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:24.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.246+0000 7f7527fff640 1 -- 172.21.15.111:0/1818041401 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c02aea0 con 0x7f75340cea30 2024-09-13T17:58:24.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.246+0000 7f7527fff640 1 -- 172.21.15.111:0/1818041401 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752c02d050 con 0x7f75340cea30 2024-09-13T17:58:24.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.246+0000 7f7527fff640 1 -- 172.21.15.111:0/1818041401 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c031820 con 0x7f75340cea30 2024-09-13T17:58:24.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.247+0000 7f753b508640 1 -- 172.21.15.111:0/1818041401 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75340cea30 msgr2=0x7f75340ab9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.247+0000 7f753b508640 1 --2- 172.21.15.111:0/1818041401 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75340cea30 0x7f75340ab9e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f752c009e50 tx=0x7f752c02a6c0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.247+0000 7f753b508640 1 -- 172.21.15.111:0/1818041401 shutdown_connections 2024-09-13T17:58:24.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.247+0000 7f753b508640 1 --2- 172.21.15.111:0/1818041401 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75340cea30 0x7f75340ab9e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.247+0000 7f753b508640 1 --2- 172.21.15.111:0/1818041401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75340b20a0 0x7f75340ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.247+0000 7f753b508640 1 -- 172.21.15.111:0/1818041401 >> 172.21.15.111:0/1818041401 conn(0x7f753406a0b0 msgr2=0x7f753406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:24.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.247+0000 7f753b508640 1 -- 172.21.15.111:0/1818041401 shutdown_connections 2024-09-13T17:58:24.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.247+0000 7f753b508640 1 -- 172.21.15.111:0/1818041401 wait complete. 2024-09-13T17:58:24.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.248+0000 7f753b508640 1 Processor -- start 2024-09-13T17:58:24.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.248+0000 7f753b508640 1 -- start start 2024-09-13T17:58:24.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.248+0000 7f753b508640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75340b20a0 0x7f7534153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:24.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f753b508640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75340cea30 0x7f75340adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:24.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f753b508640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f753415a3b0 0x7f75340aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:24.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f753b508640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75341f0d60 con 0x7f75340b20a0 2024-09-13T17:58:24.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f753b508640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75341f0150 con 0x7f753415a3b0 2024-09-13T17:58:24.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f753b508640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75341f0430 con 0x7f75340cea30 2024-09-13T17:58:24.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f75392a4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75340b20a0 0x7f7534153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:24.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f75392a4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75340b20a0 0x7f7534153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33892/0 (socket says 172.21.15.111:33892) 2024-09-13T17:58:24.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f75392a4640 1 -- 172.21.15.111:0/2121799944 learned_addr learned my addr 172.21.15.111:0/2121799944 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:24.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f75392a4640 1 -- 172.21.15.111:0/2121799944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75340cea30 msgr2=0x7f75340adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f75392a4640 1 --2- 172.21.15.111:0/2121799944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75340cea30 0x7f75340adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f75392a4640 1 -- 172.21.15.111:0/2121799944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f753415a3b0 msgr2=0x7f75340aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f75392a4640 1 --2- 172.21.15.111:0/2121799944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f753415a3b0 0x7f75340aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f75392a4640 1 -- 172.21.15.111:0/2121799944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c009b30 con 0x7f75340b20a0 2024-09-13T17:58:24.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f75392a4640 1 --2- 172.21.15.111:0/2121799944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75340b20a0 0x7f7534153770 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f752800b3f0 tx=0x7f752800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:24.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f7525ffb640 1 -- 172.21.15.111:0/2121799944 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7528011830 con 0x7f75340b20a0 2024-09-13T17:58:24.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f7525ffb640 1 -- 172.21.15.111:0/2121799944 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7528011990 con 0x7f75340b20a0 2024-09-13T17:58:24.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f7525ffb640 1 -- 172.21.15.111:0/2121799944 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7528011d40 con 0x7f75340b20a0 2024-09-13T17:58:24.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f753b508640 1 -- 172.21.15.111:0/2121799944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75341f0710 con 0x7f75340b20a0 2024-09-13T17:58:24.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.249+0000 7f753b508640 1 -- 172.21.15.111:0/2121799944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75341efd70 con 0x7f75340b20a0 2024-09-13T17:58:24.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.250+0000 7f753b508640 1 -- 172.21.15.111:0/2121799944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75340ce4f0 con 0x7f75340b20a0 2024-09-13T17:58:24.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.251+0000 7f7525ffb640 1 -- 172.21.15.111:0/2121799944 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f752801a400 con 0x7f75340b20a0 2024-09-13T17:58:24.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.251+0000 7f7525ffb640 1 --2- 172.21.15.111:0/2121799944 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f751806ac00 0x7f751806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:24.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.251+0000 7f7538aa3640 1 --2- 172.21.15.111:0/2121799944 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f751806ac00 0x7f751806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:24.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.251+0000 7f7525ffb640 1 -- 172.21.15.111:0/2121799944 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f75280895b0 con 0x7f75340b20a0 2024-09-13T17:58:24.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.252+0000 7f7538aa3640 1 --2- 172.21.15.111:0/2121799944 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f751806ac00 0x7f751806d0c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7530006fd0 tx=0x7f7530008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:24.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.256+0000 7f7525ffb640 1 -- 172.21.15.111:0/2121799944 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f75280543d0 con 0x7f75340b20a0 2024-09-13T17:58:24.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.421+0000 7f753b508640 1 -- 172.21.15.111:0/2121799944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7534071140 con 0x7f75340b20a0 2024-09-13T17:58:24.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.421+0000 7f7525ffb640 1 -- 172.21.15.111:0/2121799944 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f75280579f0 con 0x7f75340b20a0 2024-09-13T17:58:24.424 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:24.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.423+0000 7f753b508640 1 -- 172.21.15.111:0/2121799944 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f751806ac00 msgr2=0x7f751806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.423+0000 7f753b508640 1 --2- 172.21.15.111:0/2121799944 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f751806ac00 0x7f751806d0c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7530006fd0 tx=0x7f7530008040 comp rx=0 tx=0).stop 2024-09-13T17:58:24.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.423+0000 7f753b508640 1 -- 172.21.15.111:0/2121799944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75340b20a0 msgr2=0x7f7534153770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.423+0000 7f753b508640 1 --2- 172.21.15.111:0/2121799944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75340b20a0 0x7f7534153770 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f752800b3f0 tx=0x7f752800b8b0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.423+0000 7f753b508640 1 -- 172.21.15.111:0/2121799944 shutdown_connections 2024-09-13T17:58:24.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.423+0000 7f753b508640 1 --2- 172.21.15.111:0/2121799944 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f751806ac00 0x7f751806d0c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.423+0000 7f753b508640 1 --2- 172.21.15.111:0/2121799944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f753415a3b0 0x7f75340aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.423+0000 7f753b508640 1 --2- 172.21.15.111:0/2121799944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75340cea30 0x7f75340adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.423+0000 7f753b508640 1 --2- 172.21.15.111:0/2121799944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75340b20a0 0x7f7534153770 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.423+0000 7f753b508640 1 -- 172.21.15.111:0/2121799944 >> 172.21.15.111:0/2121799944 conn(0x7f753406a0b0 msgr2=0x7f75340caa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:24.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.423+0000 7f753b508640 1 -- 172.21.15.111:0/2121799944 shutdown_connections 2024-09-13T17:58:24.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.423+0000 7f753b508640 1 -- 172.21.15.111:0/2121799944 wait complete. 2024-09-13T17:58:24.437 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:58:24.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.535+0000 7f782179c640 1 Processor -- start 2024-09-13T17:58:24.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.535+0000 7f782179c640 1 -- start start 2024-09-13T17:58:24.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.536+0000 7f782179c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f781c15fbc0 0x7f781c0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:24.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.536+0000 7f782179c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f781c0cd6c0 0x7f781c0d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:24.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.536+0000 7f782179c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f781c06b6c0 con 0x7f781c0d2810 2024-09-13T17:58:24.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.536+0000 7f782179c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f781c0d2cc0 con 0x7f781c15fbc0 2024-09-13T17:58:24.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.536+0000 7f782179c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781c0ce500 con 0x7f781c0cd6c0 2024-09-13T17:58:24.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.536+0000 7f781affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f781c15fbc0 0x7f781c0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:24.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.536+0000 7f781affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f781c15fbc0 0x7f781c0cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33904/0 (socket says 172.21.15.111:33904) 2024-09-13T17:58:24.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.536+0000 7f781affd640 1 -- 172.21.15.111:0/3175777707 learned_addr learned my addr 172.21.15.111:0/3175777707 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:24.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.537+0000 7f781a7fc640 1 --2- 172.21.15.111:0/3175777707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f781c0cd6c0 0x7f781c0d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:24.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.537+0000 7f781affd640 1 -- 172.21.15.111:0/3175777707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f781c0cd6c0 msgr2=0x7f781c0d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:24.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.537+0000 7f781affd640 1 --2- 172.21.15.111:0/3175777707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f781c0cd6c0 0x7f781c0d22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.537+0000 7f781affd640 1 -- 172.21.15.111:0/3175777707 >> v1:172.21.15.188:6789/0 conn(0x7f781c0d2810 legacy=0x7f781c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:24.540 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.537+0000 7f781affd640 1 -- 172.21.15.111:0/3175777707 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c0cdc60 con 0x7f781c15fbc0 2024-09-13T17:58:24.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.537+0000 7f781affd640 1 --2- 172.21.15.111:0/3175777707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f781c15fbc0 0x7f781c0cd180 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f780c009e50 tx=0x7f780c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f678b35b047c8e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:24.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.537+0000 7f7819ffb640 1 -- 172.21.15.111:0/3175777707 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c02aea0 con 0x7f781c15fbc0 2024-09-13T17:58:24.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.537+0000 7f7819ffb640 1 -- 172.21.15.111:0/3175777707 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780c02d050 con 0x7f781c15fbc0 2024-09-13T17:58:24.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.537+0000 7f7819ffb640 1 -- 172.21.15.111:0/3175777707 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c031820 con 0x7f781c15fbc0 2024-09-13T17:58:24.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.538+0000 7f782179c640 1 -- 172.21.15.111:0/3175777707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f781c15fbc0 msgr2=0x7f781c0cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.538+0000 7f782179c640 1 --2- 172.21.15.111:0/3175777707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f781c15fbc0 0x7f781c0cd180 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f780c009e50 tx=0x7f780c02a6c0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.538+0000 7f782179c640 1 -- 172.21.15.111:0/3175777707 shutdown_connections 2024-09-13T17:58:24.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.538+0000 7f782179c640 1 --2- 172.21.15.111:0/3175777707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f781c0cd6c0 0x7f781c0d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.538+0000 7f782179c640 1 --2- 172.21.15.111:0/3175777707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f781c15fbc0 0x7f781c0cd180 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.538+0000 7f782179c640 1 -- 172.21.15.111:0/3175777707 >> 172.21.15.111:0/3175777707 conn(0x7f781c09b830 msgr2=0x7f781c089f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:24.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.538+0000 7f782179c640 1 -- 172.21.15.111:0/3175777707 shutdown_connections 2024-09-13T17:58:24.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.538+0000 7f782179c640 1 -- 172.21.15.111:0/3175777707 wait complete. 2024-09-13T17:58:24.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.539+0000 7f782179c640 1 Processor -- start 2024-09-13T17:58:24.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.539+0000 7f782179c640 1 -- start start 2024-09-13T17:58:24.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.540+0000 7f782179c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f781c0cd6c0 0x7f781c1498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:24.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.540+0000 7f782179c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f781c0d2810 0x7f781c0b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:24.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.540+0000 7f782179c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f781c15fbc0 0x7f781c0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:24.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.540+0000 7f782179c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781c156d80 con 0x7f781c0d2810 2024-09-13T17:58:24.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.540+0000 7f782179c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781c0b6390 con 0x7f781c0cd6c0 2024-09-13T17:58:24.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.540+0000 7f782179c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781c0b5780 con 0x7f781c15fbc0 2024-09-13T17:58:24.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.540+0000 7f781a7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f781c0d2810 0x7f781c0b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:24.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.540+0000 7f781a7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f781c0d2810 0x7f781c0b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33920/0 (socket says 172.21.15.111:33920) 2024-09-13T17:58:24.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.540+0000 7f781a7fc640 1 -- 172.21.15.111:0/2064774632 learned_addr learned my addr 172.21.15.111:0/2064774632 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:24.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.540+0000 7f781affd640 1 --2- 172.21.15.111:0/2064774632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f781c0cd6c0 0x7f781c1498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:24.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.540+0000 7f781b7fe640 1 --2- 172.21.15.111:0/2064774632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f781c15fbc0 0x7f781c0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:24.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.540+0000 7f781a7fc640 1 -- 172.21.15.111:0/2064774632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f781c15fbc0 msgr2=0x7f781c0b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.540+0000 7f781a7fc640 1 --2- 172.21.15.111:0/2064774632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f781c15fbc0 0x7f781c0b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.540+0000 7f781a7fc640 1 -- 172.21.15.111:0/2064774632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f781c0cd6c0 msgr2=0x7f781c1498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.540+0000 7f781a7fc640 1 --2- 172.21.15.111:0/2064774632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f781c0cd6c0 0x7f781c1498f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.540+0000 7f781a7fc640 1 -- 172.21.15.111:0/2064774632 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c009b30 con 0x7f781c0d2810 2024-09-13T17:58:24.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.541+0000 7f781b7fe640 1 --2- 172.21.15.111:0/2064774632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f781c15fbc0 0x7f781c0b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:24.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.541+0000 7f781a7fc640 1 --2- 172.21.15.111:0/2064774632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f781c0d2810 0x7f781c0b40a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f780400c610 tx=0x7f780400cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:24.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.541+0000 7f78137fe640 1 -- 172.21.15.111:0/2064774632 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804004230 con 0x7f781c0d2810 2024-09-13T17:58:24.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.541+0000 7f78137fe640 1 -- 172.21.15.111:0/2064774632 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7804004390 con 0x7f781c0d2810 2024-09-13T17:58:24.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.541+0000 7f78137fe640 1 -- 172.21.15.111:0/2064774632 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804003830 con 0x7f781c0d2810 2024-09-13T17:58:24.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.541+0000 7f782179c640 1 -- 172.21.15.111:0/2064774632 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781c0b5a60 con 0x7f781c0d2810 2024-09-13T17:58:24.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.541+0000 7f782179c640 1 -- 172.21.15.111:0/2064774632 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f781c0b6020 con 0x7f781c0d2810 2024-09-13T17:58:24.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.542+0000 7f782179c640 1 -- 172.21.15.111:0/2064774632 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f781c0af090 con 0x7f781c0d2810 2024-09-13T17:58:24.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.542+0000 7f78137fe640 1 -- 172.21.15.111:0/2064774632 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7804003990 con 0x7f781c0d2810 2024-09-13T17:58:24.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.543+0000 7f78137fe640 1 --2- 172.21.15.111:0/2064774632 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f77fc06ac00 0x7f77fc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:24.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.543+0000 7f78137fe640 1 -- 172.21.15.111:0/2064774632 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7804089120 con 0x7f781c0d2810 2024-09-13T17:58:24.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.543+0000 7f781affd640 1 --2- 172.21.15.111:0/2064774632 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f77fc06ac00 0x7f77fc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:24.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.543+0000 7f781affd640 1 --2- 172.21.15.111:0/2064774632 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f77fc06ac00 0x7f77fc06d0c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f780c033000 tx=0x7f780c009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:24.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.547+0000 7f78137fe640 1 -- 172.21.15.111:0/2064774632 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f7804054080 con 0x7f781c0d2810 2024-09-13T17:58:24.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.707+0000 7f782179c640 1 -- 172.21.15.111:0/2064774632 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f781c09bd40 con 0x7f781c0d2810 2024-09-13T17:58:24.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.707+0000 7f78137fe640 1 -- 172.21.15.111:0/2064774632 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f78040576a0 con 0x7f781c0d2810 2024-09-13T17:58:24.710 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:58:24.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.709+0000 7f782179c640 1 -- 172.21.15.111:0/2064774632 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f77fc06ac00 msgr2=0x7f77fc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.709+0000 7f782179c640 1 --2- 172.21.15.111:0/2064774632 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f77fc06ac00 0x7f77fc06d0c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f780c033000 tx=0x7f780c009000 comp rx=0 tx=0).stop 2024-09-13T17:58:24.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.709+0000 7f782179c640 1 -- 172.21.15.111:0/2064774632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f781c0d2810 msgr2=0x7f781c0b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.709+0000 7f782179c640 1 --2- 172.21.15.111:0/2064774632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f781c0d2810 0x7f781c0b40a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f780400c610 tx=0x7f780400cad0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.709+0000 7f782179c640 1 -- 172.21.15.111:0/2064774632 shutdown_connections 2024-09-13T17:58:24.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.709+0000 7f782179c640 1 --2- 172.21.15.111:0/2064774632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f781c15fbc0 0x7f781c0b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.709+0000 7f782179c640 1 --2- 172.21.15.111:0/2064774632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f781c0d2810 0x7f781c0b40a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.709+0000 7f782179c640 1 --2- 172.21.15.111:0/2064774632 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f77fc06ac00 0x7f77fc06d0c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.709+0000 7f782179c640 1 --2- 172.21.15.111:0/2064774632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f781c0cd6c0 0x7f781c1498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.709+0000 7f782179c640 1 -- 172.21.15.111:0/2064774632 >> 172.21.15.111:0/2064774632 conn(0x7f781c09b830 msgr2=0x7f781c0c8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:24.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.710+0000 7f782179c640 1 -- 172.21.15.111:0/2064774632 shutdown_connections 2024-09-13T17:58:24.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.710+0000 7f782179c640 1 -- 172.21.15.111:0/2064774632 wait complete. 2024-09-13T17:58:24.722 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:58:24.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.822+0000 7fe45d6d0640 1 Processor -- start 2024-09-13T17:58:24.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.822+0000 7fe45d6d0640 1 -- start start 2024-09-13T17:58:24.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.823+0000 7fe45d6d0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe4580cd500 0x7fe4580b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:24.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.823+0000 7fe45d6d0640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4580d1f40 con 0x7fe4580d1650 2024-09-13T17:58:24.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.823+0000 7fe45d6d0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4580d2190 con 0x7fe4580b1370 2024-09-13T17:58:24.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.823+0000 7fe45d6d0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4580ccb20 con 0x7fe4580cd500 2024-09-13T17:58:24.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.823+0000 7fe456ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe4580cd500 0x7fe4580b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:24.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.823+0000 7fe4577fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe4580d1650 0x7fe4580b1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:55924/0 (socket says 172.21.15.111:55924) 2024-09-13T17:58:24.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.823+0000 7fe456ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe4580cd500 0x7fe4580b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33930/0 (socket says 172.21.15.111:33930) 2024-09-13T17:58:24.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.823+0000 7fe4577fe640 1 -- 172.21.15.111:0/3566561622 learned_addr learned my addr 172.21.15.111:0/3566561622 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:24.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.823+0000 7fe456ffd640 1 -- 172.21.15.111:0/3566561622 >> v1:172.21.15.188:6789/0 conn(0x7fe4580b1370 legacy=0x7fe4580b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:24.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.823+0000 7fe456ffd640 1 -- 172.21.15.111:0/3566561622 >> v1:172.21.15.111:6789/0 conn(0x7fe4580d1650 legacy=0x7fe4580b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.823+0000 7fe456ffd640 1 -- 172.21.15.111:0/3566561622 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4580cce00 con 0x7fe4580cd500 2024-09-13T17:58:24.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.823+0000 7fe4567fc640 1 -- 172.21.15.111:0/3566561622 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 314568448 0 0) 0x7fe4580d1f40 con 0x7fe4580d1650 2024-09-13T17:58:24.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.824+0000 7fe456ffd640 1 --2- 172.21.15.111:0/3566561622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4580cd500 0x7fe4580b3850 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe4480097a0 tx=0x7fe44802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae261709d4e617d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:24.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.824+0000 7fe4567fc640 1 -- 172.21.15.111:0/3566561622 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448032440 con 0x7fe4580cd500 2024-09-13T17:58:24.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.824+0000 7fe4567fc640 1 -- 172.21.15.111:0/3566561622 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4480325a0 con 0x7fe4580cd500 2024-09-13T17:58:24.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.824+0000 7fe4567fc640 1 -- 172.21.15.111:0/3566561622 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448032930 con 0x7fe4580cd500 2024-09-13T17:58:24.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.824+0000 7fe45d6d0640 1 -- 172.21.15.111:0/3566561622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4580cd500 msgr2=0x7fe4580b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.824+0000 7fe45d6d0640 1 --2- 172.21.15.111:0/3566561622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4580cd500 0x7fe4580b3850 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe4480097a0 tx=0x7fe44802a1e0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.825+0000 7fe45d6d0640 1 -- 172.21.15.111:0/3566561622 shutdown_connections 2024-09-13T17:58:24.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.825+0000 7fe45d6d0640 1 --2- 172.21.15.111:0/3566561622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4580cd500 0x7fe4580b3850 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.825+0000 7fe45d6d0640 1 -- 172.21.15.111:0/3566561622 >> 172.21.15.111:0/3566561622 conn(0x7fe458089f60 msgr2=0x7fe4580d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:24.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.825+0000 7fe45d6d0640 1 -- 172.21.15.111:0/3566561622 shutdown_connections 2024-09-13T17:58:24.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.825+0000 7fe45d6d0640 1 -- 172.21.15.111:0/3566561622 wait complete. 2024-09-13T17:58:24.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.826+0000 7fe45d6d0640 1 Processor -- start 2024-09-13T17:58:24.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.826+0000 7fe45d6d0640 1 -- start start 2024-09-13T17:58:24.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe45d6d0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4580cd500 0x7fe45806c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:24.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe45d6d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4580d1650 0x7fe4580c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:24.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe45d6d0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4580c7e90 0x7fe45814d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:24.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe45d6d0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45814b550 con 0x7fe4580c7e90 2024-09-13T17:58:24.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe45d6d0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580c8660 con 0x7fe4580cd500 2024-09-13T17:58:24.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe45d6d0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580c8940 con 0x7fe4580d1650 2024-09-13T17:58:24.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe4577fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4580c7e90 0x7fe45814d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:24.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe4577fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4580c7e90 0x7fe45814d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33942/0 (socket says 172.21.15.111:33942) 2024-09-13T17:58:24.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe4577fe640 1 -- 172.21.15.111:0/971707242 learned_addr learned my addr 172.21.15.111:0/971707242 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:24.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe456ffd640 1 --2- 172.21.15.111:0/971707242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4580cd500 0x7fe45806c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:24.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe44e7fc640 1 --2- 172.21.15.111:0/971707242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4580d1650 0x7fe4580c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:24.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe4577fe640 1 -- 172.21.15.111:0/971707242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4580d1650 msgr2=0x7fe4580c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe4577fe640 1 --2- 172.21.15.111:0/971707242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4580d1650 0x7fe4580c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe4577fe640 1 -- 172.21.15.111:0/971707242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4580cd500 msgr2=0x7fe45806c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe4577fe640 1 --2- 172.21.15.111:0/971707242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4580cd500 0x7fe45806c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe4577fe640 1 -- 172.21.15.111:0/971707242 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe448009400 con 0x7fe4580c7e90 2024-09-13T17:58:24.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe4577fe640 1 --2- 172.21.15.111:0/971707242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4580c7e90 0x7fe45814d5e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe45000c700 tx=0x7fe45000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:24.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe44ffff640 1 -- 172.21.15.111:0/971707242 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45000ce10 con 0x7fe4580c7e90 2024-09-13T17:58:24.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe44ffff640 1 -- 172.21.15.111:0/971707242 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45000a640 con 0x7fe4580c7e90 2024-09-13T17:58:24.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe45d6d0640 1 -- 172.21.15.111:0/971707242 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45814b7f0 con 0x7fe4580c7e90 2024-09-13T17:58:24.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe44ffff640 1 -- 172.21.15.111:0/971707242 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45000a9f0 con 0x7fe4580c7e90 2024-09-13T17:58:24.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.827+0000 7fe45d6d0640 1 -- 172.21.15.111:0/971707242 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4580c82e0 con 0x7fe4580c7e90 2024-09-13T17:58:24.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.828+0000 7fe45d6d0640 1 -- 172.21.15.111:0/971707242 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4580b3850 con 0x7fe4580c7e90 2024-09-13T17:58:24.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.829+0000 7fe44ffff640 1 -- 172.21.15.111:0/971707242 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe45000ab50 con 0x7fe4580c7e90 2024-09-13T17:58:24.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.829+0000 7fe44ffff640 1 --2- 172.21.15.111:0/971707242 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe43806ac00 0x7fe43806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:24.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.829+0000 7fe44ffff640 1 -- 172.21.15.111:0/971707242 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe450088db0 con 0x7fe4580c7e90 2024-09-13T17:58:24.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.830+0000 7fe456ffd640 1 --2- 172.21.15.111:0/971707242 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe43806ac00 0x7fe43806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:24.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.830+0000 7fe456ffd640 1 --2- 172.21.15.111:0/971707242 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe43806ac00 0x7fe43806d0c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe448034000 tx=0x7fe44802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:24.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.834+0000 7fe44ffff640 1 -- 172.21.15.111:0/971707242 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fe450053c80 con 0x7fe4580c7e90 2024-09-13T17:58:24.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.993+0000 7fe45d6d0640 1 -- 172.21.15.111:0/971707242 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe4580b4ed0 con 0x7fe4580c7e90 2024-09-13T17:58:24.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.994+0000 7fe44ffff640 1 -- 172.21.15.111:0/971707242 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe450018020 con 0x7fe4580c7e90 2024-09-13T17:58:24.996 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:24.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.995+0000 7fe45d6d0640 1 -- 172.21.15.111:0/971707242 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe43806ac00 msgr2=0x7fe43806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.995+0000 7fe45d6d0640 1 --2- 172.21.15.111:0/971707242 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe43806ac00 0x7fe43806d0c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe448034000 tx=0x7fe44802e000 comp rx=0 tx=0).stop 2024-09-13T17:58:24.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.995+0000 7fe45d6d0640 1 -- 172.21.15.111:0/971707242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4580c7e90 msgr2=0x7fe45814d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:24.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.995+0000 7fe45d6d0640 1 --2- 172.21.15.111:0/971707242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4580c7e90 0x7fe45814d5e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe45000c700 tx=0x7fe45000cbc0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.996+0000 7fe45d6d0640 1 -- 172.21.15.111:0/971707242 shutdown_connections 2024-09-13T17:58:24.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.996+0000 7fe45d6d0640 1 --2- 172.21.15.111:0/971707242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4580c7e90 0x7fe45814d5e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.996+0000 7fe45d6d0640 1 --2- 172.21.15.111:0/971707242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4580d1650 0x7fe4580c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.996+0000 7fe45d6d0640 1 --2- 172.21.15.111:0/971707242 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe43806ac00 0x7fe43806d0c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.996+0000 7fe45d6d0640 1 --2- 172.21.15.111:0/971707242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4580cd500 0x7fe45806c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:24.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.996+0000 7fe45d6d0640 1 -- 172.21.15.111:0/971707242 >> 172.21.15.111:0/971707242 conn(0x7fe458089f60 msgr2=0x7fe45815bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:24.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.996+0000 7fe45d6d0640 1 -- 172.21.15.111:0/971707242 shutdown_connections 2024-09-13T17:58:24.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:24.996+0000 7fe45d6d0640 1 -- 172.21.15.111:0/971707242 wait complete. 2024-09-13T17:58:25.008 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:58:25.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.105+0000 7f0748b8a640 1 Processor -- start 2024-09-13T17:58:25.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.105+0000 7f0748b8a640 1 -- start start 2024-09-13T17:58:25.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.105+0000 7f0748b8a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0744072110 0x7f0744070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.106+0000 7f0748b8a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f07440c8920 0x7f07440c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.106+0000 7f0748b8a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07440abdf0 con 0x7f0744071d30 2024-09-13T17:58:25.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.106+0000 7f0748b8a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07440c9310 con 0x7f07440c8920 2024-09-13T17:58:25.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.106+0000 7f0748b8a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07440b2280 con 0x7f0744072110 2024-09-13T17:58:25.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.106+0000 7f0742d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f07440c8920 0x7f07440c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.106+0000 7f074259c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f0744071d30 0x7f07440b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:55936/0 (socket says 172.21.15.111:55936) 2024-09-13T17:58:25.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.106+0000 7f0742d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f07440c8920 0x7f07440c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33950/0 (socket says 172.21.15.111:33950) 2024-09-13T17:58:25.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.106+0000 7f074259c640 1 -- 172.21.15.111:0/1647864626 learned_addr learned my addr 172.21.15.111:0/1647864626 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:25.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.106+0000 7f0741d9b640 1 --2- 172.21.15.111:0/1647864626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0744072110 0x7f0744070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.106+0000 7f0742d9d640 1 -- 172.21.15.111:0/1647864626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0744072110 msgr2=0x7f0744070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:25.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.106+0000 7f0742d9d640 1 --2- 172.21.15.111:0/1647864626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0744072110 0x7f0744070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.106+0000 7f0742d9d640 1 -- 172.21.15.111:0/1647864626 >> v1:172.21.15.111:6789/0 conn(0x7f0744071d30 legacy=0x7f07440b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:25.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.106+0000 7f0742d9d640 1 -- 172.21.15.111:0/1647864626 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07440ab1e0 con 0x7f07440c8920 2024-09-13T17:58:25.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.106+0000 7f074159a640 1 -- 172.21.15.111:0/1647864626 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371473541 0 0) 0x7f07440abdf0 con 0x7f0744071d30 2024-09-13T17:58:25.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.107+0000 7f0742d9d640 1 --2- 172.21.15.111:0/1647864626 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07440c8920 0x7f07440c8dd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f073c009dc0 tx=0x7f073c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665bf723a60e4c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:25.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.107+0000 7f074159a640 1 -- 172.21.15.111:0/1647864626 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c007740 con 0x7f07440c8920 2024-09-13T17:58:25.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.107+0000 7f074159a640 1 -- 172.21.15.111:0/1647864626 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073c02d050 con 0x7f07440c8920 2024-09-13T17:58:25.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.107+0000 7f074159a640 1 -- 172.21.15.111:0/1647864626 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c007ac0 con 0x7f07440c8920 2024-09-13T17:58:25.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.107+0000 7f0748b8a640 1 -- 172.21.15.111:0/1647864626 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07440c8920 msgr2=0x7f07440c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:25.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.107+0000 7f0748b8a640 1 --2- 172.21.15.111:0/1647864626 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07440c8920 0x7f07440c8dd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f073c009dc0 tx=0x7f073c02a680 comp rx=0 tx=0).stop 2024-09-13T17:58:25.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.107+0000 7f0748b8a640 1 -- 172.21.15.111:0/1647864626 shutdown_connections 2024-09-13T17:58:25.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.107+0000 7f0748b8a640 1 --2- 172.21.15.111:0/1647864626 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07440c8920 0x7f07440c8dd0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.107+0000 7f0748b8a640 1 --2- 172.21.15.111:0/1647864626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0744072110 0x7f0744070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.107+0000 7f0748b8a640 1 -- 172.21.15.111:0/1647864626 >> 172.21.15.111:0/1647864626 conn(0x7f0744069ed0 msgr2=0x7f074406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:25.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.107+0000 7f0748b8a640 1 -- 172.21.15.111:0/1647864626 shutdown_connections 2024-09-13T17:58:25.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.108+0000 7f0748b8a640 1 -- 172.21.15.111:0/1647864626 wait complete. 2024-09-13T17:58:25.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.109+0000 7f0748b8a640 1 Processor -- start 2024-09-13T17:58:25.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.109+0000 7f0748b8a640 1 -- start start 2024-09-13T17:58:25.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.109+0000 7f0748b8a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0744071d30 0x7f074406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.109+0000 7f0748b8a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0744072110 0x7f0744156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.109+0000 7f0748b8a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07440c8920 0x7f07440cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0748b8a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07440c7c60 con 0x7f07440c8920 2024-09-13T17:58:25.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0748b8a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0744157570 con 0x7f0744071d30 2024-09-13T17:58:25.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0748b8a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0744157040 con 0x7f0744072110 2024-09-13T17:58:25.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0742d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07440c8920 0x7f07440cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0742d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07440c8920 0x7f07440cd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33952/0 (socket says 172.21.15.111:33952) 2024-09-13T17:58:25.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0742d9d640 1 -- 172.21.15.111:0/228652695 learned_addr learned my addr 172.21.15.111:0/228652695 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:25.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f074259c640 1 --2- 172.21.15.111:0/228652695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0744071d30 0x7f074406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0741d9b640 1 --2- 172.21.15.111:0/228652695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0744072110 0x7f0744156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0742d9d640 1 -- 172.21.15.111:0/228652695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0744072110 msgr2=0x7f0744156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:25.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0742d9d640 1 --2- 172.21.15.111:0/228652695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0744072110 0x7f0744156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0742d9d640 1 -- 172.21.15.111:0/228652695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0744071d30 msgr2=0x7f074406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:25.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0742d9d640 1 --2- 172.21.15.111:0/228652695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0744071d30 0x7f074406dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0742d9d640 1 -- 172.21.15.111:0/228652695 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073c009a70 con 0x7f07440c8920 2024-09-13T17:58:25.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f074259c640 1 --2- 172.21.15.111:0/228652695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0744071d30 0x7f074406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:25.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0742d9d640 1 --2- 172.21.15.111:0/228652695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07440c8920 0x7f07440cd400 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f073c009eb0 tx=0x7f073c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:25.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0732ffd640 1 -- 172.21.15.111:0/228652695 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c031950 con 0x7f07440c8920 2024-09-13T17:58:25.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0732ffd640 1 -- 172.21.15.111:0/228652695 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073c031ab0 con 0x7f07440c8920 2024-09-13T17:58:25.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0748b8a640 1 -- 172.21.15.111:0/228652695 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07440c7620 con 0x7f07440c8920 2024-09-13T17:58:25.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0732ffd640 1 -- 172.21.15.111:0/228652695 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c031e40 con 0x7f07440c8920 2024-09-13T17:58:25.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.110+0000 7f0748b8a640 1 -- 172.21.15.111:0/228652695 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074406ca30 con 0x7f07440c8920 2024-09-13T17:58:25.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.111+0000 7f0748b8a640 1 -- 172.21.15.111:0/228652695 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0744070190 con 0x7f07440c8920 2024-09-13T17:58:25.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.112+0000 7f0732ffd640 1 -- 172.21.15.111:0/228652695 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f073c03b410 con 0x7f07440c8920 2024-09-13T17:58:25.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.112+0000 7f0732ffd640 1 --2- 172.21.15.111:0/228652695 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f072406ac30 0x7f072406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.112+0000 7f0732ffd640 1 -- 172.21.15.111:0/228652695 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f073c0aac70 con 0x7f07440c8920 2024-09-13T17:58:25.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.112+0000 7f074259c640 1 --2- 172.21.15.111:0/228652695 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f072406ac30 0x7f072406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.113+0000 7f074259c640 1 --2- 172.21.15.111:0/228652695 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f072406ac30 0x7f072406d0f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0734000f60 tx=0x7f0734007c30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:25.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.116+0000 7f0732ffd640 1 -- 172.21.15.111:0/228652695 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f073c075b40 con 0x7f07440c8920 2024-09-13T17:58:25.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.286+0000 7f0748b8a640 1 -- 172.21.15.111:0/228652695 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f074415cb20 con 0x7f07440c8920 2024-09-13T17:58:25.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.287+0000 7f0732ffd640 1 -- 172.21.15.111:0/228652695 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f073c079160 con 0x7f07440c8920 2024-09-13T17:58:25.289 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2024-09-13T17:58:25.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.289+0000 7f0748b8a640 1 -- 172.21.15.111:0/228652695 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f072406ac30 msgr2=0x7f072406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:25.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.289+0000 7f0748b8a640 1 --2- 172.21.15.111:0/228652695 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f072406ac30 0x7f072406d0f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0734000f60 tx=0x7f0734007c30 comp rx=0 tx=0).stop 2024-09-13T17:58:25.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.289+0000 7f0748b8a640 1 -- 172.21.15.111:0/228652695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07440c8920 msgr2=0x7f07440cd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:25.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.289+0000 7f0748b8a640 1 --2- 172.21.15.111:0/228652695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07440c8920 0x7f07440cd400 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f073c009eb0 tx=0x7f073c004a40 comp rx=0 tx=0).stop 2024-09-13T17:58:25.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.289+0000 7f0748b8a640 1 -- 172.21.15.111:0/228652695 shutdown_connections 2024-09-13T17:58:25.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.289+0000 7f0748b8a640 1 --2- 172.21.15.111:0/228652695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07440c8920 0x7f07440cd400 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.289+0000 7f0748b8a640 1 --2- 172.21.15.111:0/228652695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0744072110 0x7f0744156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.289+0000 7f0748b8a640 1 --2- 172.21.15.111:0/228652695 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f072406ac30 0x7f072406d0f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.289+0000 7f0748b8a640 1 --2- 172.21.15.111:0/228652695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0744071d30 0x7f074406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.289+0000 7f0748b8a640 1 -- 172.21.15.111:0/228652695 >> 172.21.15.111:0/228652695 conn(0x7f0744069ed0 msgr2=0x7f07440adcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:25.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.289+0000 7f0748b8a640 1 -- 172.21.15.111:0/228652695 shutdown_connections 2024-09-13T17:58:25.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.289+0000 7f0748b8a640 1 -- 172.21.15.111:0/228652695 wait complete. 2024-09-13T17:58:25.302 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi111.front.sepia.ceph.com... 2024-09-13T17:58:25.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:25.302 DEBUG:teuthology.orchestra.run.smithi111:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-09-13T17:58:25.327 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:58:25.327 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T17:58:25.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:25.327 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-13T17:58:25.382 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:58:25.383 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:58:25.402 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:58:25.403 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:58:25.461 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:25.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.556+0000 7f1e7ef26640 1 Processor -- start 2024-09-13T17:58:25.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.556+0000 7f1e7ef26640 1 -- start start 2024-09-13T17:58:25.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.557+0000 7f1e7ef26640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e780d07b0 0x7f1e7806ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.557+0000 7f1e7ef26640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1e780b4660 0x7f1e7806b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.557+0000 7f1e7ef26640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e780d1840 con 0x7f1e7815c6e0 2024-09-13T17:58:25.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.557+0000 7f1e7ef26640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e780d0c30 con 0x7f1e780b4660 2024-09-13T17:58:25.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.557+0000 7f1e7ef26640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780d0f10 con 0x7f1e780d07b0 2024-09-13T17:58:25.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.557+0000 7f1e7d4c3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1e780b4660 0x7f1e7806b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.557+0000 7f1e7d4c3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1e780b4660 0x7f1e7806b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33968/0 (socket says 172.21.15.111:33968) 2024-09-13T17:58:25.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.557+0000 7f1e7d4c3640 1 -- 172.21.15.111:0/1439967280 learned_addr learned my addr 172.21.15.111:0/1439967280 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:25.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.558+0000 7f1e6ffff640 1 --2- 172.21.15.111:0/1439967280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e780d07b0 0x7f1e7806ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.558+0000 7f1e7d4c3640 1 -- 172.21.15.111:0/1439967280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e780d07b0 msgr2=0x7f1e7806ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:25.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.558+0000 7f1e7d4c3640 1 --2- 172.21.15.111:0/1439967280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e780d07b0 0x7f1e7806ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.558+0000 7f1e7d4c3640 1 -- 172.21.15.111:0/1439967280 >> v1:172.21.15.188:6789/0 conn(0x7f1e7815c6e0 legacy=0x7f1e780b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:25.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.558+0000 7f1e7d4c3640 1 -- 172.21.15.111:0/1439967280 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e780d11f0 con 0x7f1e780b4660 2024-09-13T17:58:25.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.558+0000 7f1e6ffff640 1 --2- 172.21.15.111:0/1439967280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e780d07b0 0x7f1e7806ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:25.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.558+0000 7f1e7d4c3640 1 --2- 172.21.15.111:0/1439967280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e780b4660 0x7f1e7806b250 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009e50 tx=0x7f1e7002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd60327837c7cf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:25.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.559+0000 7f1e6f7fe640 1 -- 172.21.15.111:0/1439967280 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e70031600 con 0x7f1e780b4660 2024-09-13T17:58:25.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.559+0000 7f1e6f7fe640 1 -- 172.21.15.111:0/1439967280 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7002d050 con 0x7f1e780b4660 2024-09-13T17:58:25.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.559+0000 7f1e6f7fe640 1 -- 172.21.15.111:0/1439967280 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e70031980 con 0x7f1e780b4660 2024-09-13T17:58:25.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.559+0000 7f1e7ef26640 1 -- 172.21.15.111:0/1439967280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e780b4660 msgr2=0x7f1e7806b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:25.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.559+0000 7f1e7ef26640 1 --2- 172.21.15.111:0/1439967280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e780b4660 0x7f1e7806b250 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009e50 tx=0x7f1e7002a630 comp rx=0 tx=0).stop 2024-09-13T17:58:25.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.559+0000 7f1e7ef26640 1 -- 172.21.15.111:0/1439967280 shutdown_connections 2024-09-13T17:58:25.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.559+0000 7f1e7ef26640 1 --2- 172.21.15.111:0/1439967280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e780b4660 0x7f1e7806b250 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.559+0000 7f1e7ef26640 1 --2- 172.21.15.111:0/1439967280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e780d07b0 0x7f1e7806ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.559+0000 7f1e7ef26640 1 -- 172.21.15.111:0/1439967280 >> 172.21.15.111:0/1439967280 conn(0x7f1e78089f60 msgr2=0x7f1e780d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:25.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.559+0000 7f1e7ef26640 1 -- 172.21.15.111:0/1439967280 shutdown_connections 2024-09-13T17:58:25.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.559+0000 7f1e7ef26640 1 -- 172.21.15.111:0/1439967280 wait complete. 2024-09-13T17:58:25.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.560+0000 7f1e7ef26640 1 Processor -- start 2024-09-13T17:58:25.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.560+0000 7f1e7ef26640 1 -- start start 2024-09-13T17:58:25.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.561+0000 7f1e7ef26640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e780b4660 0x7f1e7815a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.561+0000 7f1e7ef26640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e780d07b0 0x7f1e781f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.561+0000 7f1e7ef26640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e7815c6e0 0x7f1e780cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.561+0000 7f1e7ef26640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7815eb50 con 0x7f1e780d07b0 2024-09-13T17:58:25.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.561+0000 7f1e7ef26640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7815dbf0 con 0x7f1e780b4660 2024-09-13T17:58:25.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.561+0000 7f1e7ef26640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7815ded0 con 0x7f1e7815c6e0 2024-09-13T17:58:25.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.561+0000 7f1e6ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e780d07b0 0x7f1e781f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.561+0000 7f1e6ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e780d07b0 0x7f1e781f2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33974/0 (socket says 172.21.15.111:33974) 2024-09-13T17:58:25.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.561+0000 7f1e6ffff640 1 -- 172.21.15.111:0/2192739072 learned_addr learned my addr 172.21.15.111:0/2192739072 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:25.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.562+0000 7f1e7d4c3640 1 --2- 172.21.15.111:0/2192739072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e7815c6e0 0x7f1e780cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.562+0000 7f1e7ccc2640 1 --2- 172.21.15.111:0/2192739072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e780b4660 0x7f1e7815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.562+0000 7f1e6ffff640 1 -- 172.21.15.111:0/2192739072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e7815c6e0 msgr2=0x7f1e780cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:25.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.562+0000 7f1e6ffff640 1 --2- 172.21.15.111:0/2192739072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e7815c6e0 0x7f1e780cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.562+0000 7f1e6ffff640 1 -- 172.21.15.111:0/2192739072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e780b4660 msgr2=0x7f1e7815a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:25.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.562+0000 7f1e6ffff640 1 --2- 172.21.15.111:0/2192739072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e780b4660 0x7f1e7815a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.562+0000 7f1e6ffff640 1 -- 172.21.15.111:0/2192739072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e70009b30 con 0x7f1e780d07b0 2024-09-13T17:58:25.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.562+0000 7f1e7ccc2640 1 --2- 172.21.15.111:0/2192739072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e780b4660 0x7f1e7815a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:25.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.562+0000 7f1e6ffff640 1 --2- 172.21.15.111:0/2192739072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e780d07b0 0x7f1e781f2440 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800c7f0 tx=0x7f1e6800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:25.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.562+0000 7f1e6d7fa640 1 -- 172.21.15.111:0/2192739072 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68007820 con 0x7f1e780d07b0 2024-09-13T17:58:25.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.562+0000 7f1e6d7fa640 1 -- 172.21.15.111:0/2192739072 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e68007980 con 0x7f1e780d07b0 2024-09-13T17:58:25.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.562+0000 7f1e6d7fa640 1 -- 172.21.15.111:0/2192739072 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68007cf0 con 0x7f1e780d07b0 2024-09-13T17:58:25.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.562+0000 7f1e7ef26640 1 -- 172.21.15.111:0/2192739072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7815e1b0 con 0x7f1e780d07b0 2024-09-13T17:58:25.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.562+0000 7f1e7ef26640 1 -- 172.21.15.111:0/2192739072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7815e770 con 0x7f1e780d07b0 2024-09-13T17:58:25.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.563+0000 7f1e7ef26640 1 -- 172.21.15.111:0/2192739072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7815ec90 con 0x7f1e780d07b0 2024-09-13T17:58:25.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.564+0000 7f1e6d7fa640 1 -- 172.21.15.111:0/2192739072 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1e680113f0 con 0x7f1e780d07b0 2024-09-13T17:58:25.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.564+0000 7f1e6d7fa640 1 --2- 172.21.15.111:0/2192739072 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1e6406ac00 0x7f1e6406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.564+0000 7f1e6d7fa640 1 -- 172.21.15.111:0/2192739072 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1e68089ae0 con 0x7f1e780d07b0 2024-09-13T17:58:25.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.564+0000 7f1e7ccc2640 1 --2- 172.21.15.111:0/2192739072 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1e6406ac00 0x7f1e6406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.564+0000 7f1e7ccc2640 1 --2- 172.21.15.111:0/2192739072 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1e6406ac00 0x7f1e6406d0c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1e74003eb0 tx=0x7f1e74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:25.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.568+0000 7f1e6d7fa640 1 -- 172.21.15.111:0/2192739072 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f1e7815ec90 con 0x7f1e780d07b0 2024-09-13T17:58:25.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.733+0000 7f1e7ef26640 1 -- 172.21.15.111:0/2192739072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e7815ec90 con 0x7f1e780d07b0 2024-09-13T17:58:25.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.734+0000 7f1e6d7fa640 1 -- 172.21.15.111:0/2192739072 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18794 (secure 0 0 0) 0x7f1e7815ec90 con 0x7f1e780d07b0 2024-09-13T17:58:25.736 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:25.737 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":26,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:07.749113+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.737+0000 7f1e7ef26640 1 -- 172.21.15.111:0/2192739072 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1e6406ac00 msgr2=0x7f1e6406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.737+0000 7f1e7ef26640 1 --2- 172.21.15.111:0/2192739072 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1e6406ac00 0x7f1e6406d0c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1e74003eb0 tx=0x7f1e74009040 comp rx=0 tx=0).stop 2024-09-13T17:58:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.737+0000 7f1e7ef26640 1 -- 172.21.15.111:0/2192739072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e780d07b0 msgr2=0x7f1e781f2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.737+0000 7f1e7ef26640 1 --2- 172.21.15.111:0/2192739072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e780d07b0 0x7f1e781f2440 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800c7f0 tx=0x7f1e6800ccb0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.737+0000 7f1e7ef26640 1 -- 172.21.15.111:0/2192739072 shutdown_connections 2024-09-13T17:58:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.737+0000 7f1e7ef26640 1 --2- 172.21.15.111:0/2192739072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e7815c6e0 0x7f1e780cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.737+0000 7f1e7ef26640 1 --2- 172.21.15.111:0/2192739072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e780d07b0 0x7f1e781f2440 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.737+0000 7f1e7ef26640 1 --2- 172.21.15.111:0/2192739072 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1e6406ac00 0x7f1e6406d0c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.737+0000 7f1e7ef26640 1 --2- 172.21.15.111:0/2192739072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e780b4660 0x7f1e7815a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.737+0000 7f1e7ef26640 1 -- 172.21.15.111:0/2192739072 >> 172.21.15.111:0/2192739072 conn(0x7f1e78089f60 msgr2=0x7f1e780b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:25.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.737+0000 7f1e7ef26640 1 -- 172.21.15.111:0/2192739072 shutdown_connections 2024-09-13T17:58:25.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.737+0000 7f1e7ef26640 1 -- 172.21.15.111:0/2192739072 wait complete. 2024-09-13T17:58:25.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:58:25.752 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:58:25.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.852+0000 7f72d71db640 1 Processor -- start 2024-09-13T17:58:25.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.852+0000 7f72d71db640 1 -- start start 2024-09-13T17:58:25.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.853+0000 7f72d71db640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f72d015a3b0 0x7f72d00b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.853+0000 7f72d71db640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f72d00cec00 0x7f72d00ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.853+0000 7f72d71db640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d015a790 con 0x7f72d00b1720 2024-09-13T17:58:25.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.853+0000 7f72d71db640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d00b2680 con 0x7f72d015a3b0 2024-09-13T17:58:25.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.853+0000 7f72d71db640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d00b2960 con 0x7f72d00cec00 2024-09-13T17:58:25.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.853+0000 7f72d4f77640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f72d015a3b0 0x7f72d00b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.853+0000 7f72c7fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f72d00b1720 0x7f72d00ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:55938/0 (socket says 172.21.15.111:55938) 2024-09-13T17:58:25.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.853+0000 7f72c7fff640 1 -- 172.21.15.111:0/543087330 learned_addr learned my addr 172.21.15.111:0/543087330 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:25.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.853+0000 7f72d4f77640 1 --2- 172.21.15.111:0/543087330 >> v2:172.21.15.111:3300/0 conn(0x7f72d015a3b0 0x7f72d00b20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33990/0 (socket says 172.21.15.111:33990) 2024-09-13T17:58:25.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.854+0000 7f72d5778640 1 --2- 172.21.15.111:0/543087330 >> v2:172.21.15.188:3300/0 conn(0x7f72d00cec00 0x7f72d00ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.854+0000 7f72d4f77640 1 -- 172.21.15.111:0/543087330 >> v2:172.21.15.188:3300/0 conn(0x7f72d00cec00 msgr2=0x7f72d00ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:25.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.854+0000 7f72d4f77640 1 --2- 172.21.15.111:0/543087330 >> v2:172.21.15.188:3300/0 conn(0x7f72d00cec00 0x7f72d00ab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.854+0000 7f72d4f77640 1 -- 172.21.15.111:0/543087330 >> v1:172.21.15.111:6789/0 conn(0x7f72d00b1720 legacy=0x7f72d00ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:25.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.854+0000 7f72d4f77640 1 -- 172.21.15.111:0/543087330 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d00b2c40 con 0x7f72d015a3b0 2024-09-13T17:58:25.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.854+0000 7f72d5778640 1 --2- 172.21.15.111:0/543087330 >> v2:172.21.15.188:3300/0 conn(0x7f72d00cec00 0x7f72d00ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:25.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.854+0000 7f72d4f77640 1 --2- 172.21.15.111:0/543087330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72d015a3b0 0x7f72d00b20a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f72d00c5310 tx=0x7f72cc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f25f683ce36f8b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:25.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.854+0000 7f72c77fe640 1 -- 172.21.15.111:0/543087330 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc02aea0 con 0x7f72d015a3b0 2024-09-13T17:58:25.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.854+0000 7f72c77fe640 1 -- 172.21.15.111:0/543087330 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc02d050 con 0x7f72d015a3b0 2024-09-13T17:58:25.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.854+0000 7f72c77fe640 1 -- 172.21.15.111:0/543087330 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc007960 con 0x7f72d015a3b0 2024-09-13T17:58:25.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.855+0000 7f72d71db640 1 -- 172.21.15.111:0/543087330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72d015a3b0 msgr2=0x7f72d00b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:25.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.855+0000 7f72d71db640 1 --2- 172.21.15.111:0/543087330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72d015a3b0 0x7f72d00b20a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f72d00c5310 tx=0x7f72cc02a680 comp rx=0 tx=0).stop 2024-09-13T17:58:25.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.855+0000 7f72d71db640 1 -- 172.21.15.111:0/543087330 shutdown_connections 2024-09-13T17:58:25.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.855+0000 7f72d71db640 1 --2- 172.21.15.111:0/543087330 >> v2:172.21.15.188:3300/0 conn(0x7f72d00cec00 0x7f72d00ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.855+0000 7f72d71db640 1 --2- 172.21.15.111:0/543087330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72d015a3b0 0x7f72d00b20a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.855+0000 7f72d71db640 1 -- 172.21.15.111:0/543087330 >> 172.21.15.111:0/543087330 conn(0x7f72d006a0b0 msgr2=0x7f72d006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:25.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.855+0000 7f72d71db640 1 -- 172.21.15.111:0/543087330 shutdown_connections 2024-09-13T17:58:25.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.855+0000 7f72d71db640 1 -- 172.21.15.111:0/543087330 wait complete. 2024-09-13T17:58:25.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.856+0000 7f72d71db640 1 Processor -- start 2024-09-13T17:58:25.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.856+0000 7f72d71db640 1 -- start start 2024-09-13T17:58:25.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.857+0000 7f72d71db640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72d00b1720 0x7f72d0097a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.857+0000 7f72d71db640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72d00cec00 0x7f72d00adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.857+0000 7f72d71db640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72d015a3b0 0x7f72d00b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.857+0000 7f72d71db640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d015b7d0 con 0x7f72d00cec00 2024-09-13T17:58:25.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.857+0000 7f72d71db640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d00b05f0 con 0x7f72d00b1720 2024-09-13T17:58:25.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.857+0000 7f72d71db640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d00ae370 con 0x7f72d015a3b0 2024-09-13T17:58:25.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.857+0000 7f72c7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72d00cec00 0x7f72d00adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.857+0000 7f72c7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72d00cec00 0x7f72d00adc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33994/0 (socket says 172.21.15.111:33994) 2024-09-13T17:58:25.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.857+0000 7f72c7fff640 1 -- 172.21.15.111:0/2469560501 learned_addr learned my addr 172.21.15.111:0/2469560501 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:25.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.857+0000 7f72d4f77640 1 --2- 172.21.15.111:0/2469560501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72d00b1720 0x7f72d0097a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.857+0000 7f72d5778640 1 --2- 172.21.15.111:0/2469560501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72d015a3b0 0x7f72d00b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.858+0000 7f72c7fff640 1 -- 172.21.15.111:0/2469560501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72d015a3b0 msgr2=0x7f72d00b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:25.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.858+0000 7f72c7fff640 1 --2- 172.21.15.111:0/2469560501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72d015a3b0 0x7f72d00b00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.858+0000 7f72c7fff640 1 -- 172.21.15.111:0/2469560501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72d00b1720 msgr2=0x7f72d0097a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:25.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.858+0000 7f72c7fff640 1 --2- 172.21.15.111:0/2469560501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72d00b1720 0x7f72d0097a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:25.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.858+0000 7f72c7fff640 1 -- 172.21.15.111:0/2469560501 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc009a70 con 0x7f72d00cec00 2024-09-13T17:58:25.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.858+0000 7f72c7fff640 1 --2- 172.21.15.111:0/2469560501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72d00cec00 0x7f72d00adc90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f72c000c400 tx=0x7f72c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:25.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.858+0000 7f72c57fa640 1 -- 172.21.15.111:0/2469560501 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0005ce0 con 0x7f72d00cec00 2024-09-13T17:58:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.858+0000 7f72c57fa640 1 -- 172.21.15.111:0/2469560501 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c0005e40 con 0x7f72d00cec00 2024-09-13T17:58:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.858+0000 7f72d71db640 1 -- 172.21.15.111:0/2469560501 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d00ae650 con 0x7f72d00cec00 2024-09-13T17:58:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.858+0000 7f72c57fa640 1 -- 172.21.15.111:0/2469560501 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0007310 con 0x7f72d00cec00 2024-09-13T17:58:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.858+0000 7f72d71db640 1 -- 172.21.15.111:0/2469560501 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d00aec10 con 0x7f72d00cec00 2024-09-13T17:58:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.859+0000 7f72d71db640 1 -- 172.21.15.111:0/2469560501 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d015b5a0 con 0x7f72d00cec00 2024-09-13T17:58:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.859+0000 7f72c57fa640 1 -- 172.21.15.111:0/2469560501 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f72c0007470 con 0x7f72d00cec00 2024-09-13T17:58:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.860+0000 7f72c57fa640 1 --2- 172.21.15.111:0/2469560501 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f72bc06ac00 0x7f72bc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.860+0000 7f72c57fa640 1 -- 172.21.15.111:0/2469560501 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f72c0088c10 con 0x7f72d00cec00 2024-09-13T17:58:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.860+0000 7f72d4f77640 1 --2- 172.21.15.111:0/2469560501 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f72bc06ac00 0x7f72bc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.860+0000 7f72d4f77640 1 --2- 172.21.15.111:0/2469560501 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f72bc06ac00 0x7f72bc06d0c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f72cc034000 tx=0x7f72cc02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:25.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:25.864+0000 7f72c57fa640 1 -- 172.21.15.111:0/2469560501 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f72d015b5a0 con 0x7f72d00cec00 2024-09-13T17:58:26.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.034+0000 7f72d71db640 1 -- 172.21.15.111:0/2469560501 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f72d015b5a0 con 0x7f72d00cec00 2024-09-13T17:58:26.036 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.034+0000 7f72c57fa640 1 -- 172.21.15.111:0/2469560501 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f72c0017020 con 0x7f72d00cec00 2024-09-13T17:58:26.037 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:58:26.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.036+0000 7f72d71db640 1 -- 172.21.15.111:0/2469560501 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f72bc06ac00 msgr2=0x7f72bc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.036+0000 7f72d71db640 1 --2- 172.21.15.111:0/2469560501 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f72bc06ac00 0x7f72bc06d0c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f72cc034000 tx=0x7f72cc02f000 comp rx=0 tx=0).stop 2024-09-13T17:58:26.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.036+0000 7f72d71db640 1 -- 172.21.15.111:0/2469560501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72d00cec00 msgr2=0x7f72d00adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.036+0000 7f72d71db640 1 --2- 172.21.15.111:0/2469560501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72d00cec00 0x7f72d00adc90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f72c000c400 tx=0x7f72c000c8c0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.037+0000 7f72d71db640 1 -- 172.21.15.111:0/2469560501 shutdown_connections 2024-09-13T17:58:26.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.037+0000 7f72d71db640 1 --2- 172.21.15.111:0/2469560501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72d015a3b0 0x7f72d00b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.037+0000 7f72d71db640 1 --2- 172.21.15.111:0/2469560501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72d00cec00 0x7f72d00adc90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.037+0000 7f72d71db640 1 --2- 172.21.15.111:0/2469560501 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f72bc06ac00 0x7f72bc06d0c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.037+0000 7f72d71db640 1 --2- 172.21.15.111:0/2469560501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72d00b1720 0x7f72d0097a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.037+0000 7f72d71db640 1 -- 172.21.15.111:0/2469560501 >> 172.21.15.111:0/2469560501 conn(0x7f72d006a0b0 msgr2=0x7f72d0070930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:26.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.037+0000 7f72d71db640 1 -- 172.21.15.111:0/2469560501 shutdown_connections 2024-09-13T17:58:26.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.037+0000 7f72d71db640 1 -- 172.21.15.111:0/2469560501 wait complete. 2024-09-13T17:58:26.049 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:58:26.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.147+0000 7fe8ef99c640 1 Processor -- start 2024-09-13T17:58:26.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.148+0000 7fe8ef99c640 1 -- start start 2024-09-13T17:58:26.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.148+0000 7fe8ef99c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe8e8159310 0x7fe8e80ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.148+0000 7fe8ef99c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe8e80b2b40 0x7fe8e80ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.148+0000 7fe8ef99c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8e80ab680 0x7fe8e80ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.148+0000 7fe8ef99c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e80ad940 con 0x7fe8e80b2b40 2024-09-13T17:58:26.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.148+0000 7fe8ef99c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e80ccbe0 con 0x7fe8e8159310 2024-09-13T17:58:26.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.148+0000 7fe8ef99c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e80b1810 con 0x7fe8e80ab680 2024-09-13T17:58:26.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.148+0000 7fe8ecf37640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe8e80b2b40 0x7fe8e80ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.149+0000 7fe8ecf37640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe8e80b2b40 0x7fe8e80ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34006/0 (socket says 172.21.15.111:34006) 2024-09-13T17:58:26.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.149+0000 7fe8ecf37640 1 -- 172.21.15.111:0/1050882347 learned_addr learned my addr 172.21.15.111:0/1050882347 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:26.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.149+0000 7fe8ed738640 1 --2- 172.21.15.111:0/1050882347 >> v2:172.21.15.188:3300/0 conn(0x7fe8e8159310 0x7fe8e80ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.149+0000 7fe8edf39640 1 --2- 172.21.15.111:0/1050882347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8e80ab680 0x7fe8e80ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.149+0000 7fe8ecf37640 1 -- 172.21.15.111:0/1050882347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8e80ab680 msgr2=0x7fe8e80ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:26.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.149+0000 7fe8ecf37640 1 --2- 172.21.15.111:0/1050882347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8e80ab680 0x7fe8e80ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.149+0000 7fe8ecf37640 1 -- 172.21.15.111:0/1050882347 >> v2:172.21.15.188:3300/0 conn(0x7fe8e8159310 msgr2=0x7fe8e80ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:26.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.149+0000 7fe8ecf37640 1 --2- 172.21.15.111:0/1050882347 >> v2:172.21.15.188:3300/0 conn(0x7fe8e8159310 0x7fe8e80ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.149+0000 7fe8ecf37640 1 -- 172.21.15.111:0/1050882347 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e806d9e0 con 0x7fe8e80b2b40 2024-09-13T17:58:26.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.149+0000 7fe8edf39640 1 --2- 172.21.15.111:0/1050882347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8e80ab680 0x7fe8e80ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:26.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.149+0000 7fe8ed738640 1 --2- 172.21.15.111:0/1050882347 >> v2:172.21.15.188:3300/0 conn(0x7fe8e8159310 0x7fe8e80ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:26.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.150+0000 7fe8ecf37640 1 --2- 172.21.15.111:0/1050882347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8e80b2b40 0x7fe8e80ab140 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80cac90 tx=0x7fe8d802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f8fd4c4d991fe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:26.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.150+0000 7fe8dffff640 1 -- 172.21.15.111:0/1050882347 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d802ae90 con 0x7fe8e80b2b40 2024-09-13T17:58:26.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.150+0000 7fe8dffff640 1 -- 172.21.15.111:0/1050882347 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d8032440 con 0x7fe8e80b2b40 2024-09-13T17:58:26.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.150+0000 7fe8dffff640 1 -- 172.21.15.111:0/1050882347 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80327d0 con 0x7fe8e80b2b40 2024-09-13T17:58:26.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.150+0000 7fe8ef99c640 1 -- 172.21.15.111:0/1050882347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8e80b2b40 msgr2=0x7fe8e80ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.150+0000 7fe8ef99c640 1 --2- 172.21.15.111:0/1050882347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8e80b2b40 0x7fe8e80ab140 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80cac90 tx=0x7fe8d802a140 comp rx=0 tx=0).stop 2024-09-13T17:58:26.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.150+0000 7fe8ef99c640 1 -- 172.21.15.111:0/1050882347 shutdown_connections 2024-09-13T17:58:26.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.150+0000 7fe8ef99c640 1 --2- 172.21.15.111:0/1050882347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8e80ab680 0x7fe8e80ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.150+0000 7fe8ef99c640 1 --2- 172.21.15.111:0/1050882347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8e80b2b40 0x7fe8e80ab140 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.150+0000 7fe8ef99c640 1 --2- 172.21.15.111:0/1050882347 >> v2:172.21.15.188:3300/0 conn(0x7fe8e8159310 0x7fe8e80ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.150+0000 7fe8ef99c640 1 -- 172.21.15.111:0/1050882347 >> 172.21.15.111:0/1050882347 conn(0x7fe8e806a0b0 msgr2=0x7fe8e806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:26.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.150+0000 7fe8ef99c640 1 -- 172.21.15.111:0/1050882347 shutdown_connections 2024-09-13T17:58:26.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.150+0000 7fe8ef99c640 1 -- 172.21.15.111:0/1050882347 wait complete. 2024-09-13T17:58:26.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.151+0000 7fe8ef99c640 1 Processor -- start 2024-09-13T17:58:26.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.152+0000 7fe8ef99c640 1 -- start start 2024-09-13T17:58:26.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.152+0000 7fe8ef99c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8e80ab680 0x7fe8e80c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.152+0000 7fe8ef99c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8e80b2b40 0x7fe8e81528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.152+0000 7fe8ef99c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8e8159310 0x7fe8e809bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.152+0000 7fe8ef99c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e8151e90 con 0x7fe8e8159310 2024-09-13T17:58:26.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.152+0000 7fe8ef99c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e809c270 con 0x7fe8e80ab680 2024-09-13T17:58:26.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.152+0000 7fe8ef99c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e8150440 con 0x7fe8e80b2b40 2024-09-13T17:58:26.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.152+0000 7fe8edf39640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8e8159310 0x7fe8e809bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.152+0000 7fe8edf39640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8e8159310 0x7fe8e809bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34016/0 (socket says 172.21.15.111:34016) 2024-09-13T17:58:26.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.152+0000 7fe8edf39640 1 -- 172.21.15.111:0/2842493883 learned_addr learned my addr 172.21.15.111:0/2842493883 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:26.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.153+0000 7fe8edf39640 1 -- 172.21.15.111:0/2842493883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8e80b2b40 msgr2=0x7fe8e81528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.153+0000 7fe8edf39640 1 --2- 172.21.15.111:0/2842493883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8e80b2b40 0x7fe8e81528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.153+0000 7fe8ed738640 1 --2- 172.21.15.111:0/2842493883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8e80ab680 0x7fe8e80c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.153+0000 7fe8edf39640 1 -- 172.21.15.111:0/2842493883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8e80ab680 msgr2=0x7fe8e80c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.153+0000 7fe8edf39640 1 --2- 172.21.15.111:0/2842493883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8e80ab680 0x7fe8e80c49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.153+0000 7fe8edf39640 1 -- 172.21.15.111:0/2842493883 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d80094d0 con 0x7fe8e8159310 2024-09-13T17:58:26.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.153+0000 7fe8edf39640 1 --2- 172.21.15.111:0/2842493883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8e8159310 0x7fe8e809bd30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0007e10 tx=0x7fe8e000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:26.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.153+0000 7fe8ddffb640 1 -- 172.21.15.111:0/2842493883 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0003b90 con 0x7fe8e8159310 2024-09-13T17:58:26.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.153+0000 7fe8ddffb640 1 -- 172.21.15.111:0/2842493883 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e000f040 con 0x7fe8e8159310 2024-09-13T17:58:26.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.153+0000 7fe8ef99c640 1 -- 172.21.15.111:0/2842493883 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e81506e0 con 0x7fe8e8159310 2024-09-13T17:58:26.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.153+0000 7fe8ddffb640 1 -- 172.21.15.111:0/2842493883 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e000de80 con 0x7fe8e8159310 2024-09-13T17:58:26.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.153+0000 7fe8ef99c640 1 -- 172.21.15.111:0/2842493883 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e81523a0 con 0x7fe8e8159310 2024-09-13T17:58:26.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.154+0000 7fe8ef99c640 1 -- 172.21.15.111:0/2842493883 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e8152590 con 0x7fe8e8159310 2024-09-13T17:58:26.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.154+0000 7fe8ddffb640 1 -- 172.21.15.111:0/2842493883 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe8e0004030 con 0x7fe8e8159310 2024-09-13T17:58:26.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.155+0000 7fe8ddffb640 1 --2- 172.21.15.111:0/2842493883 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe8cc06ac30 0x7fe8cc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.155+0000 7fe8ed738640 1 --2- 172.21.15.111:0/2842493883 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe8cc06ac30 0x7fe8cc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.155+0000 7fe8ddffb640 1 -- 172.21.15.111:0/2842493883 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe8e0089980 con 0x7fe8e8159310 2024-09-13T17:58:26.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.155+0000 7fe8ed738640 1 --2- 172.21.15.111:0/2842493883 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe8cc06ac30 0x7fe8cc06d0f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4006fd0 tx=0x7fe8e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:26.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.159+0000 7fe8ddffb640 1 -- 172.21.15.111:0/2842493883 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fe8e0054830 con 0x7fe8e8159310 2024-09-13T17:58:26.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.320+0000 7fe8ef99c640 1 -- 172.21.15.111:0/2842493883 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe8e80cee10 con 0x7fe8e8159310 2024-09-13T17:58:26.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.320+0000 7fe8ddffb640 1 -- 172.21.15.111:0/2842493883 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe8e0057e50 con 0x7fe8e8159310 2024-09-13T17:58:26.322 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:26.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.322+0000 7fe8ef99c640 1 -- 172.21.15.111:0/2842493883 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe8cc06ac30 msgr2=0x7fe8cc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.322+0000 7fe8ef99c640 1 --2- 172.21.15.111:0/2842493883 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe8cc06ac30 0x7fe8cc06d0f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4006fd0 tx=0x7fe8e4008040 comp rx=0 tx=0).stop 2024-09-13T17:58:26.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.322+0000 7fe8ef99c640 1 -- 172.21.15.111:0/2842493883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8e8159310 msgr2=0x7fe8e809bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.322+0000 7fe8ef99c640 1 --2- 172.21.15.111:0/2842493883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8e8159310 0x7fe8e809bd30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0007e10 tx=0x7fe8e000d4a0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.322+0000 7fe8ef99c640 1 -- 172.21.15.111:0/2842493883 shutdown_connections 2024-09-13T17:58:26.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.322+0000 7fe8ef99c640 1 --2- 172.21.15.111:0/2842493883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8e8159310 0x7fe8e809bd30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.322+0000 7fe8ef99c640 1 --2- 172.21.15.111:0/2842493883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8e80b2b40 0x7fe8e81528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.322+0000 7fe8ef99c640 1 --2- 172.21.15.111:0/2842493883 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe8cc06ac30 0x7fe8cc06d0f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.322+0000 7fe8ef99c640 1 --2- 172.21.15.111:0/2842493883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8e80ab680 0x7fe8e80c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.322+0000 7fe8ef99c640 1 -- 172.21.15.111:0/2842493883 >> 172.21.15.111:0/2842493883 conn(0x7fe8e806a0b0 msgr2=0x7fe8e806a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:26.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.322+0000 7fe8ef99c640 1 -- 172.21.15.111:0/2842493883 shutdown_connections 2024-09-13T17:58:26.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.322+0000 7fe8ef99c640 1 -- 172.21.15.111:0/2842493883 wait complete. 2024-09-13T17:58:26.336 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:58:26.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.435+0000 7f4d6e1d5640 1 Processor -- start 2024-09-13T17:58:26.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.435+0000 7f4d6e1d5640 1 -- start start 2024-09-13T17:58:26.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d6e1d5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4d68159310 0x7f4d680ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d6e1d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d680b2b40 0x7f4d680ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d6e1d5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d680ccbe0 con 0x7f4d680ab680 2024-09-13T17:58:26.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d6e1d5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d680ad940 con 0x7f4d68159310 2024-09-13T17:58:26.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d6e1d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d680b1810 con 0x7f4d680b2b40 2024-09-13T17:58:26.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d67fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4d680ab680 0x7f4d680ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:55948/0 (socket says 172.21.15.111:55948) 2024-09-13T17:58:26.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d67fff640 1 -- 172.21.15.111:0/3719932516 learned_addr learned my addr 172.21.15.111:0/3719932516 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:26.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d677fe640 1 --2- 172.21.15.111:0/3719932516 >> v2:172.21.15.188:3300/0 conn(0x7f4d68159310 0x7f4d680ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d66ffd640 1 --2- 172.21.15.111:0/3719932516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d680b2b40 0x7f4d680ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d667fc640 1 -- 172.21.15.111:0/3719932516 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164390842 0 0) 0x7f4d680ccbe0 con 0x7f4d680ab680 2024-09-13T17:58:26.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d667fc640 1 -- 172.21.15.111:0/3719932516 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d50005730 con 0x7f4d680ab680 2024-09-13T17:58:26.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d667fc640 1 -- 172.21.15.111:0/3719932516 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1793231414 0 0) 0x7f4d50005730 con 0x7f4d680ab680 2024-09-13T17:58:26.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d667fc640 1 -- 172.21.15.111:0/3719932516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d680b2b40 msgr2=0x7f4d680ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:26.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d667fc640 1 --2- 172.21.15.111:0/3719932516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d680b2b40 0x7f4d680ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d667fc640 1 -- 172.21.15.111:0/3719932516 >> v2:172.21.15.188:3300/0 conn(0x7f4d68159310 msgr2=0x7f4d680ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:26.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d667fc640 1 --2- 172.21.15.111:0/3719932516 >> v2:172.21.15.188:3300/0 conn(0x7f4d68159310 0x7f4d680ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d667fc640 1 -- 172.21.15.111:0/3719932516 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6806d9e0 con 0x7f4d680ab680 2024-09-13T17:58:26.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d667fc640 1 -- 172.21.15.111:0/3719932516 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f4d600019f0 con 0x7f4d680ab680 2024-09-13T17:58:26.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d667fc640 1 -- 172.21.15.111:0/3719932516 >> v1:172.21.15.111:6789/0 conn(0x7f4d680ab680 legacy=0x7f4d680ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d667fc640 1 --2- 172.21.15.111:0/3719932516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d50005b80 0x7f4d50005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d667fc640 1 --2- 172.21.15.111:0/3719932516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d680b2b40 0x7f4d500069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d667fc640 1 --2- 172.21.15.111:0/3719932516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d50006f10 0x7f4d50007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d667fc640 1 -- 172.21.15.111:0/3719932516 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d680b1810 con 0x7f4d50005b80 2024-09-13T17:58:26.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d667fc640 1 -- 172.21.15.111:0/3719932516 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5000db90 con 0x7f4d50006f10 2024-09-13T17:58:26.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.436+0000 7f4d667fc640 1 -- 172.21.15.111:0/3719932516 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5000dd00 con 0x7f4d680b2b40 2024-09-13T17:58:26.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.437+0000 7f4d677fe640 1 --2- 172.21.15.111:0/3719932516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d50005b80 0x7f4d50005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.437+0000 7f4d67fff640 1 --2- 172.21.15.111:0/3719932516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d50006f10 0x7f4d50007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.437+0000 7f4d66ffd640 1 --2- 172.21.15.111:0/3719932516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d680b2b40 0x7f4d500069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.437+0000 7f4d677fe640 1 -- 172.21.15.111:0/3719932516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d680b2b40 msgr2=0x7f4d500069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:26.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.437+0000 7f4d677fe640 1 --2- 172.21.15.111:0/3719932516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d680b2b40 0x7f4d500069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.437+0000 7f4d677fe640 1 -- 172.21.15.111:0/3719932516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d50006f10 msgr2=0x7f4d50007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:26.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.437+0000 7f4d677fe640 1 --2- 172.21.15.111:0/3719932516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d50006f10 0x7f4d50007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.437+0000 7f4d677fe640 1 -- 172.21.15.111:0/3719932516 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d50005730 con 0x7f4d50005b80 2024-09-13T17:58:26.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.437+0000 7f4d67fff640 1 --2- 172.21.15.111:0/3719932516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d50006f10 0x7f4d50007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:26.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.438+0000 7f4d677fe640 1 --2- 172.21.15.111:0/3719932516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d50005b80 0x7f4d50005fa0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009ac0 tx=0x7f4d50007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fdcc030ec54bc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:26.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.438+0000 7f4d667fc640 1 -- 172.21.15.111:0/3719932516 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58031400 con 0x7f4d50005b80 2024-09-13T17:58:26.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.438+0000 7f4d667fc640 1 -- 172.21.15.111:0/3719932516 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5802d050 con 0x7f4d50005b80 2024-09-13T17:58:26.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.438+0000 7f4d667fc640 1 -- 172.21.15.111:0/3719932516 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58031780 con 0x7f4d50005b80 2024-09-13T17:58:26.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.438+0000 7f4d6e1d5640 1 -- 172.21.15.111:0/3719932516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d50005b80 msgr2=0x7f4d50005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.438+0000 7f4d6e1d5640 1 --2- 172.21.15.111:0/3719932516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d50005b80 0x7f4d50005fa0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009ac0 tx=0x7f4d50007a00 comp rx=0 tx=0).stop 2024-09-13T17:58:26.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.438+0000 7f4d67fff640 1 -- 172.21.15.111:0/3719932516 reap_dead start 2024-09-13T17:58:26.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.438+0000 7f4d6e1d5640 1 -- 172.21.15.111:0/3719932516 shutdown_connections 2024-09-13T17:58:26.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.438+0000 7f4d6e1d5640 1 -- 172.21.15.111:0/3719932516 >> 172.21.15.111:0/3719932516 conn(0x7f4d6806a0b0 msgr2=0x7f4d6806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:26.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.438+0000 7f4d6e1d5640 1 -- 172.21.15.111:0/3719932516 shutdown_connections 2024-09-13T17:58:26.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.438+0000 7f4d6e1d5640 1 -- 172.21.15.111:0/3719932516 wait complete. 2024-09-13T17:58:26.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.439+0000 7f4d6e1d5640 1 Processor -- start 2024-09-13T17:58:26.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.440+0000 7f4d6e1d5640 1 -- start start 2024-09-13T17:58:26.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.440+0000 7f4d6e1d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d50005b80 0x7f4d680c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.440+0000 7f4d6e1d5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6809b400 0x7f4d68098d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.440+0000 7f4d6e1d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d68099250 0x7f4d680aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.440+0000 7f4d6e1d5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d680c5120 con 0x7f4d6809b400 2024-09-13T17:58:26.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.440+0000 7f4d6e1d5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6809c180 con 0x7f4d50005b80 2024-09-13T17:58:26.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.440+0000 7f4d6e1d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6809b8e0 con 0x7f4d68099250 2024-09-13T17:58:26.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.440+0000 7f4d66ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6809b400 0x7f4d68098d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.440+0000 7f4d66ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6809b400 0x7f4d68098d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34028/0 (socket says 172.21.15.111:34028) 2024-09-13T17:58:26.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.440+0000 7f4d66ffd640 1 -- 172.21.15.111:0/4261256213 learned_addr learned my addr 172.21.15.111:0/4261256213 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:26.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.441+0000 7f4d67fff640 1 --2- 172.21.15.111:0/4261256213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d68099250 0x7f4d680aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.441+0000 7f4d677fe640 1 --2- 172.21.15.111:0/4261256213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d50005b80 0x7f4d680c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.441+0000 7f4d66ffd640 1 -- 172.21.15.111:0/4261256213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d68099250 msgr2=0x7f4d680aba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.441+0000 7f4d66ffd640 1 --2- 172.21.15.111:0/4261256213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d68099250 0x7f4d680aba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.441+0000 7f4d66ffd640 1 -- 172.21.15.111:0/4261256213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d50005b80 msgr2=0x7f4d680c4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.441+0000 7f4d66ffd640 1 --2- 172.21.15.111:0/4261256213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d50005b80 0x7f4d680c4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.441+0000 7f4d66ffd640 1 -- 172.21.15.111:0/4261256213 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6809bbc0 con 0x7f4d6809b400 2024-09-13T17:58:26.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.441+0000 7f4d66ffd640 1 --2- 172.21.15.111:0/4261256213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6809b400 0x7f4d68098d10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0119a0 tx=0x7f4d5c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:26.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.441+0000 7f4d4ffff640 1 -- 172.21.15.111:0/4261256213 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c00fa30 con 0x7f4d6809b400 2024-09-13T17:58:26.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.441+0000 7f4d4ffff640 1 -- 172.21.15.111:0/4261256213 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c00fb90 con 0x7f4d6809b400 2024-09-13T17:58:26.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.441+0000 7f4d6e1d5640 1 -- 172.21.15.111:0/4261256213 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6809bea0 con 0x7f4d6809b400 2024-09-13T17:58:26.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.441+0000 7f4d4ffff640 1 -- 172.21.15.111:0/4261256213 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c014440 con 0x7f4d6809b400 2024-09-13T17:58:26.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.441+0000 7f4d6e1d5640 1 -- 172.21.15.111:0/4261256213 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d680abfa0 con 0x7f4d6809b400 2024-09-13T17:58:26.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.442+0000 7f4d6e1d5640 1 -- 172.21.15.111:0/4261256213 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6806e280 con 0x7f4d6809b400 2024-09-13T17:58:26.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.443+0000 7f4d4ffff640 1 -- 172.21.15.111:0/4261256213 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4d5c0145a0 con 0x7f4d6809b400 2024-09-13T17:58:26.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.443+0000 7f4d4ffff640 1 --2- 172.21.15.111:0/4261256213 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d5406ac30 0x7f4d5406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.443+0000 7f4d677fe640 1 --2- 172.21.15.111:0/4261256213 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d5406ac30 0x7f4d5406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.443+0000 7f4d4ffff640 1 -- 172.21.15.111:0/4261256213 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4d5c089f90 con 0x7f4d6809b400 2024-09-13T17:58:26.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.444+0000 7f4d677fe640 1 --2- 172.21.15.111:0/4261256213 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d5406ac30 0x7f4d5406d0f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d58033000 tx=0x7f4d58008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:26.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.447+0000 7f4d4ffff640 1 -- 172.21.15.111:0/4261256213 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4d5c0bd9c0 con 0x7f4d6809b400 2024-09-13T17:58:26.517 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:58:26.520 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:58:26.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.617+0000 7f4d6e1d5640 1 -- 172.21.15.111:0/4261256213 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d680ab140 con 0x7f4d6809b400 2024-09-13T17:58:26.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.617+0000 7f4d4ffff640 1 -- 172.21.15.111:0/4261256213 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4d5c054db0 con 0x7f4d6809b400 2024-09-13T17:58:26.619 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:58:26.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.619+0000 7f4d6e1d5640 1 -- 172.21.15.111:0/4261256213 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d5406ac30 msgr2=0x7f4d5406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.619+0000 7f4d6e1d5640 1 --2- 172.21.15.111:0/4261256213 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d5406ac30 0x7f4d5406d0f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d58033000 tx=0x7f4d58008030 comp rx=0 tx=0).stop 2024-09-13T17:58:26.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.619+0000 7f4d6e1d5640 1 -- 172.21.15.111:0/4261256213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6809b400 msgr2=0x7f4d68098d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.619+0000 7f4d6e1d5640 1 --2- 172.21.15.111:0/4261256213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6809b400 0x7f4d68098d10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0119a0 tx=0x7f4d5c011da0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.619+0000 7f4d6e1d5640 1 -- 172.21.15.111:0/4261256213 shutdown_connections 2024-09-13T17:58:26.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.619+0000 7f4d6e1d5640 1 --2- 172.21.15.111:0/4261256213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d68099250 0x7f4d680aba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.619+0000 7f4d6e1d5640 1 --2- 172.21.15.111:0/4261256213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6809b400 0x7f4d68098d10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.619+0000 7f4d6e1d5640 1 --2- 172.21.15.111:0/4261256213 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4d5406ac30 0x7f4d5406d0f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.619+0000 7f4d6e1d5640 1 --2- 172.21.15.111:0/4261256213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d50005b80 0x7f4d680c4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.619+0000 7f4d6e1d5640 1 -- 172.21.15.111:0/4261256213 >> 172.21.15.111:0/4261256213 conn(0x7f4d6806a0b0 msgr2=0x7f4d680ad400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:26.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.619+0000 7f4d6e1d5640 1 -- 172.21.15.111:0/4261256213 shutdown_connections 2024-09-13T17:58:26.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.619+0000 7f4d6e1d5640 1 -- 172.21.15.111:0/4261256213 wait complete. 2024-09-13T17:58:26.632 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:58:26.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.770+0000 7f3388c61640 1 Processor -- start 2024-09-13T17:58:26.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.770+0000 7f3388c61640 1 -- start start 2024-09-13T17:58:26.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.771+0000 7f3388c61640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f338406c8a0 0x7f33840b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.771+0000 7f3388c61640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33840b2d10 con 0x7f3384158b00 2024-09-13T17:58:26.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.771+0000 7f3388c61640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33840b2ff0 con 0x7f3384071b20 2024-09-13T17:58:26.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.771+0000 7f3388c61640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3384070310 con 0x7f338406c8a0 2024-09-13T17:58:26.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.771+0000 7f337bfff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f338406c8a0 0x7f33840b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.771+0000 7f338259c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3384158b00 0x7f338406c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:55954/0 (socket says 172.21.15.111:55954) 2024-09-13T17:58:26.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.771+0000 7f338259c640 1 -- 172.21.15.111:0/717785881 learned_addr learned my addr 172.21.15.111:0/717785881 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:26.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.771+0000 7f337bfff640 1 --2- 172.21.15.111:0/717785881 >> v2:172.21.15.111:3300/0 conn(0x7f338406c8a0 0x7f33840b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34036/0 (socket says 172.21.15.111:34036) 2024-09-13T17:58:26.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.771+0000 7f337bfff640 1 -- 172.21.15.111:0/717785881 >> v1:172.21.15.188:6789/0 conn(0x7f3384071b20 legacy=0x7f33840b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:26.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.771+0000 7f337bfff640 1 -- 172.21.15.111:0/717785881 >> v1:172.21.15.111:6789/0 conn(0x7f3384158b00 legacy=0x7f338406c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.771+0000 7f3381d9b640 1 -- 172.21.15.111:0/717785881 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244376730 0 0) 0x7f33840b2d10 con 0x7f3384158b00 2024-09-13T17:58:26.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.771+0000 7f337bfff640 1 -- 172.21.15.111:0/717785881 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33840705f0 con 0x7f338406c8a0 2024-09-13T17:58:26.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.772+0000 7f337bfff640 1 --2- 172.21.15.111:0/717785881 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f338406c8a0 0x7f33840b1160 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f336c0097a0 tx=0x7f336c02b2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14551db76f06319 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:26.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.772+0000 7f3381d9b640 1 -- 172.21.15.111:0/717785881 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c033440 con 0x7f338406c8a0 2024-09-13T17:58:26.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.772+0000 7f3381d9b640 1 -- 172.21.15.111:0/717785881 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336c0335a0 con 0x7f338406c8a0 2024-09-13T17:58:26.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.772+0000 7f3381d9b640 1 -- 172.21.15.111:0/717785881 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c033930 con 0x7f338406c8a0 2024-09-13T17:58:26.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.772+0000 7f3388c61640 1 -- 172.21.15.111:0/717785881 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f338406c8a0 msgr2=0x7f33840b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.772+0000 7f3388c61640 1 --2- 172.21.15.111:0/717785881 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f338406c8a0 0x7f33840b1160 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f336c0097a0 tx=0x7f336c02b2f0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.772+0000 7f3388c61640 1 -- 172.21.15.111:0/717785881 shutdown_connections 2024-09-13T17:58:26.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.772+0000 7f3388c61640 1 --2- 172.21.15.111:0/717785881 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f338406c8a0 0x7f33840b1160 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.772+0000 7f3388c61640 1 -- 172.21.15.111:0/717785881 >> 172.21.15.111:0/717785881 conn(0x7f338406a0b0 msgr2=0x7f338406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:26.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.773+0000 7f3388c61640 1 -- 172.21.15.111:0/717785881 shutdown_connections 2024-09-13T17:58:26.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.773+0000 7f3388c61640 1 -- 172.21.15.111:0/717785881 wait complete. 2024-09-13T17:58:26.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.774+0000 7f3388c61640 1 Processor -- start 2024-09-13T17:58:26.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.774+0000 7f3388c61640 1 -- start start 2024-09-13T17:58:26.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.774+0000 7f3388c61640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f338406c8a0 0x7f33841ef640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.774+0000 7f3388c61640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3384071b20 0x7f33840c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.774+0000 7f3388c61640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3384158b00 0x7f33840af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f3388c61640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338409add0 con 0x7f338406c8a0 2024-09-13T17:58:26.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f3388c61640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33841f2e60 con 0x7f3384071b20 2024-09-13T17:58:26.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f3388c61640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338409a150 con 0x7f3384158b00 2024-09-13T17:58:26.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f338259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f338406c8a0 0x7f33841ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f338259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f338406c8a0 0x7f33841ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34038/0 (socket says 172.21.15.111:34038) 2024-09-13T17:58:26.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f338259c640 1 -- 172.21.15.111:0/2318429540 learned_addr learned my addr 172.21.15.111:0/2318429540 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:26.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f337bfff640 1 --2- 172.21.15.111:0/2318429540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3384071b20 0x7f33840c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f3382d9d640 1 --2- 172.21.15.111:0/2318429540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3384158b00 0x7f33840af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f338259c640 1 -- 172.21.15.111:0/2318429540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3384158b00 msgr2=0x7f33840af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f338259c640 1 --2- 172.21.15.111:0/2318429540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3384158b00 0x7f33840af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f338259c640 1 -- 172.21.15.111:0/2318429540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3384071b20 msgr2=0x7f33840c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f338259c640 1 --2- 172.21.15.111:0/2318429540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3384071b20 0x7f33840c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f338259c640 1 -- 172.21.15.111:0/2318429540 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336c009400 con 0x7f338406c8a0 2024-09-13T17:58:26.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f3382d9d640 1 --2- 172.21.15.111:0/2318429540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3384158b00 0x7f33840af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:26.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f338259c640 1 --2- 172.21.15.111:0/2318429540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f338406c8a0 0x7f33841ef640 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f337400c430 tx=0x7f337400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:26.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f337affd640 1 -- 172.21.15.111:0/2318429540 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33740074e0 con 0x7f338406c8a0 2024-09-13T17:58:26.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f337affd640 1 -- 172.21.15.111:0/2318429540 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3374007660 con 0x7f338406c8a0 2024-09-13T17:58:26.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f3388c61640 1 -- 172.21.15.111:0/2318429540 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338409a430 con 0x7f338406c8a0 2024-09-13T17:58:26.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f337affd640 1 -- 172.21.15.111:0/2318429540 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3374004a20 con 0x7f338406c8a0 2024-09-13T17:58:26.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.775+0000 7f3388c61640 1 -- 172.21.15.111:0/2318429540 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338409a9f0 con 0x7f338406c8a0 2024-09-13T17:58:26.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.776+0000 7f3388c61640 1 -- 172.21.15.111:0/2318429540 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33840cea70 con 0x7f338406c8a0 2024-09-13T17:58:26.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.777+0000 7f337affd640 1 -- 172.21.15.111:0/2318429540 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3374004b80 con 0x7f338406c8a0 2024-09-13T17:58:26.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.777+0000 7f337affd640 1 --2- 172.21.15.111:0/2318429540 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f336406ac00 0x7f336406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:26.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.777+0000 7f337bfff640 1 --2- 172.21.15.111:0/2318429540 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f336406ac00 0x7f336406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:26.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.777+0000 7f337affd640 1 -- 172.21.15.111:0/2318429540 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f33740893f0 con 0x7f338406c8a0 2024-09-13T17:58:26.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.778+0000 7f337bfff640 1 --2- 172.21.15.111:0/2318429540 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f336406ac00 0x7f336406d0c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f336c035000 tx=0x7f336c02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:26.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.781+0000 7f337affd640 1 -- 172.21.15.111:0/2318429540 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f33740542a0 con 0x7f338406c8a0 2024-09-13T17:58:26.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.943+0000 7f3388c61640 1 -- 172.21.15.111:0/2318429540 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f338406b150 con 0x7f338406c8a0 2024-09-13T17:58:26.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.944+0000 7f337affd640 1 -- 172.21.15.111:0/2318429540 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f33740578c0 con 0x7f338406c8a0 2024-09-13T17:58:26.948 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:26.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.945+0000 7f3388c61640 1 -- 172.21.15.111:0/2318429540 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f336406ac00 msgr2=0x7f336406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.945+0000 7f3388c61640 1 --2- 172.21.15.111:0/2318429540 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f336406ac00 0x7f336406d0c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f336c035000 tx=0x7f336c02f000 comp rx=0 tx=0).stop 2024-09-13T17:58:26.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.945+0000 7f3388c61640 1 -- 172.21.15.111:0/2318429540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f338406c8a0 msgr2=0x7f33841ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:26.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.945+0000 7f3388c61640 1 --2- 172.21.15.111:0/2318429540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f338406c8a0 0x7f33841ef640 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f337400c430 tx=0x7f337400c8f0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.945+0000 7f3388c61640 1 -- 172.21.15.111:0/2318429540 shutdown_connections 2024-09-13T17:58:26.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.945+0000 7f3388c61640 1 --2- 172.21.15.111:0/2318429540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3384158b00 0x7f33840af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.945+0000 7f3388c61640 1 --2- 172.21.15.111:0/2318429540 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f336406ac00 0x7f336406d0c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.945+0000 7f3388c61640 1 --2- 172.21.15.111:0/2318429540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3384071b20 0x7f33840c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.945+0000 7f3388c61640 1 --2- 172.21.15.111:0/2318429540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f338406c8a0 0x7f33841ef640 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:26.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.945+0000 7f3388c61640 1 -- 172.21.15.111:0/2318429540 >> 172.21.15.111:0/2318429540 conn(0x7f338406a0b0 msgr2=0x7f3384157050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:26.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.946+0000 7f3388c61640 1 -- 172.21.15.111:0/2318429540 shutdown_connections 2024-09-13T17:58:26.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:26.946+0000 7f3388c61640 1 -- 172.21.15.111:0/2318429540 wait complete. 2024-09-13T17:58:26.959 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:58:27.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.055+0000 7f0aa999c640 1 Processor -- start 2024-09-13T17:58:27.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.055+0000 7f0aa999c640 1 -- start start 2024-09-13T17:58:27.058 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.056+0000 7f0aa999c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0aa4071d30 0x7f0aa4070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.056+0000 7f0aa999c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0aa4072110 0x7f0aa40b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.056+0000 7f0aa999c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa40abdf0 con 0x7f0aa40c8920 2024-09-13T17:58:27.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.056+0000 7f0aa999c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa40ab1e0 con 0x7f0aa4071d30 2024-09-13T17:58:27.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.056+0000 7f0aa999c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa40ab4c0 con 0x7f0aa4072110 2024-09-13T17:58:27.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.056+0000 7f0aa2ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0aa4071d30 0x7f0aa4070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.056+0000 7f0aa37fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f0aa40c8920 0x7f0aa40ac280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:55966/0 (socket says 172.21.15.111:55966) 2024-09-13T17:58:27.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.056+0000 7f0aa37fe640 1 -- 172.21.15.111:0/2462622134 learned_addr learned my addr 172.21.15.111:0/2462622134 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:27.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.056+0000 7f0aa27fc640 1 --2- 172.21.15.111:0/2462622134 >> v2:172.21.15.188:3300/0 conn(0x7f0aa4072110 0x7f0aa40b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.056+0000 7f0aa2ffd640 1 -- 172.21.15.111:0/2462622134 >> v2:172.21.15.188:3300/0 conn(0x7f0aa4072110 msgr2=0x7f0aa40b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:27.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.056+0000 7f0aa2ffd640 1 --2- 172.21.15.111:0/2462622134 >> v2:172.21.15.188:3300/0 conn(0x7f0aa4072110 0x7f0aa40b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.056+0000 7f0aa2ffd640 1 -- 172.21.15.111:0/2462622134 >> v1:172.21.15.111:6789/0 conn(0x7f0aa40c8920 legacy=0x7f0aa40ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.056+0000 7f0aa1ffb640 1 -- 172.21.15.111:0/2462622134 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 617020937 0 0) 0x7f0aa40abdf0 con 0x7f0aa40c8920 2024-09-13T17:58:27.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.056+0000 7f0aa2ffd640 1 -- 172.21.15.111:0/2462622134 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa40ab7a0 con 0x7f0aa4071d30 2024-09-13T17:58:27.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.057+0000 7f0aa2ffd640 1 --2- 172.21.15.111:0/2462622134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0aa4071d30 0x7f0aa4070190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0aa40cf000 tx=0x7f0a9c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab4c1abb75b5cc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:27.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.057+0000 7f0aa1ffb640 1 -- 172.21.15.111:0/2462622134 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c032440 con 0x7f0aa4071d30 2024-09-13T17:58:27.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.057+0000 7f0aa1ffb640 1 -- 172.21.15.111:0/2462622134 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c0325a0 con 0x7f0aa4071d30 2024-09-13T17:58:27.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.057+0000 7f0aa1ffb640 1 -- 172.21.15.111:0/2462622134 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c032930 con 0x7f0aa4071d30 2024-09-13T17:58:27.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.058+0000 7f0aa999c640 1 -- 172.21.15.111:0/2462622134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0aa4071d30 msgr2=0x7f0aa4070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.058+0000 7f0aa999c640 1 --2- 172.21.15.111:0/2462622134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0aa4071d30 0x7f0aa4070190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0aa40cf000 tx=0x7f0a9c02a1a0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.058+0000 7f0aa999c640 1 -- 172.21.15.111:0/2462622134 shutdown_connections 2024-09-13T17:58:27.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.058+0000 7f0aa999c640 1 --2- 172.21.15.111:0/2462622134 >> v2:172.21.15.188:3300/0 conn(0x7f0aa4072110 0x7f0aa40b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.058+0000 7f0aa999c640 1 --2- 172.21.15.111:0/2462622134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0aa4071d30 0x7f0aa4070190 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.058+0000 7f0aa999c640 1 -- 172.21.15.111:0/2462622134 >> 172.21.15.111:0/2462622134 conn(0x7f0aa4069ed0 msgr2=0x7f0aa406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:27.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.058+0000 7f0aa999c640 1 -- 172.21.15.111:0/2462622134 shutdown_connections 2024-09-13T17:58:27.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.058+0000 7f0aa999c640 1 -- 172.21.15.111:0/2462622134 wait complete. 2024-09-13T17:58:27.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.059+0000 7f0aa999c640 1 Processor -- start 2024-09-13T17:58:27.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.059+0000 7f0aa999c640 1 -- start start 2024-09-13T17:58:27.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.060+0000 7f0aa999c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0aa4071d30 0x7f0aa4097890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.060+0000 7f0aa999c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0aa4072110 0x7f0aa415b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.060+0000 7f0aa999c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aa40c8920 0x7f0aa41670e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.060+0000 7f0aa999c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa415b2e0 con 0x7f0aa4072110 2024-09-13T17:58:27.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.060+0000 7f0aa999c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa4167a10 con 0x7f0aa4071d30 2024-09-13T17:58:27.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.060+0000 7f0aa999c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa4167cf0 con 0x7f0aa40c8920 2024-09-13T17:58:27.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.060+0000 7f0aa27fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0aa4072110 0x7f0aa415b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.060+0000 7f0aa27fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0aa4072110 0x7f0aa415b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34046/0 (socket says 172.21.15.111:34046) 2024-09-13T17:58:27.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.060+0000 7f0aa27fc640 1 -- 172.21.15.111:0/3686054830 learned_addr learned my addr 172.21.15.111:0/3686054830 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:27.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.060+0000 7f0aa2ffd640 1 --2- 172.21.15.111:0/3686054830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0aa4071d30 0x7f0aa4097890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.060+0000 7f0aa37fe640 1 --2- 172.21.15.111:0/3686054830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aa40c8920 0x7f0aa41670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.060+0000 7f0aa27fc640 1 -- 172.21.15.111:0/3686054830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aa40c8920 msgr2=0x7f0aa41670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.060+0000 7f0aa27fc640 1 --2- 172.21.15.111:0/3686054830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aa40c8920 0x7f0aa41670e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.060+0000 7f0aa27fc640 1 -- 172.21.15.111:0/3686054830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0aa4071d30 msgr2=0x7f0aa4097890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.060+0000 7f0aa27fc640 1 --2- 172.21.15.111:0/3686054830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0aa4071d30 0x7f0aa4097890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.060+0000 7f0aa27fc640 1 -- 172.21.15.111:0/3686054830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c009400 con 0x7f0aa4072110 2024-09-13T17:58:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.061+0000 7f0aa27fc640 1 --2- 172.21.15.111:0/3686054830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0aa4072110 0x7f0aa415b770 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00d800 tx=0x7f0a8c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.061+0000 7f0a9bfff640 1 -- 172.21.15.111:0/3686054830 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c00b6c0 con 0x7f0aa4072110 2024-09-13T17:58:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.061+0000 7f0a9bfff640 1 -- 172.21.15.111:0/3686054830 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c00b820 con 0x7f0aa4072110 2024-09-13T17:58:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.061+0000 7f0aa999c640 1 -- 172.21.15.111:0/3686054830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa415af70 con 0x7f0aa4072110 2024-09-13T17:58:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.061+0000 7f0a9bfff640 1 -- 172.21.15.111:0/3686054830 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c00bbb0 con 0x7f0aa4072110 2024-09-13T17:58:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.061+0000 7f0aa999c640 1 -- 172.21.15.111:0/3686054830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa415bcb0 con 0x7f0aa4072110 2024-09-13T17:58:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.062+0000 7f0aa999c640 1 -- 172.21.15.111:0/3686054830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa415add0 con 0x7f0aa4072110 2024-09-13T17:58:27.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.062+0000 7f0a9bfff640 1 -- 172.21.15.111:0/3686054830 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0a8c00bd10 con 0x7f0aa4072110 2024-09-13T17:58:27.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.063+0000 7f0a9bfff640 1 --2- 172.21.15.111:0/3686054830 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0a8406ac30 0x7f0a8406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.063+0000 7f0a9bfff640 1 -- 172.21.15.111:0/3686054830 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0a8c089b40 con 0x7f0aa4072110 2024-09-13T17:58:27.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.063+0000 7f0aa2ffd640 1 --2- 172.21.15.111:0/3686054830 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0a8406ac30 0x7f0a8406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.063+0000 7f0aa2ffd640 1 --2- 172.21.15.111:0/3686054830 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0a8406ac30 0x7f0a8406d0f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c034000 tx=0x7f0a9c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:27.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.067+0000 7f0a9bfff640 1 -- 172.21.15.111:0/3686054830 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f0aa415add0 con 0x7f0aa4072110 2024-09-13T17:58:27.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.236+0000 7f0aa999c640 1 -- 172.21.15.111:0/3686054830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0aa415add0 con 0x7f0aa4072110 2024-09-13T17:58:27.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.236+0000 7f0a9bfff640 1 -- 172.21.15.111:0/3686054830 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0aa415add0 con 0x7f0aa4072110 2024-09-13T17:58:27.238 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2024-09-13T17:58:27.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.238+0000 7f0aa999c640 1 -- 172.21.15.111:0/3686054830 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0a8406ac30 msgr2=0x7f0a8406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.238+0000 7f0aa999c640 1 --2- 172.21.15.111:0/3686054830 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0a8406ac30 0x7f0a8406d0f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c034000 tx=0x7f0a9c02e000 comp rx=0 tx=0).stop 2024-09-13T17:58:27.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.238+0000 7f0aa999c640 1 -- 172.21.15.111:0/3686054830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0aa4072110 msgr2=0x7f0aa415b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.238+0000 7f0aa999c640 1 --2- 172.21.15.111:0/3686054830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0aa4072110 0x7f0aa415b770 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00d800 tx=0x7f0a8c00dcc0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.238+0000 7f0aa999c640 1 -- 172.21.15.111:0/3686054830 shutdown_connections 2024-09-13T17:58:27.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.238+0000 7f0aa999c640 1 --2- 172.21.15.111:0/3686054830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aa40c8920 0x7f0aa41670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.238+0000 7f0aa999c640 1 --2- 172.21.15.111:0/3686054830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0aa4072110 0x7f0aa415b770 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.238+0000 7f0aa999c640 1 --2- 172.21.15.111:0/3686054830 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0a8406ac30 0x7f0a8406d0f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.238+0000 7f0aa999c640 1 --2- 172.21.15.111:0/3686054830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0aa4071d30 0x7f0aa4097890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.238+0000 7f0aa999c640 1 -- 172.21.15.111:0/3686054830 >> 172.21.15.111:0/3686054830 conn(0x7f0aa4069ed0 msgr2=0x7f0aa40ac870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:27.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.239+0000 7f0aa999c640 1 -- 172.21.15.111:0/3686054830 shutdown_connections 2024-09-13T17:58:27.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.239+0000 7f0aa999c640 1 -- 172.21.15.111:0/3686054830 wait complete. 2024-09-13T17:58:27.252 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:58:27.252 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:58:27.269 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:27.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.363+0000 7faf52de2640 1 Processor -- start 2024-09-13T17:58:27.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.363+0000 7faf52de2640 1 -- start start 2024-09-13T17:58:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.364+0000 7faf52de2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faf4c15a2e0 0x7faf4c0c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.364+0000 7faf52de2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faf4c0c9f80 0x7faf4c0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.364+0000 7faf52de2640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4c0cb670 con 0x7faf4c0cb220 2024-09-13T17:58:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.364+0000 7faf52de2640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4c0c8920 con 0x7faf4c0c9f80 2024-09-13T17:58:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.364+0000 7faf52de2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4c15a6c0 con 0x7faf4c15a2e0 2024-09-13T17:58:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.364+0000 7faf43fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faf4c0c9f80 0x7faf4c0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.364+0000 7faf5137f640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7faf4c0cb220 0x7faf4c0ce260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:55972/0 (socket says 172.21.15.111:55972) 2024-09-13T17:58:27.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.364+0000 7faf43fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faf4c0c9f80 0x7faf4c0cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34048/0 (socket says 172.21.15.111:34048) 2024-09-13T17:58:27.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.364+0000 7faf5137f640 1 -- 172.21.15.111:0/274111049 learned_addr learned my addr 172.21.15.111:0/274111049 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:27.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.364+0000 7faf50b7e640 1 --2- 172.21.15.111:0/274111049 >> v2:172.21.15.188:3300/0 conn(0x7faf4c15a2e0 0x7faf4c0c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.365+0000 7faf43fff640 1 -- 172.21.15.111:0/274111049 >> v2:172.21.15.188:3300/0 conn(0x7faf4c15a2e0 msgr2=0x7faf4c0c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:27.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.365+0000 7faf43fff640 1 --2- 172.21.15.111:0/274111049 >> v2:172.21.15.188:3300/0 conn(0x7faf4c15a2e0 0x7faf4c0c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.365+0000 7faf43fff640 1 -- 172.21.15.111:0/274111049 >> v1:172.21.15.111:6789/0 conn(0x7faf4c0cb220 legacy=0x7faf4c0ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.365+0000 7faf43fff640 1 -- 172.21.15.111:0/274111049 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c0ca500 con 0x7faf4c0c9f80 2024-09-13T17:58:27.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.365+0000 7faf50b7e640 1 --2- 172.21.15.111:0/274111049 >> v2:172.21.15.188:3300/0 conn(0x7faf4c15a2e0 0x7faf4c0c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:27.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.365+0000 7faf43fff640 1 --2- 172.21.15.111:0/274111049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf4c0c9f80 0x7faf4c0cace0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0cf340 tx=0x7faf3c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b804a3f3935363 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:27.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.365+0000 7faf437fe640 1 -- 172.21.15.111:0/274111049 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c032440 con 0x7faf4c0c9f80 2024-09-13T17:58:27.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.365+0000 7faf437fe640 1 -- 172.21.15.111:0/274111049 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3c0325a0 con 0x7faf4c0c9f80 2024-09-13T17:58:27.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.365+0000 7faf437fe640 1 -- 172.21.15.111:0/274111049 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c032930 con 0x7faf4c0c9f80 2024-09-13T17:58:27.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.366+0000 7faf52de2640 1 -- 172.21.15.111:0/274111049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf4c0c9f80 msgr2=0x7faf4c0cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.366+0000 7faf52de2640 1 --2- 172.21.15.111:0/274111049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf4c0c9f80 0x7faf4c0cace0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0cf340 tx=0x7faf3c02a1a0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.366+0000 7faf52de2640 1 -- 172.21.15.111:0/274111049 shutdown_connections 2024-09-13T17:58:27.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.366+0000 7faf52de2640 1 --2- 172.21.15.111:0/274111049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf4c0c9f80 0x7faf4c0cace0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.366+0000 7faf52de2640 1 --2- 172.21.15.111:0/274111049 >> v2:172.21.15.188:3300/0 conn(0x7faf4c15a2e0 0x7faf4c0c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.366+0000 7faf52de2640 1 -- 172.21.15.111:0/274111049 >> 172.21.15.111:0/274111049 conn(0x7faf4c069ed0 msgr2=0x7faf4c06a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.366+0000 7faf52de2640 1 -- 172.21.15.111:0/274111049 shutdown_connections 2024-09-13T17:58:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.366+0000 7faf52de2640 1 -- 172.21.15.111:0/274111049 wait complete. 2024-09-13T17:58:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.367+0000 7faf52de2640 1 Processor -- start 2024-09-13T17:58:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.367+0000 7faf52de2640 1 -- start start 2024-09-13T17:58:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.368+0000 7faf52de2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf4c0c9f80 0x7faf4c0975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.368+0000 7faf52de2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf4c0cb220 0x7faf4c1f3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.368+0000 7faf52de2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf4c15a2e0 0x7faf4c1f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.368+0000 7faf52de2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c15b450 con 0x7faf4c0c9f80 2024-09-13T17:58:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.368+0000 7faf52de2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c097b30 con 0x7faf4c15a2e0 2024-09-13T17:58:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.368+0000 7faf52de2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c15c7d0 con 0x7faf4c0cb220 2024-09-13T17:58:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.368+0000 7faf50b7e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf4c0c9f80 0x7faf4c0975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.368+0000 7faf50b7e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf4c0c9f80 0x7faf4c0975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34062/0 (socket says 172.21.15.111:34062) 2024-09-13T17:58:27.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.368+0000 7faf50b7e640 1 -- 172.21.15.111:0/397724711 learned_addr learned my addr 172.21.15.111:0/397724711 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:27.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.368+0000 7faf5137f640 1 --2- 172.21.15.111:0/397724711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf4c15a2e0 0x7faf4c1f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.368+0000 7faf43fff640 1 --2- 172.21.15.111:0/397724711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf4c0cb220 0x7faf4c1f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.368+0000 7faf50b7e640 1 -- 172.21.15.111:0/397724711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf4c0cb220 msgr2=0x7faf4c1f3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.368+0000 7faf50b7e640 1 --2- 172.21.15.111:0/397724711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf4c0cb220 0x7faf4c1f3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.368+0000 7faf50b7e640 1 -- 172.21.15.111:0/397724711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf4c15a2e0 msgr2=0x7faf4c1f3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.368+0000 7faf50b7e640 1 --2- 172.21.15.111:0/397724711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf4c15a2e0 0x7faf4c1f3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.368+0000 7faf50b7e640 1 -- 172.21.15.111:0/397724711 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c009400 con 0x7faf4c0c9f80 2024-09-13T17:58:27.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.369+0000 7faf50b7e640 1 --2- 172.21.15.111:0/397724711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf4c0c9f80 0x7faf4c0975f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7faf4800d7d0 tx=0x7faf4800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:27.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.369+0000 7faf417fa640 1 -- 172.21.15.111:0/397724711 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48004030 con 0x7faf4c0c9f80 2024-09-13T17:58:27.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.369+0000 7faf417fa640 1 -- 172.21.15.111:0/397724711 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf48004190 con 0x7faf4c0c9f80 2024-09-13T17:58:27.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.369+0000 7faf52de2640 1 -- 172.21.15.111:0/397724711 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c15cab0 con 0x7faf4c0c9f80 2024-09-13T17:58:27.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.369+0000 7faf417fa640 1 -- 172.21.15.111:0/397724711 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48004520 con 0x7faf4c0c9f80 2024-09-13T17:58:27.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.369+0000 7faf52de2640 1 -- 172.21.15.111:0/397724711 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c15d070 con 0x7faf4c0c9f80 2024-09-13T17:58:27.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.370+0000 7faf52de2640 1 -- 172.21.15.111:0/397724711 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4c1e2910 con 0x7faf4c0c9f80 2024-09-13T17:58:27.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.370+0000 7faf417fa640 1 -- 172.21.15.111:0/397724711 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7faf4800b6c0 con 0x7faf4c0c9f80 2024-09-13T17:58:27.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.371+0000 7faf417fa640 1 --2- 172.21.15.111:0/397724711 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7faf3806ac00 0x7faf3806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.371+0000 7faf417fa640 1 -- 172.21.15.111:0/397724711 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7faf480897b0 con 0x7faf4c0c9f80 2024-09-13T17:58:27.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.371+0000 7faf43fff640 1 --2- 172.21.15.111:0/397724711 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7faf3806ac00 0x7faf3806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.371+0000 7faf43fff640 1 --2- 172.21.15.111:0/397724711 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7faf3806ac00 0x7faf3806d0c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faf3c034000 tx=0x7faf3c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:27.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.375+0000 7faf417fa640 1 -- 172.21.15.111:0/397724711 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7faf48054600 con 0x7faf4c0c9f80 2024-09-13T17:58:27.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.536+0000 7faf52de2640 1 -- 172.21.15.111:0/397724711 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf4c0c9630 con 0x7faf4c0c9f80 2024-09-13T17:58:27.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.537+0000 7faf417fa640 1 -- 172.21.15.111:0/397724711 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18794 (secure 0 0 0) 0x7faf48057c20 con 0x7faf4c0c9f80 2024-09-13T17:58:27.539 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:27.540 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":26,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:07.749113+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:27.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.539+0000 7faf52de2640 1 -- 172.21.15.111:0/397724711 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7faf3806ac00 msgr2=0x7faf3806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.539+0000 7faf52de2640 1 --2- 172.21.15.111:0/397724711 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7faf3806ac00 0x7faf3806d0c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faf3c034000 tx=0x7faf3c02e000 comp rx=0 tx=0).stop 2024-09-13T17:58:27.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.539+0000 7faf52de2640 1 -- 172.21.15.111:0/397724711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf4c0c9f80 msgr2=0x7faf4c0975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.539+0000 7faf52de2640 1 --2- 172.21.15.111:0/397724711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf4c0c9f80 0x7faf4c0975f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7faf4800d7d0 tx=0x7faf4800dc90 comp rx=0 tx=0).stop 2024-09-13T17:58:27.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.540+0000 7faf52de2640 1 -- 172.21.15.111:0/397724711 shutdown_connections 2024-09-13T17:58:27.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.540+0000 7faf52de2640 1 --2- 172.21.15.111:0/397724711 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7faf3806ac00 0x7faf3806d0c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.540+0000 7faf52de2640 1 --2- 172.21.15.111:0/397724711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf4c15a2e0 0x7faf4c1f3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.540+0000 7faf52de2640 1 --2- 172.21.15.111:0/397724711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf4c0cb220 0x7faf4c1f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.540+0000 7faf52de2640 1 --2- 172.21.15.111:0/397724711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf4c0c9f80 0x7faf4c0975f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.540+0000 7faf52de2640 1 -- 172.21.15.111:0/397724711 >> 172.21.15.111:0/397724711 conn(0x7faf4c069ed0 msgr2=0x7faf4c0713e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:27.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.540+0000 7faf52de2640 1 -- 172.21.15.111:0/397724711 shutdown_connections 2024-09-13T17:58:27.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.540+0000 7faf52de2640 1 -- 172.21.15.111:0/397724711 wait complete. 2024-09-13T17:58:27.553 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:58:27.553 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:58:27.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.652+0000 7fc18f71f640 1 Processor -- start 2024-09-13T17:58:27.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.652+0000 7fc18f71f640 1 -- start start 2024-09-13T17:58:27.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.653+0000 7fc18f71f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc188159310 0x7fc1880ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.653+0000 7fc18f71f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc1880b2b40 0x7fc1880ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.653+0000 7fc18f71f640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1880ccbe0 con 0x7fc1880ab680 2024-09-13T17:58:27.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.653+0000 7fc18f71f640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1880ad940 con 0x7fc1880b2b40 2024-09-13T17:58:27.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.653+0000 7fc18f71f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1880b1810 con 0x7fc188159310 2024-09-13T17:58:27.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.653+0000 7fc18ccba640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc1880b2b40 0x7fc1880ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.653+0000 7fc18dcbc640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc1880ab680 0x7fc1880ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:55982/0 (socket says 172.21.15.111:55982) 2024-09-13T17:58:27.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.653+0000 7fc18ccba640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc1880b2b40 0x7fc1880ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34078/0 (socket says 172.21.15.111:34078) 2024-09-13T17:58:27.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.653+0000 7fc18dcbc640 1 -- 172.21.15.111:0/281208714 learned_addr learned my addr 172.21.15.111:0/281208714 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:27.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.653+0000 7fc18d4bb640 1 --2- 172.21.15.111:0/281208714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc188159310 0x7fc1880ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.654+0000 7fc18ccba640 1 -- 172.21.15.111:0/281208714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc188159310 msgr2=0x7fc1880ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:27.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.654+0000 7fc18ccba640 1 --2- 172.21.15.111:0/281208714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc188159310 0x7fc1880ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.654+0000 7fc18ccba640 1 -- 172.21.15.111:0/281208714 >> v1:172.21.15.111:6789/0 conn(0x7fc1880ab680 legacy=0x7fc1880ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.654+0000 7fc18ccba640 1 -- 172.21.15.111:0/281208714 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18806d9e0 con 0x7fc1880b2b40 2024-09-13T17:58:27.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.654+0000 7fc17ffff640 1 -- 172.21.15.111:0/281208714 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253339552 0 0) 0x7fc1880ccbe0 con 0x7fc1880ab680 2024-09-13T17:58:27.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.654+0000 7fc18ccba640 1 --2- 172.21.15.111:0/281208714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1880b2b40 0x7fc1880ab140 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc1780097a0 tx=0x7fc17802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec6fe7ab9bee1c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:27.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.654+0000 7fc17ffff640 1 -- 172.21.15.111:0/281208714 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17802ae50 con 0x7fc1880b2b40 2024-09-13T17:58:27.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.654+0000 7fc17ffff640 1 -- 172.21.15.111:0/281208714 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc178032440 con 0x7fc1880b2b40 2024-09-13T17:58:27.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.654+0000 7fc17ffff640 1 -- 172.21.15.111:0/281208714 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1780327d0 con 0x7fc1880b2b40 2024-09-13T17:58:27.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.655+0000 7fc18f71f640 1 -- 172.21.15.111:0/281208714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1880b2b40 msgr2=0x7fc1880ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.655+0000 7fc18f71f640 1 --2- 172.21.15.111:0/281208714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1880b2b40 0x7fc1880ab140 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc1780097a0 tx=0x7fc17802a100 comp rx=0 tx=0).stop 2024-09-13T17:58:27.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.655+0000 7fc18f71f640 1 -- 172.21.15.111:0/281208714 shutdown_connections 2024-09-13T17:58:27.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.655+0000 7fc18f71f640 1 --2- 172.21.15.111:0/281208714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1880b2b40 0x7fc1880ab140 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.655+0000 7fc18f71f640 1 --2- 172.21.15.111:0/281208714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc188159310 0x7fc1880ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.655+0000 7fc18f71f640 1 -- 172.21.15.111:0/281208714 >> 172.21.15.111:0/281208714 conn(0x7fc18806a0b0 msgr2=0x7fc18806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:27.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.655+0000 7fc18f71f640 1 -- 172.21.15.111:0/281208714 shutdown_connections 2024-09-13T17:58:27.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.655+0000 7fc18f71f640 1 -- 172.21.15.111:0/281208714 wait complete. 2024-09-13T17:58:27.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.656+0000 7fc18f71f640 1 Processor -- start 2024-09-13T17:58:27.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.656+0000 7fc18f71f640 1 -- start start 2024-09-13T17:58:27.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18f71f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1880ab680 0x7fc1880c4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18f71f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1880b2b40 0x7fc188156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18f71f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc188159310 0x7fc1880b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18f71f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1881e4520 con 0x7fc1880ab680 2024-09-13T17:58:27.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18f71f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1880c4eb0 con 0x7fc1880b2b40 2024-09-13T17:58:27.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18f71f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18809bad0 con 0x7fc188159310 2024-09-13T17:58:27.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18d4bb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1880ab680 0x7fc1880c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18d4bb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1880ab680 0x7fc1880c4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34090/0 (socket says 172.21.15.111:34090) 2024-09-13T17:58:27.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18d4bb640 1 -- 172.21.15.111:0/2936783628 learned_addr learned my addr 172.21.15.111:0/2936783628 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:27.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18ccba640 1 --2- 172.21.15.111:0/2936783628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1880b2b40 0x7fc188156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18dcbc640 1 --2- 172.21.15.111:0/2936783628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc188159310 0x7fc1880b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18d4bb640 1 -- 172.21.15.111:0/2936783628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc188159310 msgr2=0x7fc1880b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18d4bb640 1 --2- 172.21.15.111:0/2936783628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc188159310 0x7fc1880b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18d4bb640 1 -- 172.21.15.111:0/2936783628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1880b2b40 msgr2=0x7fc188156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18d4bb640 1 --2- 172.21.15.111:0/2936783628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1880b2b40 0x7fc188156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18d4bb640 1 -- 172.21.15.111:0/2936783628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc178009400 con 0x7fc1880ab680 2024-09-13T17:58:27.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18ccba640 1 --2- 172.21.15.111:0/2936783628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1880b2b40 0x7fc188156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:27.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.657+0000 7fc18dcbc640 1 --2- 172.21.15.111:0/2936783628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc188159310 0x7fc1880b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:27.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.658+0000 7fc18d4bb640 1 --2- 172.21.15.111:0/2936783628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1880ab680 0x7fc1880c4970 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc18400e870 tx=0x7fc18400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:27.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.658+0000 7fc17dffb640 1 -- 172.21.15.111:0/2936783628 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1840094f0 con 0x7fc1880ab680 2024-09-13T17:58:27.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.658+0000 7fc17dffb640 1 -- 172.21.15.111:0/2936783628 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18400c940 con 0x7fc1880ab680 2024-09-13T17:58:27.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.658+0000 7fc18f71f640 1 -- 172.21.15.111:0/2936783628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18809b1e0 con 0x7fc1880ab680 2024-09-13T17:58:27.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.658+0000 7fc17dffb640 1 -- 172.21.15.111:0/2936783628 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18400ccf0 con 0x7fc1880ab680 2024-09-13T17:58:27.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.658+0000 7fc18f71f640 1 -- 172.21.15.111:0/2936783628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18809b760 con 0x7fc1880ab680 2024-09-13T17:58:27.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.659+0000 7fc18f71f640 1 -- 172.21.15.111:0/2936783628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1880ab140 con 0x7fc1880ab680 2024-09-13T17:58:27.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.660+0000 7fc17dffb640 1 -- 172.21.15.111:0/2936783628 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc184013470 con 0x7fc1880ab680 2024-09-13T17:58:27.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.660+0000 7fc17dffb640 1 --2- 172.21.15.111:0/2936783628 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc16c06ac30 0x7fc16c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.660+0000 7fc17dffb640 1 -- 172.21.15.111:0/2936783628 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc184089b80 con 0x7fc1880ab680 2024-09-13T17:58:27.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.660+0000 7fc18ccba640 1 --2- 172.21.15.111:0/2936783628 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc16c06ac30 0x7fc16c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.660+0000 7fc18ccba640 1 --2- 172.21.15.111:0/2936783628 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc16c06ac30 0x7fc16c06d0f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc178034000 tx=0x7fc17802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:27.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.664+0000 7fc17dffb640 1 -- 172.21.15.111:0/2936783628 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc1840549a0 con 0x7fc1880ab680 2024-09-13T17:58:27.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.825+0000 7fc18f71f640 1 -- 172.21.15.111:0/2936783628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc18806e4b0 con 0x7fc1880ab680 2024-09-13T17:58:27.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.826+0000 7fc17dffb640 1 -- 172.21.15.111:0/2936783628 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc1840170a0 con 0x7fc1880ab680 2024-09-13T17:58:27.828 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:58:27.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.828+0000 7fc18f71f640 1 -- 172.21.15.111:0/2936783628 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc16c06ac30 msgr2=0x7fc16c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.828+0000 7fc18f71f640 1 --2- 172.21.15.111:0/2936783628 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc16c06ac30 0x7fc16c06d0f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc178034000 tx=0x7fc17802e000 comp rx=0 tx=0).stop 2024-09-13T17:58:27.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.828+0000 7fc18f71f640 1 -- 172.21.15.111:0/2936783628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1880ab680 msgr2=0x7fc1880c4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.828+0000 7fc18f71f640 1 --2- 172.21.15.111:0/2936783628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1880ab680 0x7fc1880c4970 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc18400e870 tx=0x7fc18400ed30 comp rx=0 tx=0).stop 2024-09-13T17:58:27.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.828+0000 7fc18f71f640 1 -- 172.21.15.111:0/2936783628 shutdown_connections 2024-09-13T17:58:27.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.828+0000 7fc18f71f640 1 --2- 172.21.15.111:0/2936783628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc188159310 0x7fc1880b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.828+0000 7fc18f71f640 1 --2- 172.21.15.111:0/2936783628 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc16c06ac30 0x7fc16c06d0f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.828+0000 7fc18f71f640 1 --2- 172.21.15.111:0/2936783628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1880b2b40 0x7fc188156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.828+0000 7fc18f71f640 1 --2- 172.21.15.111:0/2936783628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1880ab680 0x7fc1880c4970 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.828+0000 7fc18f71f640 1 -- 172.21.15.111:0/2936783628 >> 172.21.15.111:0/2936783628 conn(0x7fc18806a0b0 msgr2=0x7fc188070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:27.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.828+0000 7fc18f71f640 1 -- 172.21.15.111:0/2936783628 shutdown_connections 2024-09-13T17:58:27.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.828+0000 7fc18f71f640 1 -- 172.21.15.111:0/2936783628 wait complete. 2024-09-13T17:58:27.841 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:58:27.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.940+0000 7f9ed8165640 1 Processor -- start 2024-09-13T17:58:27.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.940+0000 7f9ed8165640 1 -- start start 2024-09-13T17:58:27.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.940+0000 7f9ed8165640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed00b2b40 0x7f9ed00ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.940+0000 7f9ed8165640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed00ccbe0 con 0x7f9ed0159310 2024-09-13T17:58:27.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.940+0000 7f9ed8165640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed00b1810 con 0x7f9ed00ce4f0 2024-09-13T17:58:27.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.940+0000 7f9ed8165640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed00ad940 con 0x7f9ed00b2b40 2024-09-13T17:58:27.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.941+0000 7f9ed5f01640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f9ed0159310 0x7f9ed00ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:55994/0 (socket says 172.21.15.111:55994) 2024-09-13T17:58:27.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.941+0000 7f9ed5f01640 1 -- 172.21.15.111:0/3926886884 learned_addr learned my addr 172.21.15.111:0/3926886884 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:27.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.941+0000 7f9ed5700640 1 --2- 172.21.15.111:0/3926886884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed00b2b40 0x7f9ed00ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.941+0000 7f9ed4eff640 1 -- 172.21.15.111:0/3926886884 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175173747 0 0) 0x7f9ed00ccbe0 con 0x7f9ed0159310 2024-09-13T17:58:27.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.941+0000 7f9ed4eff640 1 -- 172.21.15.111:0/3926886884 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ebc003520 con 0x7f9ed0159310 2024-09-13T17:58:27.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.941+0000 7f9ed4eff640 1 -- 172.21.15.111:0/3926886884 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1430210861 0 0) 0x7f9ed00b1810 con 0x7f9ed00ce4f0 2024-09-13T17:58:27.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.941+0000 7f9ed4eff640 1 -- 172.21.15.111:0/3926886884 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ed00ccbe0 con 0x7f9ed00ce4f0 2024-09-13T17:58:27.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.941+0000 7f9ed5700640 1 -- 172.21.15.111:0/3926886884 >> v1:172.21.15.188:6789/0 conn(0x7f9ed00ce4f0 legacy=0x7f9ed00b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.941+0000 7f9ed5700640 1 -- 172.21.15.111:0/3926886884 >> v1:172.21.15.111:6789/0 conn(0x7f9ed0159310 legacy=0x7f9ed00ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.941+0000 7f9ed5700640 1 -- 172.21.15.111:0/3926886884 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed006d9e0 con 0x7f9ed00b2b40 2024-09-13T17:58:27.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.942+0000 7f9ed5700640 1 --2- 172.21.15.111:0/3926886884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed00b2b40 0x7f9ed00ad400 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00097d0 tx=0x7f9ec002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd5a0980eee7256b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:27.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.942+0000 7f9ed4eff640 1 -- 172.21.15.111:0/3926886884 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec002ae50 con 0x7f9ed00b2b40 2024-09-13T17:58:27.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.942+0000 7f9ed4eff640 1 -- 172.21.15.111:0/3926886884 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec0032440 con 0x7f9ed00b2b40 2024-09-13T17:58:27.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.942+0000 7f9ed4eff640 1 -- 172.21.15.111:0/3926886884 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec00327b0 con 0x7f9ed00b2b40 2024-09-13T17:58:27.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.942+0000 7f9ed8165640 1 -- 172.21.15.111:0/3926886884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed00b2b40 msgr2=0x7f9ed00ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.942+0000 7f9ed8165640 1 --2- 172.21.15.111:0/3926886884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed00b2b40 0x7f9ed00ad400 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00097d0 tx=0x7f9ec002a100 comp rx=0 tx=0).stop 2024-09-13T17:58:27.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.943+0000 7f9ed8165640 1 -- 172.21.15.111:0/3926886884 shutdown_connections 2024-09-13T17:58:27.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.943+0000 7f9ed8165640 1 --2- 172.21.15.111:0/3926886884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed00b2b40 0x7f9ed00ad400 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.943+0000 7f9ed8165640 1 -- 172.21.15.111:0/3926886884 >> 172.21.15.111:0/3926886884 conn(0x7f9ed006a0b0 msgr2=0x7f9ed006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:27.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.943+0000 7f9ed8165640 1 -- 172.21.15.111:0/3926886884 shutdown_connections 2024-09-13T17:58:27.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.943+0000 7f9ed8165640 1 -- 172.21.15.111:0/3926886884 wait complete. 2024-09-13T17:58:27.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.944+0000 7f9ed8165640 1 Processor -- start 2024-09-13T17:58:27.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.944+0000 7f9ed8165640 1 -- start start 2024-09-13T17:58:27.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.944+0000 7f9ed8165640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ed00b2b40 0x7f9ed00c3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.944+0000 7f9ed8165640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed00ce4f0 0x7f9ed00cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.944+0000 7f9ed8165640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed0159310 0x7f9ed00cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.944+0000 7f9ed8165640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed01efdd0 con 0x7f9ed0159310 2024-09-13T17:58:27.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.944+0000 7f9ed8165640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed00c61d0 con 0x7f9ed00b2b40 2024-09-13T17:58:27.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.944+0000 7f9ed8165640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed00c5ef0 con 0x7f9ed00ce4f0 2024-09-13T17:58:27.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.945+0000 7f9ed6702640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed0159310 0x7f9ed00cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.945+0000 7f9ed6702640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed0159310 0x7f9ed00cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34096/0 (socket says 172.21.15.111:34096) 2024-09-13T17:58:27.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.945+0000 7f9ed6702640 1 -- 172.21.15.111:0/1973607950 learned_addr learned my addr 172.21.15.111:0/1973607950 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:27.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.945+0000 7f9ed5f01640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ed00b2b40 0x7f9ed00c3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.945+0000 7f9ed5700640 1 --2- 172.21.15.111:0/1973607950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed00ce4f0 0x7f9ed00cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.945+0000 7f9ed6702640 1 -- 172.21.15.111:0/1973607950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed00ce4f0 msgr2=0x7f9ed00cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.945+0000 7f9ed6702640 1 --2- 172.21.15.111:0/1973607950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed00ce4f0 0x7f9ed00cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.945+0000 7f9ed6702640 1 -- 172.21.15.111:0/1973607950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ed00b2b40 msgr2=0x7f9ed00c3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:27.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.945+0000 7f9ed6702640 1 --2- 172.21.15.111:0/1973607950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ed00b2b40 0x7f9ed00c3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:27.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.945+0000 7f9ed6702640 1 -- 172.21.15.111:0/1973607950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec0009400 con 0x7f9ed0159310 2024-09-13T17:58:27.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.945+0000 7f9ed6702640 1 --2- 172.21.15.111:0/1973607950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed0159310 0x7f9ed00cb950 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800c790 tx=0x7f9ec800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:27.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.945+0000 7f9ec67fc640 1 -- 172.21.15.111:0/1973607950 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec800cea0 con 0x7f9ed0159310 2024-09-13T17:58:27.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.945+0000 7f9ec67fc640 1 -- 172.21.15.111:0/1973607950 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec8003b70 con 0x7f9ed0159310 2024-09-13T17:58:27.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.945+0000 7f9ed8165640 1 -- 172.21.15.111:0/1973607950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed00c65f0 con 0x7f9ed0159310 2024-09-13T17:58:27.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.945+0000 7f9ec67fc640 1 -- 172.21.15.111:0/1973607950 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec800a850 con 0x7f9ed0159310 2024-09-13T17:58:27.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.945+0000 7f9ed8165640 1 -- 172.21.15.111:0/1973607950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed00c6bb0 con 0x7f9ed0159310 2024-09-13T17:58:27.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.946+0000 7f9ed8165640 1 -- 172.21.15.111:0/1973607950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed00ad400 con 0x7f9ed0159310 2024-09-13T17:58:27.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.947+0000 7f9ec67fc640 1 -- 172.21.15.111:0/1973607950 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9ec800a9b0 con 0x7f9ed0159310 2024-09-13T17:58:27.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.947+0000 7f9ec67fc640 1 --2- 172.21.15.111:0/1973607950 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9eb406ac00 0x7f9eb406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:27.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.947+0000 7f9ed5f01640 1 --2- 172.21.15.111:0/1973607950 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9eb406ac00 0x7f9eb406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:27.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.947+0000 7f9ec67fc640 1 -- 172.21.15.111:0/1973607950 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9ec8089410 con 0x7f9ed0159310 2024-09-13T17:58:27.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.948+0000 7f9ed5f01640 1 --2- 172.21.15.111:0/1973607950 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9eb406ac00 0x7f9eb406d0c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc000f60 tx=0x7f9ecc007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:27.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:27.951+0000 7f9ec67fc640 1 -- 172.21.15.111:0/1973607950 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f9ec8054370 con 0x7f9ed0159310 2024-09-13T17:58:28.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.112+0000 7f9ed8165640 1 -- 172.21.15.111:0/1973607950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9ed006e4b0 con 0x7f9ed0159310 2024-09-13T17:58:28.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.112+0000 7f9ec67fc640 1 -- 172.21.15.111:0/1973607950 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9ec8057990 con 0x7f9ed0159310 2024-09-13T17:58:28.114 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:28.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.114+0000 7f9ed8165640 1 -- 172.21.15.111:0/1973607950 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9eb406ac00 msgr2=0x7f9eb406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.114+0000 7f9ed8165640 1 --2- 172.21.15.111:0/1973607950 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9eb406ac00 0x7f9eb406d0c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc000f60 tx=0x7f9ecc007d30 comp rx=0 tx=0).stop 2024-09-13T17:58:28.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.114+0000 7f9ed8165640 1 -- 172.21.15.111:0/1973607950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed0159310 msgr2=0x7f9ed00cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.114+0000 7f9ed8165640 1 --2- 172.21.15.111:0/1973607950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed0159310 0x7f9ed00cb950 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800c790 tx=0x7f9ec800cc50 comp rx=0 tx=0).stop 2024-09-13T17:58:28.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.114+0000 7f9ed8165640 1 -- 172.21.15.111:0/1973607950 shutdown_connections 2024-09-13T17:58:28.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.114+0000 7f9ed8165640 1 --2- 172.21.15.111:0/1973607950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed0159310 0x7f9ed00cb950 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.114+0000 7f9ed8165640 1 --2- 172.21.15.111:0/1973607950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed00ce4f0 0x7f9ed00cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.114+0000 7f9ed8165640 1 --2- 172.21.15.111:0/1973607950 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9eb406ac00 0x7f9eb406d0c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.114+0000 7f9ed8165640 1 --2- 172.21.15.111:0/1973607950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ed00b2b40 0x7f9ed00c3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.114+0000 7f9ed8165640 1 -- 172.21.15.111:0/1973607950 >> 172.21.15.111:0/1973607950 conn(0x7f9ed006a0b0 msgr2=0x7f9ed00ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:28.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.114+0000 7f9ed8165640 1 -- 172.21.15.111:0/1973607950 shutdown_connections 2024-09-13T17:58:28.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.114+0000 7f9ed8165640 1 -- 172.21.15.111:0/1973607950 wait complete. 2024-09-13T17:58:28.127 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:58:28.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.226+0000 7f432a3b5640 1 Processor -- start 2024-09-13T17:58:28.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.226+0000 7f432a3b5640 1 -- start start 2024-09-13T17:58:28.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.226+0000 7f432a3b5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f432415c500 0x7f43240d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.226+0000 7f432a3b5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f43240d2590 0x7f43240cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.226+0000 7f432a3b5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43240cd450 con 0x7f43240cde20 2024-09-13T17:58:28.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.226+0000 7f432a3b5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43240d29f0 con 0x7f43240d2590 2024-09-13T17:58:28.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.226+0000 7f432a3b5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43240d0cf0 con 0x7f432415c500 2024-09-13T17:58:28.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.227+0000 7f43237fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f43240d2590 0x7f43240cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.227+0000 7f43237fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f43240d2590 0x7f43240cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34106/0 (socket says 172.21.15.111:34106) 2024-09-13T17:58:28.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.227+0000 7f43237fe640 1 -- 172.21.15.111:0/2084714214 learned_addr learned my addr 172.21.15.111:0/2084714214 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:28.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.227+0000 7f4323fff640 1 --2- 172.21.15.111:0/2084714214 >> v2:172.21.15.188:3300/0 conn(0x7f432415c500 0x7f43240d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.227+0000 7f43237fe640 1 -- 172.21.15.111:0/2084714214 >> v2:172.21.15.188:3300/0 conn(0x7f432415c500 msgr2=0x7f43240d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:28.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.227+0000 7f43237fe640 1 --2- 172.21.15.111:0/2084714214 >> v2:172.21.15.188:3300/0 conn(0x7f432415c500 0x7f43240d2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.227+0000 7f43237fe640 1 -- 172.21.15.111:0/2084714214 >> v1:172.21.15.188:6789/0 conn(0x7f43240cde20 legacy=0x7f43240d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.227+0000 7f43237fe640 1 -- 172.21.15.111:0/2084714214 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43240d0fd0 con 0x7f43240d2590 2024-09-13T17:58:28.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.228+0000 7f43237fe640 1 --2- 172.21.15.111:0/2084714214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43240d2590 0x7f43240cd8e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f43240cfec0 tx=0x7f431802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95552837be693ee server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:28.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.228+0000 7f4322ffd640 1 -- 172.21.15.111:0/2084714214 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318032440 con 0x7f43240d2590 2024-09-13T17:58:28.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.228+0000 7f4322ffd640 1 -- 172.21.15.111:0/2084714214 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43180325a0 con 0x7f43240d2590 2024-09-13T17:58:28.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.228+0000 7f4322ffd640 1 -- 172.21.15.111:0/2084714214 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318032930 con 0x7f43240d2590 2024-09-13T17:58:28.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.228+0000 7f432a3b5640 1 -- 172.21.15.111:0/2084714214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43240d2590 msgr2=0x7f43240cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.228+0000 7f432a3b5640 1 --2- 172.21.15.111:0/2084714214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43240d2590 0x7f43240cd8e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f43240cfec0 tx=0x7f431802a270 comp rx=0 tx=0).stop 2024-09-13T17:58:28.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.228+0000 7f432a3b5640 1 -- 172.21.15.111:0/2084714214 shutdown_connections 2024-09-13T17:58:28.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.228+0000 7f432a3b5640 1 --2- 172.21.15.111:0/2084714214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43240d2590 0x7f43240cd8e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.228+0000 7f432a3b5640 1 --2- 172.21.15.111:0/2084714214 >> v2:172.21.15.188:3300/0 conn(0x7f432415c500 0x7f43240d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.228+0000 7f432a3b5640 1 -- 172.21.15.111:0/2084714214 >> 172.21.15.111:0/2084714214 conn(0x7f4324089f60 msgr2=0x7f432409b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:28.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.228+0000 7f432a3b5640 1 -- 172.21.15.111:0/2084714214 shutdown_connections 2024-09-13T17:58:28.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.228+0000 7f432a3b5640 1 -- 172.21.15.111:0/2084714214 wait complete. 2024-09-13T17:58:28.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.229+0000 7f432a3b5640 1 Processor -- start 2024-09-13T17:58:28.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.230+0000 7f432a3b5640 1 -- start start 2024-09-13T17:58:28.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.230+0000 7f432a3b5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43240cde20 0x7f432415f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.230+0000 7f432a3b5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43240d2590 0x7f43241e7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.230+0000 7f432a3b5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f432415c500 0x7f432415ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.230+0000 7f432a3b5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43240b7000 con 0x7f432415c500 2024-09-13T17:58:28.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.230+0000 7f432a3b5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43241e7b40 con 0x7f43240cde20 2024-09-13T17:58:28.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.230+0000 7f432a3b5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43241e7d30 con 0x7f43240d2590 2024-09-13T17:58:28.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.230+0000 7f4328952640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f432415c500 0x7f432415ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.231+0000 7f4323fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43240cde20 0x7f432415f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.231+0000 7f4328952640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f432415c500 0x7f432415ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34112/0 (socket says 172.21.15.111:34112) 2024-09-13T17:58:28.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.231+0000 7f4328952640 1 -- 172.21.15.111:0/3861137446 learned_addr learned my addr 172.21.15.111:0/3861137446 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:28.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.231+0000 7f43237fe640 1 --2- 172.21.15.111:0/3861137446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43240d2590 0x7f43241e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.231+0000 7f4328952640 1 -- 172.21.15.111:0/3861137446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43240d2590 msgr2=0x7f43241e7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.231+0000 7f4328952640 1 --2- 172.21.15.111:0/3861137446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43240d2590 0x7f43241e7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.231+0000 7f4328952640 1 -- 172.21.15.111:0/3861137446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43240cde20 msgr2=0x7f432415f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.231+0000 7f4328952640 1 --2- 172.21.15.111:0/3861137446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43240cde20 0x7f432415f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.231+0000 7f4328952640 1 -- 172.21.15.111:0/3861137446 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43180094d0 con 0x7f432415c500 2024-09-13T17:58:28.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.231+0000 7f4328952640 1 --2- 172.21.15.111:0/3861137446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f432415c500 0x7f432415ca60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f431c00c7a0 tx=0x7f431c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:28.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.231+0000 7f4320ff9640 1 -- 172.21.15.111:0/3861137446 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c003b70 con 0x7f432415c500 2024-09-13T17:58:28.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.231+0000 7f4320ff9640 1 -- 172.21.15.111:0/3861137446 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c003cf0 con 0x7f432415c500 2024-09-13T17:58:28.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.231+0000 7f4320ff9640 1 -- 172.21.15.111:0/3861137446 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c003ea0 con 0x7f432415c500 2024-09-13T17:58:28.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.231+0000 7f432a3b5640 1 -- 172.21.15.111:0/3861137446 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43241e8010 con 0x7f432415c500 2024-09-13T17:58:28.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.231+0000 7f432a3b5640 1 -- 172.21.15.111:0/3861137446 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43240b5540 con 0x7f432415c500 2024-09-13T17:58:28.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.232+0000 7f432a3b5640 1 -- 172.21.15.111:0/3861137446 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43240cd8e0 con 0x7f432415c500 2024-09-13T17:58:28.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.233+0000 7f4320ff9640 1 -- 172.21.15.111:0/3861137446 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f431c00aa90 con 0x7f432415c500 2024-09-13T17:58:28.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.233+0000 7f4320ff9640 1 --2- 172.21.15.111:0/3861137446 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f431006ac00 0x7f431006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.233+0000 7f4323fff640 1 --2- 172.21.15.111:0/3861137446 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f431006ac00 0x7f431006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.233+0000 7f4320ff9640 1 -- 172.21.15.111:0/3861137446 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f431c089440 con 0x7f432415c500 2024-09-13T17:58:28.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.234+0000 7f4323fff640 1 --2- 172.21.15.111:0/3861137446 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f431006ac00 0x7f431006d0c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4314009aa0 tx=0x7f4314009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:28.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.237+0000 7f4320ff9640 1 -- 172.21.15.111:0/3861137446 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f431c054310 con 0x7f432415c500 2024-09-13T17:58:28.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.400+0000 7f432a3b5640 1 -- 172.21.15.111:0/3861137446 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f432406b160 con 0x7f432415c500 2024-09-13T17:58:28.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.400+0000 7f4320ff9640 1 -- 172.21.15.111:0/3861137446 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f431c057930 con 0x7f432415c500 2024-09-13T17:58:28.402 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:58:28.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.402+0000 7f432a3b5640 1 -- 172.21.15.111:0/3861137446 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f431006ac00 msgr2=0x7f431006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.402+0000 7f432a3b5640 1 --2- 172.21.15.111:0/3861137446 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f431006ac00 0x7f431006d0c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4314009aa0 tx=0x7f4314009190 comp rx=0 tx=0).stop 2024-09-13T17:58:28.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.402+0000 7f432a3b5640 1 -- 172.21.15.111:0/3861137446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f432415c500 msgr2=0x7f432415ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.402+0000 7f432a3b5640 1 --2- 172.21.15.111:0/3861137446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f432415c500 0x7f432415ca60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f431c00c7a0 tx=0x7f431c00cc60 comp rx=0 tx=0).stop 2024-09-13T17:58:28.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.402+0000 7f432a3b5640 1 -- 172.21.15.111:0/3861137446 shutdown_connections 2024-09-13T17:58:28.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.402+0000 7f432a3b5640 1 --2- 172.21.15.111:0/3861137446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f432415c500 0x7f432415ca60 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.402+0000 7f432a3b5640 1 --2- 172.21.15.111:0/3861137446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43240d2590 0x7f43241e7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.402+0000 7f432a3b5640 1 --2- 172.21.15.111:0/3861137446 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f431006ac00 0x7f431006d0c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.402+0000 7f432a3b5640 1 --2- 172.21.15.111:0/3861137446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43240cde20 0x7f432415f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.402+0000 7f432a3b5640 1 -- 172.21.15.111:0/3861137446 >> 172.21.15.111:0/3861137446 conn(0x7f4324089f60 msgr2=0x7f43240b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:28.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.403+0000 7f432a3b5640 1 -- 172.21.15.111:0/3861137446 shutdown_connections 2024-09-13T17:58:28.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.403+0000 7f432a3b5640 1 -- 172.21.15.111:0/3861137446 wait complete. 2024-09-13T17:58:28.415 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:58:28.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.514+0000 7f2a5cd1f640 1 Processor -- start 2024-09-13T17:58:28.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.514+0000 7f2a5cd1f640 1 -- start start 2024-09-13T17:58:28.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.515+0000 7f2a5cd1f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a5815fc50 0x7f2a580d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.515+0000 7f2a5cd1f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a580cdb30 0x7f2a580b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.515+0000 7f2a5cd1f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a580d1900 con 0x7f2a580b3060 2024-09-13T17:58:28.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.515+0000 7f2a5cd1f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a580d2380 con 0x7f2a580cdb30 2024-09-13T17:58:28.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.515+0000 7f2a5cd1f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580b5f70 con 0x7f2a5815fc50 2024-09-13T17:58:28.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.516+0000 7f2a56d9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a580cdb30 0x7f2a580b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.516+0000 7f2a5659c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a5815fc50 0x7f2a580d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.516+0000 7f2a56d9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a580cdb30 0x7f2a580b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:42130/0 (socket says 172.21.15.111:42130) 2024-09-13T17:58:28.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.516+0000 7f2a55d9b640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f2a580b3060 0x7f2a580b5320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:51472/0 (socket says 172.21.15.111:51472) 2024-09-13T17:58:28.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.516+0000 7f2a5659c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a5815fc50 0x7f2a580d13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:40742/0 (socket says 172.21.15.111:40742) 2024-09-13T17:58:28.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.516+0000 7f2a56d9d640 1 -- 172.21.15.111:0/3196961398 learned_addr learned my addr 172.21.15.111:0/3196961398 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:28.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.516+0000 7f2a56d9d640 1 -- 172.21.15.111:0/3196961398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a5815fc50 msgr2=0x7f2a580d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:28.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.516+0000 7f2a56d9d640 1 --2- 172.21.15.111:0/3196961398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a5815fc50 0x7f2a580d13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.516+0000 7f2a56d9d640 1 -- 172.21.15.111:0/3196961398 >> v1:172.21.15.188:6789/0 conn(0x7f2a580b3060 legacy=0x7f2a580b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.516+0000 7f2a5559a640 1 -- 172.21.15.111:0/3196961398 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575645665 0 0) 0x7f2a4c001110 con 0x7f2a580b3060 2024-09-13T17:58:28.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.516+0000 7f2a56d9d640 1 -- 172.21.15.111:0/3196961398 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a580d2be0 con 0x7f2a580cdb30 2024-09-13T17:58:28.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.517+0000 7f2a56d9d640 1 --2- 172.21.15.111:0/3196961398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a580cdb30 0x7f2a580b5a30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009870 tx=0x7f2a5002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c8c87e38a2a0761 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:28.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.517+0000 7f2a5559a640 1 -- 172.21.15.111:0/3196961398 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50032440 con 0x7f2a580cdb30 2024-09-13T17:58:28.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.517+0000 7f2a5559a640 1 -- 172.21.15.111:0/3196961398 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a500325a0 con 0x7f2a580cdb30 2024-09-13T17:58:28.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.517+0000 7f2a5559a640 1 -- 172.21.15.111:0/3196961398 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50032930 con 0x7f2a580cdb30 2024-09-13T17:58:28.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.517+0000 7f2a5cd1f640 1 -- 172.21.15.111:0/3196961398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a580cdb30 msgr2=0x7f2a580b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.517+0000 7f2a5cd1f640 1 --2- 172.21.15.111:0/3196961398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a580cdb30 0x7f2a580b5a30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009870 tx=0x7f2a5002a270 comp rx=0 tx=0).stop 2024-09-13T17:58:28.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.517+0000 7f2a5cd1f640 1 -- 172.21.15.111:0/3196961398 shutdown_connections 2024-09-13T17:58:28.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.517+0000 7f2a5cd1f640 1 --2- 172.21.15.111:0/3196961398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a580cdb30 0x7f2a580b5a30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.517+0000 7f2a5cd1f640 1 --2- 172.21.15.111:0/3196961398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a5815fc50 0x7f2a580d13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.517+0000 7f2a5cd1f640 1 -- 172.21.15.111:0/3196961398 >> 172.21.15.111:0/3196961398 conn(0x7f2a58089f60 msgr2=0x7f2a580d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:28.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.518+0000 7f2a5cd1f640 1 -- 172.21.15.111:0/3196961398 shutdown_connections 2024-09-13T17:58:28.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.518+0000 7f2a5cd1f640 1 -- 172.21.15.111:0/3196961398 wait complete. 2024-09-13T17:58:28.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.519+0000 7f2a5cd1f640 1 Processor -- start 2024-09-13T17:58:28.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.519+0000 7f2a5cd1f640 1 -- start start 2024-09-13T17:58:28.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.519+0000 7f2a5cd1f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a580b3060 0x7f2a580b0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.519+0000 7f2a5cd1f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a580cdb30 0x7f2a580b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.519+0000 7f2a5cd1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a5815fc50 0x7f2a5809fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.519+0000 7f2a5cd1f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5809ffd0 con 0x7f2a5815fc50 2024-09-13T17:58:28.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.519+0000 7f2a5cd1f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580b1b70 con 0x7f2a580cdb30 2024-09-13T17:58:28.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a5cd1f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580afd70 con 0x7f2a580b3060 2024-09-13T17:58:28.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a56d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a5815fc50 0x7f2a5809fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a55d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a580cdb30 0x7f2a580b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a56d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a5815fc50 0x7f2a5809fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34120/0 (socket says 172.21.15.111:34120) 2024-09-13T17:58:28.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a56d9d640 1 -- 172.21.15.111:0/3965959969 learned_addr learned my addr 172.21.15.111:0/3965959969 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:28.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a5659c640 1 --2- 172.21.15.111:0/3965959969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a580b3060 0x7f2a580b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.527 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a56d9d640 1 -- 172.21.15.111:0/3965959969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a580b3060 msgr2=0x7f2a580b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.527 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a56d9d640 1 --2- 172.21.15.111:0/3965959969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a580b3060 0x7f2a580b0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.527 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a56d9d640 1 -- 172.21.15.111:0/3965959969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a580cdb30 msgr2=0x7f2a580b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.527 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a56d9d640 1 --2- 172.21.15.111:0/3965959969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a580cdb30 0x7f2a580b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a56d9d640 1 -- 172.21.15.111:0/3965959969 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a500094d0 con 0x7f2a5815fc50 2024-09-13T17:58:28.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a55d9b640 1 --2- 172.21.15.111:0/3965959969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a580cdb30 0x7f2a580b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:28.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a56d9d640 1 --2- 172.21.15.111:0/3965959969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a5815fc50 0x7f2a5809fa90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2a500090b0 tx=0x7f2a500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:28.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a46ffd640 1 -- 172.21.15.111:0/3965959969 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50032440 con 0x7f2a5815fc50 2024-09-13T17:58:28.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a46ffd640 1 -- 172.21.15.111:0/3965959969 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a50004030 con 0x7f2a5815fc50 2024-09-13T17:58:28.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a46ffd640 1 -- 172.21.15.111:0/3965959969 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a500041c0 con 0x7f2a5815fc50 2024-09-13T17:58:28.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a5cd1f640 1 -- 172.21.15.111:0/3965959969 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a580b0050 con 0x7f2a5815fc50 2024-09-13T17:58:28.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.520+0000 7f2a5cd1f640 1 -- 172.21.15.111:0/3965959969 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a580afa40 con 0x7f2a5815fc50 2024-09-13T17:58:28.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.521+0000 7f2a5cd1f640 1 -- 172.21.15.111:0/3965959969 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a580b0570 con 0x7f2a5815fc50 2024-09-13T17:58:28.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.522+0000 7f2a46ffd640 1 -- 172.21.15.111:0/3965959969 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2a50039050 con 0x7f2a5815fc50 2024-09-13T17:58:28.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.522+0000 7f2a46ffd640 1 --2- 172.21.15.111:0/3965959969 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f2a3806ac00 0x7f2a3806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.522+0000 7f2a46ffd640 1 -- 172.21.15.111:0/3965959969 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2a500a9c60 con 0x7f2a5815fc50 2024-09-13T17:58:28.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.522+0000 7f2a5659c640 1 --2- 172.21.15.111:0/3965959969 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f2a3806ac00 0x7f2a3806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.523+0000 7f2a5659c640 1 --2- 172.21.15.111:0/3965959969 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f2a3806ac00 0x7f2a3806d0c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2a48011340 tx=0x7f2a4800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:28.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.526+0000 7f2a46ffd640 1 -- 172.21.15.111:0/3965959969 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f2a580b0570 con 0x7f2a5815fc50 2024-09-13T17:58:28.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.687+0000 7f2a5cd1f640 1 -- 172.21.15.111:0/3965959969 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2a580b0570 con 0x7f2a5815fc50 2024-09-13T17:58:28.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.687+0000 7f2a46ffd640 1 -- 172.21.15.111:0/3965959969 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2a580b0570 con 0x7f2a5815fc50 2024-09-13T17:58:28.689 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:28.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.690+0000 7f2a5cd1f640 1 -- 172.21.15.111:0/3965959969 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f2a3806ac00 msgr2=0x7f2a3806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.690+0000 7f2a5cd1f640 1 --2- 172.21.15.111:0/3965959969 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f2a3806ac00 0x7f2a3806d0c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2a48011340 tx=0x7f2a4800b040 comp rx=0 tx=0).stop 2024-09-13T17:58:28.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.690+0000 7f2a5cd1f640 1 -- 172.21.15.111:0/3965959969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a5815fc50 msgr2=0x7f2a5809fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.690+0000 7f2a5cd1f640 1 --2- 172.21.15.111:0/3965959969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a5815fc50 0x7f2a5809fa90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2a500090b0 tx=0x7f2a500046c0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.690+0000 7f2a5cd1f640 1 -- 172.21.15.111:0/3965959969 shutdown_connections 2024-09-13T17:58:28.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.690+0000 7f2a5cd1f640 1 --2- 172.21.15.111:0/3965959969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a5815fc50 0x7f2a5809fa90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.690+0000 7f2a5cd1f640 1 --2- 172.21.15.111:0/3965959969 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f2a3806ac00 0x7f2a3806d0c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.690+0000 7f2a5cd1f640 1 --2- 172.21.15.111:0/3965959969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a580cdb30 0x7f2a580b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.690+0000 7f2a5cd1f640 1 --2- 172.21.15.111:0/3965959969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a580b3060 0x7f2a580b0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.690+0000 7f2a5cd1f640 1 -- 172.21.15.111:0/3965959969 >> 172.21.15.111:0/3965959969 conn(0x7f2a58089f60 msgr2=0x7f2a580b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:28.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.690+0000 7f2a5cd1f640 1 -- 172.21.15.111:0/3965959969 shutdown_connections 2024-09-13T17:58:28.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.690+0000 7f2a5cd1f640 1 -- 172.21.15.111:0/3965959969 wait complete. 2024-09-13T17:58:28.703 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:58:28.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.800+0000 7f46d98e5640 1 Processor -- start 2024-09-13T17:58:28.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.800+0000 7f46d98e5640 1 -- start start 2024-09-13T17:58:28.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.801+0000 7f46d98e5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f46d406b5f0 0x7f46d40abae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.801+0000 7f46d98e5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f46d415c660 0x7f46d40ac020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.801+0000 7f46d98e5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d4066090 con 0x7f46d415c280 2024-09-13T17:58:28.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.801+0000 7f46d98e5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d40ac6f0 con 0x7f46d406b5f0 2024-09-13T17:58:28.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.801+0000 7f46d98e5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d40ac9d0 con 0x7f46d415c660 2024-09-13T17:58:28.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.801+0000 7f46d27fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f46d406b5f0 0x7f46d40abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.801+0000 7f46d2ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f46d415c280 0x7f46d406aee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56008/0 (socket says 172.21.15.111:56008) 2024-09-13T17:58:28.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.801+0000 7f46d2ffd640 1 -- 172.21.15.111:0/1345200368 learned_addr learned my addr 172.21.15.111:0/1345200368 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:28.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.801+0000 7f46d37fe640 1 --2- 172.21.15.111:0/1345200368 >> v2:172.21.15.188:3300/0 conn(0x7f46d415c660 0x7f46d40ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.801+0000 7f46d27fc640 1 -- 172.21.15.111:0/1345200368 >> v2:172.21.15.188:3300/0 conn(0x7f46d415c660 msgr2=0x7f46d40ac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:28.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.801+0000 7f46d27fc640 1 --2- 172.21.15.111:0/1345200368 >> v2:172.21.15.188:3300/0 conn(0x7f46d415c660 0x7f46d40ac020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.801+0000 7f46d27fc640 1 -- 172.21.15.111:0/1345200368 >> v1:172.21.15.111:6789/0 conn(0x7f46d415c280 legacy=0x7f46d406aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.801+0000 7f46d1ffb640 1 -- 172.21.15.111:0/1345200368 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272041894 0 0) 0x7f46d4066090 con 0x7f46d415c280 2024-09-13T17:58:28.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.801+0000 7f46d27fc640 1 -- 172.21.15.111:0/1345200368 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d40accb0 con 0x7f46d406b5f0 2024-09-13T17:58:28.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.801+0000 7f46d37fe640 1 --2- 172.21.15.111:0/1345200368 >> v2:172.21.15.188:3300/0 conn(0x7f46d415c660 0x7f46d40ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:28.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.802+0000 7f46d27fc640 1 --2- 172.21.15.111:0/1345200368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46d406b5f0 0x7f46d40abae0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f46d40adc10 tx=0x7f46c802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee799cfe2a81222c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:28.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.802+0000 7f46d1ffb640 1 -- 172.21.15.111:0/1345200368 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8032440 con 0x7f46d406b5f0 2024-09-13T17:58:28.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.802+0000 7f46d1ffb640 1 -- 172.21.15.111:0/1345200368 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c80325a0 con 0x7f46d406b5f0 2024-09-13T17:58:28.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.802+0000 7f46d1ffb640 1 -- 172.21.15.111:0/1345200368 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8032930 con 0x7f46d406b5f0 2024-09-13T17:58:28.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.803+0000 7f46d98e5640 1 -- 172.21.15.111:0/1345200368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46d406b5f0 msgr2=0x7f46d40abae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.803+0000 7f46d98e5640 1 --2- 172.21.15.111:0/1345200368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46d406b5f0 0x7f46d40abae0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f46d40adc10 tx=0x7f46c802a1a0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.803+0000 7f46d98e5640 1 -- 172.21.15.111:0/1345200368 shutdown_connections 2024-09-13T17:58:28.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.803+0000 7f46d98e5640 1 --2- 172.21.15.111:0/1345200368 >> v2:172.21.15.188:3300/0 conn(0x7f46d415c660 0x7f46d40ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.803+0000 7f46d98e5640 1 --2- 172.21.15.111:0/1345200368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46d406b5f0 0x7f46d40abae0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.803+0000 7f46d98e5640 1 -- 172.21.15.111:0/1345200368 >> 172.21.15.111:0/1345200368 conn(0x7f46d40709f0 msgr2=0x7f46d4070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:28.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.803+0000 7f46d98e5640 1 -- 172.21.15.111:0/1345200368 shutdown_connections 2024-09-13T17:58:28.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.803+0000 7f46d98e5640 1 -- 172.21.15.111:0/1345200368 wait complete. 2024-09-13T17:58:28.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.804+0000 7f46d98e5640 1 Processor -- start 2024-09-13T17:58:28.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.804+0000 7f46d98e5640 1 -- start start 2024-09-13T17:58:28.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d98e5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46d406b5f0 0x7f46d41f1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d98e5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46d415c280 0x7f46d40ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d98e5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46d415c660 0x7f46d40c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d98e5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d41f19d0 con 0x7f46d415c660 2024-09-13T17:58:28.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d98e5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d40c9390 con 0x7f46d415c280 2024-09-13T17:58:28.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d98e5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d40c9670 con 0x7f46d406b5f0 2024-09-13T17:58:28.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46d415c660 0x7f46d40c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46d415c660 0x7f46d40c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34124/0 (socket says 172.21.15.111:34124) 2024-09-13T17:58:28.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d37fe640 1 -- 172.21.15.111:0/2604256210 learned_addr learned my addr 172.21.15.111:0/2604256210 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:28.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d27fc640 1 --2- 172.21.15.111:0/2604256210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46d415c280 0x7f46d40ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d2ffd640 1 --2- 172.21.15.111:0/2604256210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46d406b5f0 0x7f46d41f1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d37fe640 1 -- 172.21.15.111:0/2604256210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46d406b5f0 msgr2=0x7f46d41f1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d37fe640 1 --2- 172.21.15.111:0/2604256210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46d406b5f0 0x7f46d41f1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d37fe640 1 -- 172.21.15.111:0/2604256210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46d415c280 msgr2=0x7f46d40ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d37fe640 1 --2- 172.21.15.111:0/2604256210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46d415c280 0x7f46d40ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d37fe640 1 -- 172.21.15.111:0/2604256210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c8009400 con 0x7f46d415c660 2024-09-13T17:58:28.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d2ffd640 1 --2- 172.21.15.111:0/2604256210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46d406b5f0 0x7f46d41f1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:28.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46d37fe640 1 --2- 172.21.15.111:0/2604256210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46d415c660 0x7f46d40c8940 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00cd50 tx=0x7f46cc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:28.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.805+0000 7f46bbfff640 1 -- 172.21.15.111:0/2604256210 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc00f680 con 0x7f46d415c660 2024-09-13T17:58:28.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.806+0000 7f46bbfff640 1 -- 172.21.15.111:0/2604256210 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc00f800 con 0x7f46d415c660 2024-09-13T17:58:28.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.806+0000 7f46d98e5640 1 -- 172.21.15.111:0/2604256210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d40c9950 con 0x7f46d415c660 2024-09-13T17:58:28.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.806+0000 7f46bbfff640 1 -- 172.21.15.111:0/2604256210 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc00fbb0 con 0x7f46d415c660 2024-09-13T17:58:28.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.806+0000 7f46d98e5640 1 -- 172.21.15.111:0/2604256210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d40c9f10 con 0x7f46d415c660 2024-09-13T17:58:28.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.806+0000 7f46d98e5640 1 -- 172.21.15.111:0/2604256210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d40ca100 con 0x7f46d415c660 2024-09-13T17:58:28.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.807+0000 7f46bbfff640 1 -- 172.21.15.111:0/2604256210 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f46cc00b040 con 0x7f46d415c660 2024-09-13T17:58:28.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.807+0000 7f46bbfff640 1 --2- 172.21.15.111:0/2604256210 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f46c006ac00 0x7f46c006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.808+0000 7f46bbfff640 1 -- 172.21.15.111:0/2604256210 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f46cc08a000 con 0x7f46d415c660 2024-09-13T17:58:28.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.808+0000 7f46d2ffd640 1 --2- 172.21.15.111:0/2604256210 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f46c006ac00 0x7f46c006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.808+0000 7f46d2ffd640 1 --2- 172.21.15.111:0/2604256210 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f46c006ac00 0x7f46c006d0c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f46c4000f60 tx=0x7f46c4007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:28.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.812+0000 7f46bbfff640 1 -- 172.21.15.111:0/2604256210 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f46cc054eb0 con 0x7f46d415c660 2024-09-13T17:58:28.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.983+0000 7f46d98e5640 1 -- 172.21.15.111:0/2604256210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f46d406aee0 con 0x7f46d415c660 2024-09-13T17:58:28.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.983+0000 7f46bbfff640 1 -- 172.21.15.111:0/2604256210 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f46cc0584d0 con 0x7f46d415c660 2024-09-13T17:58:28.985 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2024-09-13T17:58:28.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.985+0000 7f46d98e5640 1 -- 172.21.15.111:0/2604256210 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f46c006ac00 msgr2=0x7f46c006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.985+0000 7f46d98e5640 1 --2- 172.21.15.111:0/2604256210 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f46c006ac00 0x7f46c006d0c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f46c4000f60 tx=0x7f46c4007c10 comp rx=0 tx=0).stop 2024-09-13T17:58:28.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.985+0000 7f46d98e5640 1 -- 172.21.15.111:0/2604256210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46d415c660 msgr2=0x7f46d40c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.985+0000 7f46d98e5640 1 --2- 172.21.15.111:0/2604256210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46d415c660 0x7f46d40c8940 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00cd50 tx=0x7f46cc00a650 comp rx=0 tx=0).stop 2024-09-13T17:58:28.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.985+0000 7f46d98e5640 1 -- 172.21.15.111:0/2604256210 shutdown_connections 2024-09-13T17:58:28.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.985+0000 7f46d98e5640 1 --2- 172.21.15.111:0/2604256210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46d415c660 0x7f46d40c8940 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.985+0000 7f46d98e5640 1 --2- 172.21.15.111:0/2604256210 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f46c006ac00 0x7f46c006d0c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.985+0000 7f46d98e5640 1 --2- 172.21.15.111:0/2604256210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46d415c280 0x7f46d40ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.985+0000 7f46d98e5640 1 --2- 172.21.15.111:0/2604256210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46d406b5f0 0x7f46d41f1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.985+0000 7f46d98e5640 1 -- 172.21.15.111:0/2604256210 >> 172.21.15.111:0/2604256210 conn(0x7f46d40709f0 msgr2=0x7f46d40b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:28.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.985+0000 7f46d98e5640 1 -- 172.21.15.111:0/2604256210 shutdown_connections 2024-09-13T17:58:28.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:28.985+0000 7f46d98e5640 1 -- 172.21.15.111:0/2604256210 wait complete. 2024-09-13T17:58:28.998 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi188.front.sepia.ceph.com... 2024-09-13T17:58:28.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:28.998 DEBUG:teuthology.orchestra.run.smithi188:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-09-13T17:58:29.028 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:58:29.028 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-09-13T17:58:29.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:29.028 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-13T17:58:29.085 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-13T17:58:29.086 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:58:29.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.181+0000 7fa261b3e640 1 Processor -- start 2024-09-13T17:58:29.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.181+0000 7fa261b3e640 1 -- start start 2024-09-13T17:58:29.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.182+0000 7fa261b3e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa25c15c6e0 0x7fa25c0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.182+0000 7fa261b3e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25c0d1840 con 0x7fa25c0d07b0 2024-09-13T17:58:29.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.182+0000 7fa261b3e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25c06bd20 con 0x7fa25c0ccb20 2024-09-13T17:58:29.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.182+0000 7fa261b3e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa25c160e50 con 0x7fa25c15c6e0 2024-09-13T17:58:29.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.182+0000 7fa25b7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa25c15c6e0 0x7fa25c0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.182+0000 7fa25b7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa25c15c6e0 0x7fa25c0b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34126/0 (socket says 172.21.15.111:34126) 2024-09-13T17:58:29.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.182+0000 7fa252ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa25c0d07b0 0x7fa25c160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56016/0 (socket says 172.21.15.111:56016) 2024-09-13T17:58:29.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.182+0000 7fa25b7fe640 1 -- 172.21.15.111:0/109782946 learned_addr learned my addr 172.21.15.111:0/109782946 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:29.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.183+0000 7fa25b7fe640 1 -- 172.21.15.111:0/109782946 >> v1:172.21.15.188:6789/0 conn(0x7fa25c0ccb20 legacy=0x7fa25c160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:29.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.183+0000 7fa25b7fe640 1 -- 172.21.15.111:0/109782946 >> v1:172.21.15.111:6789/0 conn(0x7fa25c0d07b0 legacy=0x7fa25c160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.183+0000 7fa25b7fe640 1 -- 172.21.15.111:0/109782946 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c0d0c30 con 0x7fa25c15c6e0 2024-09-13T17:58:29.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.183+0000 7fa25b7fe640 1 --2- 172.21.15.111:0/109782946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa25c15c6e0 0x7fa25c0b3f50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009870 tx=0x7fa24c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1387d9672984e275 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.184+0000 7fa25affd640 1 -- 172.21.15.111:0/109782946 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c032440 con 0x7fa25c15c6e0 2024-09-13T17:58:29.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.184+0000 7fa25affd640 1 -- 172.21.15.111:0/109782946 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24c0325a0 con 0x7fa25c15c6e0 2024-09-13T17:58:29.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.184+0000 7fa25affd640 1 -- 172.21.15.111:0/109782946 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c032930 con 0x7fa25c15c6e0 2024-09-13T17:58:29.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.184+0000 7fa261b3e640 1 -- 172.21.15.111:0/109782946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa25c15c6e0 msgr2=0x7fa25c0b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.184+0000 7fa261b3e640 1 --2- 172.21.15.111:0/109782946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa25c15c6e0 0x7fa25c0b3f50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009870 tx=0x7fa24c02a210 comp rx=0 tx=0).stop 2024-09-13T17:58:29.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.184+0000 7fa261b3e640 1 -- 172.21.15.111:0/109782946 shutdown_connections 2024-09-13T17:58:29.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.184+0000 7fa261b3e640 1 --2- 172.21.15.111:0/109782946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa25c15c6e0 0x7fa25c0b3f50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.184+0000 7fa261b3e640 1 -- 172.21.15.111:0/109782946 >> 172.21.15.111:0/109782946 conn(0x7fa25c089f60 msgr2=0x7fa25c0d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:29.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.184+0000 7fa261b3e640 1 -- 172.21.15.111:0/109782946 shutdown_connections 2024-09-13T17:58:29.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.184+0000 7fa261b3e640 1 -- 172.21.15.111:0/109782946 wait complete. 2024-09-13T17:58:29.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.185+0000 7fa261b3e640 1 Processor -- start 2024-09-13T17:58:29.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.186+0000 7fa261b3e640 1 -- start start 2024-09-13T17:58:29.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.186+0000 7fa261b3e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa25c0ccb20 0x7fa25c1544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.186+0000 7fa261b3e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa25c0d07b0 0x7fa25c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.186+0000 7fa261b3e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa25c15c6e0 0x7fa25c0b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.186+0000 7fa261b3e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c154030 con 0x7fa25c0ccb20 2024-09-13T17:58:29.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.186+0000 7fa261b3e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c154a00 con 0x7fa25c15c6e0 2024-09-13T17:58:29.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.186+0000 7fa261b3e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c0b0870 con 0x7fa25c0d07b0 2024-09-13T17:58:29.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.186+0000 7fa25b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa25c0ccb20 0x7fa25c1544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.186+0000 7fa25b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa25c0ccb20 0x7fa25c1544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34134/0 (socket says 172.21.15.111:34134) 2024-09-13T17:58:29.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.186+0000 7fa25b7fe640 1 -- 172.21.15.111:0/3662265846 learned_addr learned my addr 172.21.15.111:0/3662265846 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:29.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.187+0000 7fa252ffd640 1 --2- 172.21.15.111:0/3662265846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa25c0d07b0 0x7fa25c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.187+0000 7fa25bfff640 1 --2- 172.21.15.111:0/3662265846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa25c15c6e0 0x7fa25c0b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.187+0000 7fa25b7fe640 1 -- 172.21.15.111:0/3662265846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa25c0d07b0 msgr2=0x7fa25c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.187+0000 7fa25b7fe640 1 --2- 172.21.15.111:0/3662265846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa25c0d07b0 0x7fa25c0aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.187+0000 7fa25b7fe640 1 -- 172.21.15.111:0/3662265846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa25c15c6e0 msgr2=0x7fa25c0b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.187+0000 7fa25b7fe640 1 --2- 172.21.15.111:0/3662265846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa25c15c6e0 0x7fa25c0b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.187+0000 7fa25b7fe640 1 -- 172.21.15.111:0/3662265846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24c0094d0 con 0x7fa25c0ccb20 2024-09-13T17:58:29.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.187+0000 7fa252ffd640 1 --2- 172.21.15.111:0/3662265846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa25c0d07b0 0x7fa25c0aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:29.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.187+0000 7fa25bfff640 1 --2- 172.21.15.111:0/3662265846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa25c15c6e0 0x7fa25c0b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:29.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.187+0000 7fa25b7fe640 1 --2- 172.21.15.111:0/3662265846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa25c0ccb20 0x7fa25c1544c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009870 tx=0x7fa24c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.187+0000 7fa258ff9640 1 -- 172.21.15.111:0/3662265846 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c030b00 con 0x7fa25c0ccb20 2024-09-13T17:58:29.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.187+0000 7fa258ff9640 1 -- 172.21.15.111:0/3662265846 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24c030c60 con 0x7fa25c0ccb20 2024-09-13T17:58:29.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.187+0000 7fa261b3e640 1 -- 172.21.15.111:0/3662265846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25c153700 con 0x7fa25c0ccb20 2024-09-13T17:58:29.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.187+0000 7fa258ff9640 1 -- 172.21.15.111:0/3662265846 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c03d420 con 0x7fa25c0ccb20 2024-09-13T17:58:29.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.187+0000 7fa261b3e640 1 -- 172.21.15.111:0/3662265846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25c153cc0 con 0x7fa25c0ccb20 2024-09-13T17:58:29.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.188+0000 7fa261b3e640 1 -- 172.21.15.111:0/3662265846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25c153e50 con 0x7fa25c0ccb20 2024-09-13T17:58:29.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.188+0000 7fa258ff9640 1 -- 172.21.15.111:0/3662265846 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa24c039040 con 0x7fa25c0ccb20 2024-09-13T17:58:29.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.189+0000 7fa258ff9640 1 --2- 172.21.15.111:0/3662265846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa23c06ac00 0x7fa23c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.189+0000 7fa252ffd640 1 --2- 172.21.15.111:0/3662265846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa23c06ac00 0x7fa23c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.189+0000 7fa258ff9640 1 -- 172.21.15.111:0/3662265846 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa24c0aa4b0 con 0x7fa25c0ccb20 2024-09-13T17:58:29.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.189+0000 7fa252ffd640 1 --2- 172.21.15.111:0/3662265846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa23c06ac00 0x7fa23c06d0c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa25c1f5ae0 tx=0x7fa244009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.193+0000 7fa258ff9640 1 -- 172.21.15.111:0/3662265846 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fa24c075240 con 0x7fa25c0ccb20 2024-09-13T17:58:29.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.392+0000 7fa261b3e640 1 -- 172.21.15.111:0/3662265846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa25c0b4c80 con 0x7fa25c0ccb20 2024-09-13T17:58:29.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.392+0000 7fa258ff9640 1 -- 172.21.15.111:0/3662265846 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4742 (secure 0 0 0) 0x7fa24c078860 con 0x7fa25c0ccb20 2024-09-13T17:58:29.394 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:29.394 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4001406804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":4001406804},{"type":"v1","addr":"172.21.15.188:6835","nonce":4001406804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2565466103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2565466103},{"type":"v1","addr":"172.21.15.111:6837","nonce":2565466103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/1252690799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1252690799},{"type":"v1","addr":"172.21.15.111:6836","nonce":1252690799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:57:25.702997+0000","modified":"2024-09-13T17:57:46.351493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/1266310843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1266310843},{"type":"v1","addr":"172.21.15.188:6834","nonce":1266310843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:58:29.395 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-09-13T17:58:29.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.394+0000 7fa261b3e640 1 -- 172.21.15.111:0/3662265846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa23c06ac00 msgr2=0x7fa23c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.394+0000 7fa261b3e640 1 --2- 172.21.15.111:0/3662265846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa23c06ac00 0x7fa23c06d0c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa25c1f5ae0 tx=0x7fa244009040 comp rx=0 tx=0).stop 2024-09-13T17:58:29.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.394+0000 7fa261b3e640 1 -- 172.21.15.111:0/3662265846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa25c0ccb20 msgr2=0x7fa25c1544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.394+0000 7fa261b3e640 1 --2- 172.21.15.111:0/3662265846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa25c0ccb20 0x7fa25c1544c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009870 tx=0x7fa24c0046c0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.394+0000 7fa261b3e640 1 -- 172.21.15.111:0/3662265846 shutdown_connections 2024-09-13T17:58:29.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.394+0000 7fa261b3e640 1 --2- 172.21.15.111:0/3662265846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa23c06ac00 0x7fa23c06d0c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.394+0000 7fa261b3e640 1 --2- 172.21.15.111:0/3662265846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa25c15c6e0 0x7fa25c0b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.394+0000 7fa261b3e640 1 --2- 172.21.15.111:0/3662265846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa25c0d07b0 0x7fa25c0aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.394+0000 7fa261b3e640 1 --2- 172.21.15.111:0/3662265846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa25c0ccb20 0x7fa25c1544c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.394+0000 7fa261b3e640 1 -- 172.21.15.111:0/3662265846 >> 172.21.15.111:0/3662265846 conn(0x7fa25c089f60 msgr2=0x7fa25c0d2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:29.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.394+0000 7fa261b3e640 1 -- 172.21.15.111:0/3662265846 shutdown_connections 2024-09-13T17:58:29.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.394+0000 7fa261b3e640 1 -- 172.21.15.111:0/3662265846 wait complete. 2024-09-13T17:58:29.407 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:29.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.503+0000 7fcee9a3d640 1 Processor -- start 2024-09-13T17:58:29.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.503+0000 7fcee9a3d640 1 -- start start 2024-09-13T17:58:29.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.504+0000 7fcee9a3d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcee40d07b0 0x7fcee406ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.504+0000 7fcee9a3d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcee40b4660 0x7fcee406b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.504+0000 7fcee9a3d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee40d1840 con 0x7fcee415c6e0 2024-09-13T17:58:29.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.504+0000 7fcee9a3d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee40d0c30 con 0x7fcee40b4660 2024-09-13T17:58:29.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.504+0000 7fcee9a3d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee40d0f10 con 0x7fcee40d07b0 2024-09-13T17:58:29.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.504+0000 7fcee2ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fcee415c6e0 0x7fcee40b3f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56022/0 (socket says 172.21.15.111:56022) 2024-09-13T17:58:29.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.504+0000 7fcee37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcee40b4660 0x7fcee406b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.504+0000 7fcee2ffd640 1 -- 172.21.15.111:0/221247970 learned_addr learned my addr 172.21.15.111:0/221247970 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:29.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.505+0000 7fceda7fc640 1 --2- 172.21.15.111:0/221247970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcee40d07b0 0x7fcee406ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.505+0000 7fcee37fe640 1 -- 172.21.15.111:0/221247970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcee40d07b0 msgr2=0x7fcee406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:29.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.505+0000 7fcee37fe640 1 --2- 172.21.15.111:0/221247970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcee40d07b0 0x7fcee406ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.505+0000 7fcee37fe640 1 -- 172.21.15.111:0/221247970 >> v1:172.21.15.111:6789/0 conn(0x7fcee415c6e0 legacy=0x7fcee40b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.505+0000 7fcee37fe640 1 -- 172.21.15.111:0/221247970 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee40d11f0 con 0x7fcee40b4660 2024-09-13T17:58:29.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.505+0000 7fcee27fc640 1 -- 172.21.15.111:0/221247970 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 688664467 0 0) 0x7fcee40d1840 con 0x7fcee415c6e0 2024-09-13T17:58:29.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.505+0000 7fcee37fe640 1 --2- 172.21.15.111:0/221247970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee40b4660 0x7fcee406b250 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcedc009d90 tx=0x7fcedc02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af07f4f02aa65e1a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.506+0000 7fcee27fc640 1 -- 172.21.15.111:0/221247970 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc031600 con 0x7fcee40b4660 2024-09-13T17:58:29.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.506+0000 7fcee27fc640 1 -- 172.21.15.111:0/221247970 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcedc02d050 con 0x7fcee40b4660 2024-09-13T17:58:29.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.506+0000 7fcee27fc640 1 -- 172.21.15.111:0/221247970 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc031980 con 0x7fcee40b4660 2024-09-13T17:58:29.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.506+0000 7fcee9a3d640 1 -- 172.21.15.111:0/221247970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee40b4660 msgr2=0x7fcee406b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.506+0000 7fcee9a3d640 1 --2- 172.21.15.111:0/221247970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee40b4660 0x7fcee406b250 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcedc009d90 tx=0x7fcedc02a5d0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.506+0000 7fcee9a3d640 1 -- 172.21.15.111:0/221247970 shutdown_connections 2024-09-13T17:58:29.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.506+0000 7fcee9a3d640 1 --2- 172.21.15.111:0/221247970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee40b4660 0x7fcee406b250 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.506+0000 7fcee9a3d640 1 --2- 172.21.15.111:0/221247970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcee40d07b0 0x7fcee406ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.506+0000 7fcee9a3d640 1 -- 172.21.15.111:0/221247970 >> 172.21.15.111:0/221247970 conn(0x7fcee4089f60 msgr2=0x7fcee40d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:29.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.506+0000 7fcee9a3d640 1 -- 172.21.15.111:0/221247970 shutdown_connections 2024-09-13T17:58:29.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.506+0000 7fcee9a3d640 1 -- 172.21.15.111:0/221247970 wait complete. 2024-09-13T17:58:29.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.507+0000 7fcee9a3d640 1 Processor -- start 2024-09-13T17:58:29.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.508+0000 7fcee9a3d640 1 -- start start 2024-09-13T17:58:29.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.508+0000 7fcee9a3d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcee40b4660 0x7fcee41515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.508+0000 7fcee9a3d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee40d07b0 0x7fcee40b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.508+0000 7fcee9a3d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcee415c6e0 0x7fcee40b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.508+0000 7fcee9a3d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee415cf30 con 0x7fcee40d07b0 2024-09-13T17:58:29.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.508+0000 7fcee9a3d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee415d210 con 0x7fcee415c6e0 2024-09-13T17:58:29.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.508+0000 7fcee9a3d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee415d4f0 con 0x7fcee40b4660 2024-09-13T17:58:29.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.508+0000 7fceda7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee40d07b0 0x7fcee40b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.508+0000 7fceda7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee40d07b0 0x7fcee40b74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34158/0 (socket says 172.21.15.111:34158) 2024-09-13T17:58:29.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.508+0000 7fceda7fc640 1 -- 172.21.15.111:0/1887252523 learned_addr learned my addr 172.21.15.111:0/1887252523 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:29.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.508+0000 7fcee2ffd640 1 --2- 172.21.15.111:0/1887252523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcee40b4660 0x7fcee41515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.508+0000 7fcee37fe640 1 --2- 172.21.15.111:0/1887252523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcee415c6e0 0x7fcee40b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.509+0000 7fceda7fc640 1 -- 172.21.15.111:0/1887252523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcee40b4660 msgr2=0x7fcee41515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.509+0000 7fceda7fc640 1 --2- 172.21.15.111:0/1887252523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcee40b4660 0x7fcee41515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.509+0000 7fceda7fc640 1 -- 172.21.15.111:0/1887252523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcee415c6e0 msgr2=0x7fcee40b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.509+0000 7fceda7fc640 1 --2- 172.21.15.111:0/1887252523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcee415c6e0 0x7fcee40b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.509+0000 7fceda7fc640 1 -- 172.21.15.111:0/1887252523 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc009a70 con 0x7fcee40d07b0 2024-09-13T17:58:29.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.509+0000 7fcee2ffd640 1 --2- 172.21.15.111:0/1887252523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcee40b4660 0x7fcee41515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:29.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.509+0000 7fceda7fc640 1 --2- 172.21.15.111:0/1887252523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee40d07b0 0x7fcee40b74a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00d7d0 tx=0x7fcecc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.509+0000 7fcee0ff9640 1 -- 172.21.15.111:0/1887252523 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc00b6c0 con 0x7fcee40d07b0 2024-09-13T17:58:29.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.509+0000 7fcee0ff9640 1 -- 172.21.15.111:0/1887252523 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc00b840 con 0x7fcee40d07b0 2024-09-13T17:58:29.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.509+0000 7fcee9a3d640 1 -- 172.21.15.111:0/1887252523 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee415d7d0 con 0x7fcee40d07b0 2024-09-13T17:58:29.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.509+0000 7fcee0ff9640 1 -- 172.21.15.111:0/1887252523 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc00bbf0 con 0x7fcee40d07b0 2024-09-13T17:58:29.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.509+0000 7fcee9a3d640 1 -- 172.21.15.111:0/1887252523 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee4151ae0 con 0x7fcee40d07b0 2024-09-13T17:58:29.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.510+0000 7fcee9a3d640 1 -- 172.21.15.111:0/1887252523 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee40ccb20 con 0x7fcee40d07b0 2024-09-13T17:58:29.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.510+0000 7fcee0ff9640 1 -- 172.21.15.111:0/1887252523 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcecc00bd50 con 0x7fcee40d07b0 2024-09-13T17:58:29.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.511+0000 7fcee0ff9640 1 --2- 172.21.15.111:0/1887252523 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcec406ac00 0x7fcec406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.511+0000 7fcee2ffd640 1 --2- 172.21.15.111:0/1887252523 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcec406ac00 0x7fcec406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.511+0000 7fcee0ff9640 1 -- 172.21.15.111:0/1887252523 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcecc089b90 con 0x7fcee40d07b0 2024-09-13T17:58:29.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.511+0000 7fcee2ffd640 1 --2- 172.21.15.111:0/1887252523 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcec406ac00 0x7fcec406d0c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fced4000f60 tx=0x7fced4007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.515+0000 7fcee0ff9640 1 -- 172.21.15.111:0/1887252523 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fcecc054a40 con 0x7fcee40d07b0 2024-09-13T17:58:29.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.676+0000 7fcee9a3d640 1 -- 172.21.15.111:0/1887252523 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcee408c610 con 0x7fcee40d07b0 2024-09-13T17:58:29.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.677+0000 7fcee0ff9640 1 -- 172.21.15.111:0/1887252523 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18794 (secure 0 0 0) 0x7fcecc058060 con 0x7fcee40d07b0 2024-09-13T17:58:29.679 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:29.679 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":26,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:07.749113+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:29.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.679+0000 7fcee9a3d640 1 -- 172.21.15.111:0/1887252523 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcec406ac00 msgr2=0x7fcec406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.679+0000 7fcee9a3d640 1 --2- 172.21.15.111:0/1887252523 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcec406ac00 0x7fcec406d0c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fced4000f60 tx=0x7fced4007c10 comp rx=0 tx=0).stop 2024-09-13T17:58:29.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.679+0000 7fcee9a3d640 1 -- 172.21.15.111:0/1887252523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee40d07b0 msgr2=0x7fcee40b74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.679+0000 7fcee9a3d640 1 --2- 172.21.15.111:0/1887252523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee40d07b0 0x7fcee40b74a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00d7d0 tx=0x7fcecc00dc90 comp rx=0 tx=0).stop 2024-09-13T17:58:29.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.679+0000 7fcee9a3d640 1 -- 172.21.15.111:0/1887252523 shutdown_connections 2024-09-13T17:58:29.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.679+0000 7fcee9a3d640 1 --2- 172.21.15.111:0/1887252523 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcec406ac00 0x7fcec406d0c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.679+0000 7fcee9a3d640 1 --2- 172.21.15.111:0/1887252523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcee415c6e0 0x7fcee40b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.679+0000 7fcee9a3d640 1 --2- 172.21.15.111:0/1887252523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee40d07b0 0x7fcee40b74a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.679+0000 7fcee9a3d640 1 --2- 172.21.15.111:0/1887252523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcee40b4660 0x7fcee41515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.679+0000 7fcee9a3d640 1 -- 172.21.15.111:0/1887252523 >> 172.21.15.111:0/1887252523 conn(0x7fcee4089f60 msgr2=0x7fcee40d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:29.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.679+0000 7fcee9a3d640 1 -- 172.21.15.111:0/1887252523 shutdown_connections 2024-09-13T17:58:29.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.679+0000 7fcee9a3d640 1 -- 172.21.15.111:0/1887252523 wait complete. 2024-09-13T17:58:29.693 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:57:24.089739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:57:24.667768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:57:27.731417+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:58:29.693 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:58:29.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.794+0000 7f48b6041640 1 Processor -- start 2024-09-13T17:58:29.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.794+0000 7f48b6041640 1 -- start start 2024-09-13T17:58:29.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.795+0000 7f48b6041640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f48b015c500 0x7f48b00d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.795+0000 7f48b6041640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b00d2590 0x7f48b00cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.795+0000 7f48b6041640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b00cd450 con 0x7f48b00cde20 2024-09-13T17:58:29.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.795+0000 7f48b6041640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b00d29f0 con 0x7f48b015c500 2024-09-13T17:58:29.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.795+0000 7f48b6041640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b00d0cf0 con 0x7f48b00d2590 2024-09-13T17:58:29.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.795+0000 7f48af7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f48b015c500 0x7f48b00d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.795+0000 7f48af7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f48b015c500 0x7f48b00d2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34162/0 (socket says 172.21.15.111:34162) 2024-09-13T17:58:29.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.795+0000 7f48af7fe640 1 -- 172.21.15.111:0/250693746 learned_addr learned my addr 172.21.15.111:0/250693746 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:29.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.795+0000 7f48a7fff640 1 --2- 172.21.15.111:0/250693746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b00d2590 0x7f48b00cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.795+0000 7f48af7fe640 1 -- 172.21.15.111:0/250693746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b00d2590 msgr2=0x7f48b00cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:29.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.795+0000 7f48af7fe640 1 --2- 172.21.15.111:0/250693746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b00d2590 0x7f48b00cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.795+0000 7f48af7fe640 1 -- 172.21.15.111:0/250693746 >> v1:172.21.15.188:6789/0 conn(0x7f48b00cde20 legacy=0x7f48b00d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.795+0000 7f48af7fe640 1 -- 172.21.15.111:0/250693746 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b00d0fd0 con 0x7f48b015c500 2024-09-13T17:58:29.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.796+0000 7f48af7fe640 1 --2- 172.21.15.111:0/250693746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b015c500 0x7f48b00d2050 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009870 tx=0x7f48a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc0b70c8d630d81 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.796+0000 7f48aeffd640 1 -- 172.21.15.111:0/250693746 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0032440 con 0x7f48b015c500 2024-09-13T17:58:29.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.796+0000 7f48aeffd640 1 -- 172.21.15.111:0/250693746 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a00325a0 con 0x7f48b015c500 2024-09-13T17:58:29.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.796+0000 7f48aeffd640 1 -- 172.21.15.111:0/250693746 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0032930 con 0x7f48b015c500 2024-09-13T17:58:29.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.797+0000 7f48b6041640 1 -- 172.21.15.111:0/250693746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b015c500 msgr2=0x7f48b00d2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.797+0000 7f48b6041640 1 --2- 172.21.15.111:0/250693746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b015c500 0x7f48b00d2050 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009870 tx=0x7f48a002a270 comp rx=0 tx=0).stop 2024-09-13T17:58:29.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.797+0000 7f48b6041640 1 -- 172.21.15.111:0/250693746 shutdown_connections 2024-09-13T17:58:29.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.797+0000 7f48b6041640 1 --2- 172.21.15.111:0/250693746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b00d2590 0x7f48b00cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.797+0000 7f48b6041640 1 --2- 172.21.15.111:0/250693746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b015c500 0x7f48b00d2050 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.797+0000 7f48b6041640 1 -- 172.21.15.111:0/250693746 >> 172.21.15.111:0/250693746 conn(0x7f48b0089f60 msgr2=0x7f48b009b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:29.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.797+0000 7f48b6041640 1 -- 172.21.15.111:0/250693746 shutdown_connections 2024-09-13T17:58:29.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.797+0000 7f48b6041640 1 -- 172.21.15.111:0/250693746 wait complete. 2024-09-13T17:58:29.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.798+0000 7f48b6041640 1 Processor -- start 2024-09-13T17:58:29.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.798+0000 7f48b6041640 1 -- start start 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48b6041640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b00cde20 0x7f48b00c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48b6041640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b00d2590 0x7f48b0151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48b6041640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48b015c500 0x7f48b00b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48b6041640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b015f2a0 con 0x7f48b00cde20 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48b6041640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b00b7b60 con 0x7f48b015c500 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48b6041640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b00b7db0 con 0x7f48b00d2590 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48af7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b00cde20 0x7f48b00c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48af7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b00cde20 0x7f48b00c99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34178/0 (socket says 172.21.15.111:34178) 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48af7fe640 1 -- 172.21.15.111:0/1780461216 learned_addr learned my addr 172.21.15.111:0/1780461216 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48a7fff640 1 --2- 172.21.15.111:0/1780461216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b00d2590 0x7f48b0151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48affff640 1 --2- 172.21.15.111:0/1780461216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48b015c500 0x7f48b00b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48af7fe640 1 -- 172.21.15.111:0/1780461216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b00d2590 msgr2=0x7f48b0151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48af7fe640 1 --2- 172.21.15.111:0/1780461216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b00d2590 0x7f48b0151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48af7fe640 1 -- 172.21.15.111:0/1780461216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48b015c500 msgr2=0x7f48b00b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48af7fe640 1 --2- 172.21.15.111:0/1780461216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48b015c500 0x7f48b00b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48af7fe640 1 -- 172.21.15.111:0/1780461216 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a00094d0 con 0x7f48b00cde20 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48af7fe640 1 --2- 172.21.15.111:0/1780461216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b00cde20 0x7f48b00c99d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f48a0000c00 tx=0x7f48a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48acff9640 1 -- 172.21.15.111:0/1780461216 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0032440 con 0x7f48b00cde20 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48acff9640 1 -- 172.21.15.111:0/1780461216 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a0030b80 con 0x7f48b00cde20 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48b6041640 1 -- 172.21.15.111:0/1780461216 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b00b8090 con 0x7f48b00cde20 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48acff9640 1 -- 172.21.15.111:0/1780461216 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a003d440 con 0x7f48b00cde20 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.799+0000 7f48b6041640 1 -- 172.21.15.111:0/1780461216 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b0151f80 con 0x7f48b00cde20 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.800+0000 7f48b6041640 1 -- 172.21.15.111:0/1780461216 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b00cd8e0 con 0x7f48b00cde20 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.801+0000 7f48acff9640 1 -- 172.21.15.111:0/1780461216 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f48a0039040 con 0x7f48b00cde20 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.801+0000 7f48acff9640 1 --2- 172.21.15.111:0/1780461216 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f489006ac00 0x7f489006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.801+0000 7f48a7fff640 1 --2- 172.21.15.111:0/1780461216 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f489006ac00 0x7f489006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.801+0000 7f48acff9640 1 -- 172.21.15.111:0/1780461216 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f48a00aa110 con 0x7f48b00cde20 2024-09-13T17:58:29.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.802+0000 7f48a7fff640 1 --2- 172.21.15.111:0/1780461216 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f489006ac00 0x7f489006d0c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f48b015b670 tx=0x7f4898008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.805+0000 7f48acff9640 1 -- 172.21.15.111:0/1780461216 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f48a0074fc0 con 0x7f48b00cde20 2024-09-13T17:58:29.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.969+0000 7f48b6041640 1 -- 172.21.15.111:0/1780461216 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48b00b5c70 con 0x7f48b00cde20 2024-09-13T17:58:29.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.969+0000 7f48acff9640 1 -- 172.21.15.111:0/1780461216 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48a00069a0 con 0x7f48b00cde20 2024-09-13T17:58:29.971 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:58:29.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.971+0000 7f48b6041640 1 -- 172.21.15.111:0/1780461216 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f489006ac00 msgr2=0x7f489006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.971+0000 7f48b6041640 1 --2- 172.21.15.111:0/1780461216 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f489006ac00 0x7f489006d0c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f48b015b670 tx=0x7f4898008040 comp rx=0 tx=0).stop 2024-09-13T17:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.971+0000 7f48b6041640 1 -- 172.21.15.111:0/1780461216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b00cde20 msgr2=0x7f48b00c99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.971+0000 7f48b6041640 1 --2- 172.21.15.111:0/1780461216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b00cde20 0x7f48b00c99d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f48a0000c00 tx=0x7f48a00046c0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.971+0000 7f48b6041640 1 -- 172.21.15.111:0/1780461216 shutdown_connections 2024-09-13T17:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.971+0000 7f48b6041640 1 --2- 172.21.15.111:0/1780461216 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f489006ac00 0x7f489006d0c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.971+0000 7f48b6041640 1 --2- 172.21.15.111:0/1780461216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48b015c500 0x7f48b00b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.971+0000 7f48b6041640 1 --2- 172.21.15.111:0/1780461216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b00d2590 0x7f48b0151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.971+0000 7f48b6041640 1 --2- 172.21.15.111:0/1780461216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b00cde20 0x7f48b00c99d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.971+0000 7f48b6041640 1 -- 172.21.15.111:0/1780461216 >> 172.21.15.111:0/1780461216 conn(0x7f48b0089f60 msgr2=0x7f48b00b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.971+0000 7f48b6041640 1 -- 172.21.15.111:0/1780461216 shutdown_connections 2024-09-13T17:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:29.971+0000 7f48b6041640 1 -- 172.21.15.111:0/1780461216 wait complete. 2024-09-13T17:58:29.985 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:58:30.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.082+0000 7f471a908640 1 Processor -- start 2024-09-13T17:58:30.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.082+0000 7f471a908640 1 -- start start 2024-09-13T17:58:30.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.083+0000 7f471a908640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f47140cec00 0x7f47140ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.083+0000 7f471a908640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47140ccbe0 con 0x7f4714159310 2024-09-13T17:58:30.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.083+0000 7f471a908640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47140b1810 con 0x7f47140b2b40 2024-09-13T17:58:30.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.083+0000 7f471a908640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47140ad940 con 0x7f47140cec00 2024-09-13T17:58:30.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.083+0000 7f4713fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4714159310 0x7f47140ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56032/0 (socket says 172.21.15.111:56032) 2024-09-13T17:58:30.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.083+0000 7f4713fff640 1 -- 172.21.15.111:0/2406484246 learned_addr learned my addr 172.21.15.111:0/2406484246 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:30.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.083+0000 7f47137fe640 1 -- 172.21.15.111:0/2406484246 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455176474 0 0) 0x7f47140ccbe0 con 0x7f4714159310 2024-09-13T17:58:30.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.083+0000 7f47137fe640 1 -- 172.21.15.111:0/2406484246 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4700003520 con 0x7f4714159310 2024-09-13T17:58:30.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.083+0000 7f4718ea5640 1 --2- 172.21.15.111:0/2406484246 >> v2:172.21.15.188:3300/0 conn(0x7f47140cec00 0x7f47140ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.083+0000 7f47137fe640 1 -- 172.21.15.111:0/2406484246 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 796828778 0 0) 0x7f4700003520 con 0x7f4714159310 2024-09-13T17:58:30.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f47137fe640 1 -- 172.21.15.111:0/2406484246 >> v2:172.21.15.188:3300/0 conn(0x7f47140cec00 msgr2=0x7f47140ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:30.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f47137fe640 1 --2- 172.21.15.111:0/2406484246 >> v2:172.21.15.188:3300/0 conn(0x7f47140cec00 0x7f47140ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f47137fe640 1 -- 172.21.15.111:0/2406484246 >> v1:172.21.15.188:6789/0 conn(0x7f47140b2b40 legacy=0x7f47140ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f47137fe640 1 -- 172.21.15.111:0/2406484246 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471406d9e0 con 0x7f4714159310 2024-09-13T17:58:30.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f47137fe640 1 -- 172.21.15.111:0/2406484246 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f4704001670 con 0x7f4714159310 2024-09-13T17:58:30.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f47137fe640 1 -- 172.21.15.111:0/2406484246 >> v1:172.21.15.111:6789/0 conn(0x7f4714159310 legacy=0x7f47140ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f47137fe640 1 --2- 172.21.15.111:0/2406484246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47000039c0 0x7f4700003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f47137fe640 1 --2- 172.21.15.111:0/2406484246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47000047c0 0x7f4700004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f47137fe640 1 --2- 172.21.15.111:0/2406484246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47000051d0 0x7f470000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f47137fe640 1 -- 172.21.15.111:0/2406484246 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470000c0f0 con 0x7f47000039c0 2024-09-13T17:58:30.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f47137fe640 1 -- 172.21.15.111:0/2406484246 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470000c260 con 0x7f47000047c0 2024-09-13T17:58:30.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f47137fe640 1 -- 172.21.15.111:0/2406484246 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470000c420 con 0x7f47000051d0 2024-09-13T17:58:30.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f4713fff640 1 --2- 172.21.15.111:0/2406484246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47000039c0 0x7f4700003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f470bfff640 1 --2- 172.21.15.111:0/2406484246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47000047c0 0x7f4700004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f4718ea5640 1 --2- 172.21.15.111:0/2406484246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47000051d0 0x7f470000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f4713fff640 1 -- 172.21.15.111:0/2406484246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47000051d0 msgr2=0x7f470000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:30.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f4713fff640 1 --2- 172.21.15.111:0/2406484246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47000051d0 0x7f470000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f4713fff640 1 -- 172.21.15.111:0/2406484246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47000047c0 msgr2=0x7f4700004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:30.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f4713fff640 1 --2- 172.21.15.111:0/2406484246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47000047c0 0x7f4700004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f4713fff640 1 -- 172.21.15.111:0/2406484246 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4700003520 con 0x7f47000039c0 2024-09-13T17:58:30.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f4718ea5640 1 -- 172.21.15.111:0/2406484246 reap_dead start 2024-09-13T17:58:30.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.084+0000 7f470bfff640 1 --2- 172.21.15.111:0/2406484246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47000047c0 0x7f4700004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:30.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.085+0000 7f4713fff640 1 --2- 172.21.15.111:0/2406484246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47000039c0 0x7f4700003de0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4704001eb0 tx=0x7f47000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3735e9348e29ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:30.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.085+0000 7f47137fe640 1 -- 172.21.15.111:0/2406484246 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704005ad0 con 0x7f47000039c0 2024-09-13T17:58:30.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.085+0000 7f47137fe640 1 -- 172.21.15.111:0/2406484246 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470402d070 con 0x7f47000039c0 2024-09-13T17:58:30.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.085+0000 7f47137fe640 1 -- 172.21.15.111:0/2406484246 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704005e90 con 0x7f47000039c0 2024-09-13T17:58:30.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.085+0000 7f471a908640 1 -- 172.21.15.111:0/2406484246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47000039c0 msgr2=0x7f4700003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.085+0000 7f471a908640 1 --2- 172.21.15.111:0/2406484246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47000039c0 0x7f4700003de0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4704001eb0 tx=0x7f47000057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.085+0000 7f471a908640 1 -- 172.21.15.111:0/2406484246 shutdown_connections 2024-09-13T17:58:30.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.085+0000 7f471a908640 1 --2- 172.21.15.111:0/2406484246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47000039c0 0x7f4700003de0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.085+0000 7f471a908640 1 -- 172.21.15.111:0/2406484246 >> 172.21.15.111:0/2406484246 conn(0x7f471406a0b0 msgr2=0x7f471406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:30.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.085+0000 7f471a908640 1 -- 172.21.15.111:0/2406484246 shutdown_connections 2024-09-13T17:58:30.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.085+0000 7f471a908640 1 -- 172.21.15.111:0/2406484246 wait complete. 2024-09-13T17:58:30.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.087+0000 7f471a908640 1 Processor -- start 2024-09-13T17:58:30.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.087+0000 7f471a908640 1 -- start start 2024-09-13T17:58:30.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.087+0000 7f471a908640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47000039c0 0x7f4714155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.087+0000 7f471a908640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47141528e0 0x7f4714153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.087+0000 7f471a908640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4714157a70 0x7f4714157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.087+0000 7f471a908640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47140c2960 con 0x7f47141528e0 2024-09-13T17:58:30.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.087+0000 7f471a908640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4714158430 con 0x7f4714157a70 2024-09-13T17:58:30.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.087+0000 7f471a908640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47141df450 con 0x7f47000039c0 2024-09-13T17:58:30.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.087+0000 7f470bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47141528e0 0x7f4714153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.088+0000 7f470bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47141528e0 0x7f4714153110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34196/0 (socket says 172.21.15.111:34196) 2024-09-13T17:58:30.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.088+0000 7f470bfff640 1 -- 172.21.15.111:0/3269478729 learned_addr learned my addr 172.21.15.111:0/3269478729 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:30.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.088+0000 7f4713fff640 1 --2- 172.21.15.111:0/3269478729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47000039c0 0x7f4714155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.088+0000 7f4718ea5640 1 --2- 172.21.15.111:0/3269478729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4714157a70 0x7f4714157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.088+0000 7f470bfff640 1 -- 172.21.15.111:0/3269478729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47000039c0 msgr2=0x7f4714155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.088+0000 7f470bfff640 1 --2- 172.21.15.111:0/3269478729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47000039c0 0x7f4714155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.088+0000 7f470bfff640 1 -- 172.21.15.111:0/3269478729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4714157a70 msgr2=0x7f4714157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.088+0000 7f470bfff640 1 --2- 172.21.15.111:0/3269478729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4714157a70 0x7f4714157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.088+0000 7f470bfff640 1 -- 172.21.15.111:0/3269478729 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4714153940 con 0x7f47141528e0 2024-09-13T17:58:30.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.088+0000 7f470bfff640 1 --2- 172.21.15.111:0/3269478729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47141528e0 0x7f4714153110 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00e440 tx=0x7f46fc00e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:30.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.088+0000 7f47117fa640 1 -- 172.21.15.111:0/3269478729 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0052e0 con 0x7f47141528e0 2024-09-13T17:58:30.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.088+0000 7f47117fa640 1 -- 172.21.15.111:0/3269478729 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46fc005440 con 0x7f47141528e0 2024-09-13T17:58:30.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.088+0000 7f47117fa640 1 -- 172.21.15.111:0/3269478729 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc003830 con 0x7f47141528e0 2024-09-13T17:58:30.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.088+0000 7f471a908640 1 -- 172.21.15.111:0/3269478729 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4714153c20 con 0x7f47141528e0 2024-09-13T17:58:30.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.088+0000 7f471a908640 1 -- 172.21.15.111:0/3269478729 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47141541e0 con 0x7f47141528e0 2024-09-13T17:58:30.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.089+0000 7f471a908640 1 -- 172.21.15.111:0/3269478729 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4714154400 con 0x7f47141528e0 2024-09-13T17:58:30.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.089+0000 7f47117fa640 1 -- 172.21.15.111:0/3269478729 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f46fc003990 con 0x7f47141528e0 2024-09-13T17:58:30.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.090+0000 7f47117fa640 1 --2- 172.21.15.111:0/3269478729 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f46f406ac30 0x7f46f406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.090+0000 7f47117fa640 1 -- 172.21.15.111:0/3269478729 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f46fc088b90 con 0x7f47141528e0 2024-09-13T17:58:30.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.090+0000 7f4713fff640 1 --2- 172.21.15.111:0/3269478729 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f46f406ac30 0x7f46f406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.090+0000 7f4713fff640 1 --2- 172.21.15.111:0/3269478729 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f46f406ac30 0x7f46f406d0f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4704032000 tx=0x7f470402d000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:30.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.094+0000 7f47117fa640 1 -- 172.21.15.111:0/3269478729 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f46fc053a90 con 0x7f47141528e0 2024-09-13T17:58:30.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.256+0000 7f471a908640 1 -- 172.21.15.111:0/3269478729 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f47140ad790 con 0x7f47141528e0 2024-09-13T17:58:30.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.257+0000 7f47117fa640 1 -- 172.21.15.111:0/3269478729 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f46fc0190a0 con 0x7f47141528e0 2024-09-13T17:58:30.259 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:30.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.259+0000 7f471a908640 1 -- 172.21.15.111:0/3269478729 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f46f406ac30 msgr2=0x7f46f406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.259+0000 7f471a908640 1 --2- 172.21.15.111:0/3269478729 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f46f406ac30 0x7f46f406d0f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4704032000 tx=0x7f470402d000 comp rx=0 tx=0).stop 2024-09-13T17:58:30.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.259+0000 7f471a908640 1 -- 172.21.15.111:0/3269478729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47141528e0 msgr2=0x7f4714153110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.259+0000 7f471a908640 1 --2- 172.21.15.111:0/3269478729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47141528e0 0x7f4714153110 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00e440 tx=0x7f46fc00e900 comp rx=0 tx=0).stop 2024-09-13T17:58:30.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.259+0000 7f471a908640 1 -- 172.21.15.111:0/3269478729 shutdown_connections 2024-09-13T17:58:30.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.259+0000 7f471a908640 1 --2- 172.21.15.111:0/3269478729 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f46f406ac30 0x7f46f406d0f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.259+0000 7f471a908640 1 --2- 172.21.15.111:0/3269478729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4714157a70 0x7f4714157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.259+0000 7f471a908640 1 --2- 172.21.15.111:0/3269478729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47141528e0 0x7f4714153110 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.259+0000 7f471a908640 1 --2- 172.21.15.111:0/3269478729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47000039c0 0x7f4714155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.259+0000 7f471a908640 1 -- 172.21.15.111:0/3269478729 >> 172.21.15.111:0/3269478729 conn(0x7f471406a0b0 msgr2=0x7f47140ae310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:30.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.259+0000 7f471a908640 1 -- 172.21.15.111:0/3269478729 shutdown_connections 2024-09-13T17:58:30.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.259+0000 7f471a908640 1 -- 172.21.15.111:0/3269478729 wait complete. 2024-09-13T17:58:30.272 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:58:30.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.371+0000 7f8548165640 1 Processor -- start 2024-09-13T17:58:30.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.371+0000 7f8548165640 1 -- start start 2024-09-13T17:58:30.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.372+0000 7f8548165640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85400adc30 0x7f85400cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.372+0000 7f8548165640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f85400ccb40 0x7f854015dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.372+0000 7f8548165640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8540070560 con 0x7f854013da60 2024-09-13T17:58:30.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.372+0000 7f8548165640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85400ae010 con 0x7f85400ccb40 2024-09-13T17:58:30.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.372+0000 7f8548165640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8540070800 con 0x7f85400adc30 2024-09-13T17:58:30.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.372+0000 7f8546702640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f85400ccb40 0x7f854015dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.372+0000 7f8546702640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f85400ccb40 0x7f854015dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:42236/0 (socket says 172.21.15.111:42236) 2024-09-13T17:58:30.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.372+0000 7f8546702640 1 -- 172.21.15.111:0/3953354473 learned_addr learned my addr 172.21.15.111:0/3953354473 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:30.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.372+0000 7f8545f01640 1 --2- 172.21.15.111:0/3953354473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85400adc30 0x7f85400cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.372+0000 7f8545700640 1 --1- 172.21.15.111:0/3953354473 >> v1:172.21.15.188:6789/0 conn(0x7f854013da60 0x7f85400cc430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:51502/0 (socket says 172.21.15.111:51502) 2024-09-13T17:58:30.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.372+0000 7f8546702640 1 -- 172.21.15.111:0/3953354473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85400adc30 msgr2=0x7f85400cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:30.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.372+0000 7f8546702640 1 --2- 172.21.15.111:0/3953354473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85400adc30 0x7f85400cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.372+0000 7f8546702640 1 -- 172.21.15.111:0/3953354473 >> v1:172.21.15.188:6789/0 conn(0x7f854013da60 legacy=0x7f85400cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.372+0000 7f8544eff640 1 -- 172.21.15.111:0/3953354473 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783455219 0 0) 0x7f8540070560 con 0x7f854013da60 2024-09-13T17:58:30.375 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.373+0000 7f8546702640 1 -- 172.21.15.111:0/3953354473 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8540070ae0 con 0x7f85400ccb40 2024-09-13T17:58:30.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.373+0000 7f8546702640 1 --2- 172.21.15.111:0/3953354473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85400ccb40 0x7f854015dea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8538009e50 tx=0x7f853802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c61fa60f773ba75 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:30.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.373+0000 7f8544eff640 1 -- 172.21.15.111:0/3953354473 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538031600 con 0x7f85400ccb40 2024-09-13T17:58:30.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.373+0000 7f8544eff640 1 -- 172.21.15.111:0/3953354473 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853802d050 con 0x7f85400ccb40 2024-09-13T17:58:30.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.373+0000 7f8544eff640 1 -- 172.21.15.111:0/3953354473 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538031980 con 0x7f85400ccb40 2024-09-13T17:58:30.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.374+0000 7f8548165640 1 -- 172.21.15.111:0/3953354473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85400ccb40 msgr2=0x7f854015dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.374+0000 7f8548165640 1 --2- 172.21.15.111:0/3953354473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85400ccb40 0x7f854015dea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8538009e50 tx=0x7f853802a710 comp rx=0 tx=0).stop 2024-09-13T17:58:30.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.374+0000 7f8548165640 1 -- 172.21.15.111:0/3953354473 shutdown_connections 2024-09-13T17:58:30.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.374+0000 7f8548165640 1 --2- 172.21.15.111:0/3953354473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85400ccb40 0x7f854015dea0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.374+0000 7f8548165640 1 --2- 172.21.15.111:0/3953354473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85400adc30 0x7f85400cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.374+0000 7f8548165640 1 -- 172.21.15.111:0/3953354473 >> 172.21.15.111:0/3953354473 conn(0x7f8540065ff0 msgr2=0x7f85400b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:30.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.374+0000 7f8548165640 1 -- 172.21.15.111:0/3953354473 shutdown_connections 2024-09-13T17:58:30.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.374+0000 7f8548165640 1 -- 172.21.15.111:0/3953354473 wait complete. 2024-09-13T17:58:30.379 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.375+0000 7f8548165640 1 Processor -- start 2024-09-13T17:58:30.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.375+0000 7f8548165640 1 -- start start 2024-09-13T17:58:30.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8548165640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85400adc30 0x7f854013c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8548165640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85400ccb40 0x7f85401437d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8548165640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f854013da60 0x7f8540156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8548165640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85401410e0 con 0x7f85400adc30 2024-09-13T17:58:30.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8548165640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8540155a70 con 0x7f85400ccb40 2024-09-13T17:58:30.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8548165640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8540155d50 con 0x7f854013da60 2024-09-13T17:58:30.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8545f01640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85400adc30 0x7f854013c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8545f01640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85400adc30 0x7f854013c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34208/0 (socket says 172.21.15.111:34208) 2024-09-13T17:58:30.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8545f01640 1 -- 172.21.15.111:0/2361681342 learned_addr learned my addr 172.21.15.111:0/2361681342 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:30.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8545700640 1 --2- 172.21.15.111:0/2361681342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85400ccb40 0x7f85401437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8546702640 1 --2- 172.21.15.111:0/2361681342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f854013da60 0x7f8540156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8545f01640 1 -- 172.21.15.111:0/2361681342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f854013da60 msgr2=0x7f8540156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8545f01640 1 --2- 172.21.15.111:0/2361681342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f854013da60 0x7f8540156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8545f01640 1 -- 172.21.15.111:0/2361681342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85400ccb40 msgr2=0x7f85401437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8545f01640 1 --2- 172.21.15.111:0/2361681342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85400ccb40 0x7f85401437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8545f01640 1 -- 172.21.15.111:0/2361681342 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8538009b30 con 0x7f85400adc30 2024-09-13T17:58:30.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8545700640 1 --2- 172.21.15.111:0/2361681342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85400ccb40 0x7f85401437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:30.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.376+0000 7f8546702640 1 --2- 172.21.15.111:0/2361681342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f854013da60 0x7f8540156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:30.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.377+0000 7f8545f01640 1 --2- 172.21.15.111:0/2361681342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85400adc30 0x7f854013c3b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f853c00e930 tx=0x7f853c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:30.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.377+0000 7f85367fc640 1 -- 172.21.15.111:0/2361681342 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c009690 con 0x7f85400adc30 2024-09-13T17:58:30.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.377+0000 7f85367fc640 1 -- 172.21.15.111:0/2361681342 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853c00ca00 con 0x7f85400adc30 2024-09-13T17:58:30.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.377+0000 7f8548165640 1 -- 172.21.15.111:0/2361681342 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8540156030 con 0x7f85400adc30 2024-09-13T17:58:30.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.377+0000 7f85367fc640 1 -- 172.21.15.111:0/2361681342 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c00cdb0 con 0x7f85400adc30 2024-09-13T17:58:30.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.377+0000 7f8548165640 1 -- 172.21.15.111:0/2361681342 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f854013ceb0 con 0x7f85400adc30 2024-09-13T17:58:30.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.378+0000 7f8548165640 1 -- 172.21.15.111:0/2361681342 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8540141250 con 0x7f85400adc30 2024-09-13T17:58:30.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.378+0000 7f85367fc640 1 -- 172.21.15.111:0/2361681342 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f853c013470 con 0x7f85400adc30 2024-09-13T17:58:30.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.379+0000 7f85367fc640 1 --2- 172.21.15.111:0/2361681342 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f852406ac00 0x7f852406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.379+0000 7f8545700640 1 --2- 172.21.15.111:0/2361681342 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f852406ac00 0x7f852406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.379+0000 7f85367fc640 1 -- 172.21.15.111:0/2361681342 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f853c089a90 con 0x7f85400adc30 2024-09-13T17:58:30.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.379+0000 7f8545700640 1 --2- 172.21.15.111:0/2361681342 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f852406ac00 0x7f852406d0c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f854013e2b0 tx=0x7f8530007ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:30.385 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.383+0000 7f85367fc640 1 -- 172.21.15.111:0/2361681342 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f853c0548e0 con 0x7f85400adc30 2024-09-13T17:58:30.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.546+0000 7f8548165640 1 -- 172.21.15.111:0/2361681342 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f854006ae30 con 0x7f85400adc30 2024-09-13T17:58:30.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.546+0000 7f85367fc640 1 -- 172.21.15.111:0/2361681342 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f853c0210a0 con 0x7f85400adc30 2024-09-13T17:58:30.548 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:58:30.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.548+0000 7f8548165640 1 -- 172.21.15.111:0/2361681342 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f852406ac00 msgr2=0x7f852406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.548+0000 7f8548165640 1 --2- 172.21.15.111:0/2361681342 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f852406ac00 0x7f852406d0c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f854013e2b0 tx=0x7f8530007ca0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.548+0000 7f8548165640 1 -- 172.21.15.111:0/2361681342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85400adc30 msgr2=0x7f854013c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.548+0000 7f8548165640 1 --2- 172.21.15.111:0/2361681342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85400adc30 0x7f854013c3b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f853c00e930 tx=0x7f853c00edf0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.548+0000 7f8548165640 1 -- 172.21.15.111:0/2361681342 shutdown_connections 2024-09-13T17:58:30.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.548+0000 7f8548165640 1 --2- 172.21.15.111:0/2361681342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f854013da60 0x7f8540156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.548+0000 7f8548165640 1 --2- 172.21.15.111:0/2361681342 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f852406ac00 0x7f852406d0c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.548+0000 7f8548165640 1 --2- 172.21.15.111:0/2361681342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85400ccb40 0x7f85401437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.548+0000 7f8548165640 1 --2- 172.21.15.111:0/2361681342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85400adc30 0x7f854013c3b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.548+0000 7f8548165640 1 -- 172.21.15.111:0/2361681342 >> 172.21.15.111:0/2361681342 conn(0x7f8540065ff0 msgr2=0x7f8540069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:30.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.549+0000 7f8548165640 1 -- 172.21.15.111:0/2361681342 shutdown_connections 2024-09-13T17:58:30.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.549+0000 7f8548165640 1 -- 172.21.15.111:0/2361681342 wait complete. 2024-09-13T17:58:30.561 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:58:30.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.659+0000 7ff449896640 1 Processor -- start 2024-09-13T17:58:30.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.659+0000 7ff449896640 1 -- start start 2024-09-13T17:58:30.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.660+0000 7ff449896640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff4440cb6d0 0x7ff444160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.660+0000 7ff449896640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff444160570 0x7ff4440b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.660+0000 7ff449896640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44415d310 con 0x7ff4440ccd90 2024-09-13T17:58:30.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.660+0000 7ff449896640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44415ca40 con 0x7ff444160570 2024-09-13T17:58:30.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.660+0000 7ff449896640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff444160d20 con 0x7ff4440cb6d0 2024-09-13T17:58:30.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.660+0000 7ff4437fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff444160570 0x7ff4440b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.660+0000 7ff4437fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff444160570 0x7ff4440b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34218/0 (socket says 172.21.15.111:34218) 2024-09-13T17:58:30.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.660+0000 7ff4437fe640 1 -- 172.21.15.111:0/2998555859 learned_addr learned my addr 172.21.15.111:0/2998555859 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:30.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.661+0000 7ff43a7fc640 1 --2- 172.21.15.111:0/2998555859 >> v2:172.21.15.188:3300/0 conn(0x7ff4440cb6d0 0x7ff444160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.661+0000 7ff4437fe640 1 -- 172.21.15.111:0/2998555859 >> v2:172.21.15.188:3300/0 conn(0x7ff4440cb6d0 msgr2=0x7ff444160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:30.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.661+0000 7ff4437fe640 1 --2- 172.21.15.111:0/2998555859 >> v2:172.21.15.188:3300/0 conn(0x7ff4440cb6d0 0x7ff444160030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.661+0000 7ff4437fe640 1 -- 172.21.15.111:0/2998555859 >> v1:172.21.15.188:6789/0 conn(0x7ff4440ccd90 legacy=0x7ff4440cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:30.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.661+0000 7ff4437fe640 1 -- 172.21.15.111:0/2998555859 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44415ccc0 con 0x7ff444160570 2024-09-13T17:58:30.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.661+0000 7ff4437fe640 1 --2- 172.21.15.111:0/2998555859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff444160570 0x7ff4440b1a70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff4440b3360 tx=0x7ff43c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7032cc3fed3585d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:30.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.661+0000 7ff4427fc640 1 -- 172.21.15.111:0/2998555859 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c031600 con 0x7ff444160570 2024-09-13T17:58:30.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.662+0000 7ff4427fc640 1 -- 172.21.15.111:0/2998555859 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43c02d050 con 0x7ff444160570 2024-09-13T17:58:30.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.662+0000 7ff4427fc640 1 -- 172.21.15.111:0/2998555859 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c031980 con 0x7ff444160570 2024-09-13T17:58:30.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.662+0000 7ff449896640 1 -- 172.21.15.111:0/2998555859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff444160570 msgr2=0x7ff4440b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.662+0000 7ff449896640 1 --2- 172.21.15.111:0/2998555859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff444160570 0x7ff4440b1a70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff4440b3360 tx=0x7ff43c02a6f0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.662+0000 7ff449896640 1 -- 172.21.15.111:0/2998555859 shutdown_connections 2024-09-13T17:58:30.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.662+0000 7ff449896640 1 --2- 172.21.15.111:0/2998555859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff444160570 0x7ff4440b1a70 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.662+0000 7ff449896640 1 --2- 172.21.15.111:0/2998555859 >> v2:172.21.15.188:3300/0 conn(0x7ff4440cb6d0 0x7ff444160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.662+0000 7ff449896640 1 -- 172.21.15.111:0/2998555859 >> 172.21.15.111:0/2998555859 conn(0x7ff444065ff0 msgr2=0x7ff4440b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:30.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.662+0000 7ff449896640 1 -- 172.21.15.111:0/2998555859 shutdown_connections 2024-09-13T17:58:30.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.662+0000 7ff449896640 1 -- 172.21.15.111:0/2998555859 wait complete. 2024-09-13T17:58:30.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.663+0000 7ff449896640 1 Processor -- start 2024-09-13T17:58:30.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.664+0000 7ff449896640 1 -- start start 2024-09-13T17:58:30.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.664+0000 7ff449896640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4440cb6d0 0x7ff444142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.664+0000 7ff449896640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4440ccd90 0x7ff444158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.664+0000 7ff449896640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff444160570 0x7ff444150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.664+0000 7ff449896640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff444151eb0 con 0x7ff4440ccd90 2024-09-13T17:58:30.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.664+0000 7ff449896640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff444151230 con 0x7ff4440cb6d0 2024-09-13T17:58:30.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.664+0000 7ff449896640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff444151510 con 0x7ff444160570 2024-09-13T17:58:30.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.664+0000 7ff43a7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4440ccd90 0x7ff444158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.664+0000 7ff43a7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4440ccd90 0x7ff444158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34234/0 (socket says 172.21.15.111:34234) 2024-09-13T17:58:30.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.664+0000 7ff43a7fc640 1 -- 172.21.15.111:0/4216296629 learned_addr learned my addr 172.21.15.111:0/4216296629 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:30.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.665+0000 7ff4437fe640 1 --2- 172.21.15.111:0/4216296629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff444160570 0x7ff444150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.665+0000 7ff442ffd640 1 --2- 172.21.15.111:0/4216296629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4440cb6d0 0x7ff444142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.665+0000 7ff43a7fc640 1 -- 172.21.15.111:0/4216296629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff444160570 msgr2=0x7ff444150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.665+0000 7ff43a7fc640 1 --2- 172.21.15.111:0/4216296629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff444160570 0x7ff444150780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.665+0000 7ff43a7fc640 1 -- 172.21.15.111:0/4216296629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4440cb6d0 msgr2=0x7ff444142070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.665+0000 7ff43a7fc640 1 --2- 172.21.15.111:0/4216296629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4440cb6d0 0x7ff444142070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.665+0000 7ff43a7fc640 1 -- 172.21.15.111:0/4216296629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c009b30 con 0x7ff4440ccd90 2024-09-13T17:58:30.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.665+0000 7ff4437fe640 1 --2- 172.21.15.111:0/4216296629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff444160570 0x7ff444150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:30.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.665+0000 7ff442ffd640 1 --2- 172.21.15.111:0/4216296629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4440cb6d0 0x7ff444142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:30.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.665+0000 7ff43a7fc640 1 --2- 172.21.15.111:0/4216296629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4440ccd90 0x7ff444158f70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00d800 tx=0x7ff42c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:30.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.665+0000 7ff440ff9640 1 -- 172.21.15.111:0/4216296629 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c00b6c0 con 0x7ff4440ccd90 2024-09-13T17:58:30.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.665+0000 7ff440ff9640 1 -- 172.21.15.111:0/4216296629 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42c00b820 con 0x7ff4440ccd90 2024-09-13T17:58:30.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.665+0000 7ff449896640 1 -- 172.21.15.111:0/4216296629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4441517f0 con 0x7ff4440ccd90 2024-09-13T17:58:30.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.665+0000 7ff440ff9640 1 -- 172.21.15.111:0/4216296629 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c00bbb0 con 0x7ff4440ccd90 2024-09-13T17:58:30.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.665+0000 7ff449896640 1 -- 172.21.15.111:0/4216296629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4441594b0 con 0x7ff4440ccd90 2024-09-13T17:58:30.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.666+0000 7ff449896640 1 -- 172.21.15.111:0/4216296629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4441609b0 con 0x7ff4440ccd90 2024-09-13T17:58:30.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.666+0000 7ff440ff9640 1 -- 172.21.15.111:0/4216296629 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff42c00bd10 con 0x7ff4440ccd90 2024-09-13T17:58:30.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.667+0000 7ff440ff9640 1 --2- 172.21.15.111:0/4216296629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff42406ac30 0x7ff42406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.667+0000 7ff440ff9640 1 -- 172.21.15.111:0/4216296629 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff42c089bc0 con 0x7ff4440ccd90 2024-09-13T17:58:30.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.667+0000 7ff442ffd640 1 --2- 172.21.15.111:0/4216296629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff42406ac30 0x7ff42406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.667+0000 7ff442ffd640 1 --2- 172.21.15.111:0/4216296629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff42406ac30 0x7ff42406d0f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff4440b2900 tx=0x7ff434009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:30.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.671+0000 7ff440ff9640 1 -- 172.21.15.111:0/4216296629 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff4441609b0 con 0x7ff4440ccd90 2024-09-13T17:58:30.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.832+0000 7ff449896640 1 -- 172.21.15.111:0/4216296629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff4441609b0 con 0x7ff4440ccd90 2024-09-13T17:58:30.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.832+0000 7ff440ff9640 1 -- 172.21.15.111:0/4216296629 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff4441609b0 con 0x7ff4440ccd90 2024-09-13T17:58:30.834 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:58:30.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.834+0000 7ff449896640 1 -- 172.21.15.111:0/4216296629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff42406ac30 msgr2=0x7ff42406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.834+0000 7ff449896640 1 --2- 172.21.15.111:0/4216296629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff42406ac30 0x7ff42406d0f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff4440b2900 tx=0x7ff434009040 comp rx=0 tx=0).stop 2024-09-13T17:58:30.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.834+0000 7ff449896640 1 -- 172.21.15.111:0/4216296629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4440ccd90 msgr2=0x7ff444158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.834+0000 7ff449896640 1 --2- 172.21.15.111:0/4216296629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4440ccd90 0x7ff444158f70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00d800 tx=0x7ff42c00dcc0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.834+0000 7ff449896640 1 -- 172.21.15.111:0/4216296629 shutdown_connections 2024-09-13T17:58:30.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.834+0000 7ff449896640 1 --2- 172.21.15.111:0/4216296629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff444160570 0x7ff444150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.834+0000 7ff449896640 1 --2- 172.21.15.111:0/4216296629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4440ccd90 0x7ff444158f70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.834+0000 7ff449896640 1 --2- 172.21.15.111:0/4216296629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff42406ac30 0x7ff42406d0f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.834+0000 7ff449896640 1 --2- 172.21.15.111:0/4216296629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4440cb6d0 0x7ff444142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.834+0000 7ff449896640 1 -- 172.21.15.111:0/4216296629 >> 172.21.15.111:0/4216296629 conn(0x7ff444065ff0 msgr2=0x7ff4440b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:30.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.834+0000 7ff449896640 1 -- 172.21.15.111:0/4216296629 shutdown_connections 2024-09-13T17:58:30.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.834+0000 7ff449896640 1 -- 172.21.15.111:0/4216296629 wait complete. 2024-09-13T17:58:30.847 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:58:30.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.942+0000 7ffba6b95640 1 Processor -- start 2024-09-13T17:58:30.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.942+0000 7ffba6b95640 1 -- start start 2024-09-13T17:58:30.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.943+0000 7ffba6b95640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffba00cb150 0x7ffba006d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.943+0000 7ffba6b95640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba00c8920 con 0x7ffba00cace0 2024-09-13T17:58:30.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.943+0000 7ffba6b95640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba015a6c0 con 0x7ffba015a2e0 2024-09-13T17:58:30.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.943+0000 7ffba6b95640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba00ca500 con 0x7ffba00cb150 2024-09-13T17:58:30.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.943+0000 7ffb97fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ffba00cace0 0x7ffba006d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56048/0 (socket says 172.21.15.111:56048) 2024-09-13T17:58:30.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.943+0000 7ffb97fff640 1 -- 172.21.15.111:0/2261543241 learned_addr learned my addr 172.21.15.111:0/2261543241 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:30.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.944+0000 7ffba5132640 1 --2- 172.21.15.111:0/2261543241 >> v2:172.21.15.188:3300/0 conn(0x7ffba00cb150 0x7ffba006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.944+0000 7ffb977fe640 1 -- 172.21.15.111:0/2261543241 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710519704 0 0) 0x7ffba00c8920 con 0x7ffba00cace0 2024-09-13T17:58:30.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.944+0000 7ffb977fe640 1 -- 172.21.15.111:0/2261543241 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb88003520 con 0x7ffba00cace0 2024-09-13T17:58:30.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.944+0000 7ffb977fe640 1 -- 172.21.15.111:0/2261543241 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2814880933 0 0) 0x7ffb88003520 con 0x7ffba00cace0 2024-09-13T17:58:30.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.944+0000 7ffba5132640 1 -- 172.21.15.111:0/2261543241 >> v1:172.21.15.188:6789/0 conn(0x7ffba015a2e0 legacy=0x7ffba00c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.944+0000 7ffba5132640 1 -- 172.21.15.111:0/2261543241 >> v1:172.21.15.111:6789/0 conn(0x7ffba00cace0 legacy=0x7ffba006d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.944+0000 7ffba5132640 1 -- 172.21.15.111:0/2261543241 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba00c8b70 con 0x7ffba00cb150 2024-09-13T17:58:30.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.945+0000 7ffba5132640 1 --2- 172.21.15.111:0/2261543241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba00cb150 0x7ffba006d740 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009d90 tx=0x7ffb9802a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=607ba9c9aba2dbf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:30.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.945+0000 7ffb977fe640 1 -- 172.21.15.111:0/2261543241 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90001670 con 0x7ffba00cb150 2024-09-13T17:58:30.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.945+0000 7ffb977fe640 1 -- 172.21.15.111:0/2261543241 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9802d050 con 0x7ffba00cb150 2024-09-13T17:58:30.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.945+0000 7ffb977fe640 1 -- 172.21.15.111:0/2261543241 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98007740 con 0x7ffba00cb150 2024-09-13T17:58:30.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.945+0000 7ffba6b95640 1 -- 172.21.15.111:0/2261543241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba00cb150 msgr2=0x7ffba006d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.945+0000 7ffba6b95640 1 --2- 172.21.15.111:0/2261543241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba00cb150 0x7ffba006d740 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009d90 tx=0x7ffb9802a5b0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.945+0000 7ffba6b95640 1 -- 172.21.15.111:0/2261543241 shutdown_connections 2024-09-13T17:58:30.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.945+0000 7ffba6b95640 1 --2- 172.21.15.111:0/2261543241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba00cb150 0x7ffba006d740 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.945+0000 7ffba6b95640 1 -- 172.21.15.111:0/2261543241 >> 172.21.15.111:0/2261543241 conn(0x7ffba0069ed0 msgr2=0x7ffba006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:30.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.946+0000 7ffba6b95640 1 -- 172.21.15.111:0/2261543241 shutdown_connections 2024-09-13T17:58:30.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.946+0000 7ffba6b95640 1 -- 172.21.15.111:0/2261543241 wait complete. 2024-09-13T17:58:30.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.947+0000 7ffba6b95640 1 Processor -- start 2024-09-13T17:58:30.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.947+0000 7ffba6b95640 1 -- start start 2024-09-13T17:58:30.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffba6b95640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba00cace0 0x7ffba0154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffba6b95640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffba00cb150 0x7ffba00c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffba6b95640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba015a2e0 0x7ffba015cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffba6b95640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba015c6a0 con 0x7ffba00cb150 2024-09-13T17:58:30.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffba6b95640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba0154940 con 0x7ffba015a2e0 2024-09-13T17:58:30.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffba6b95640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba015ba90 con 0x7ffba00cace0 2024-09-13T17:58:30.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffb97fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffba00cb150 0x7ffba00c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffb97fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffba00cb150 0x7ffba00c7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34236/0 (socket says 172.21.15.111:34236) 2024-09-13T17:58:30.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffb97fff640 1 -- 172.21.15.111:0/3640693815 learned_addr learned my addr 172.21.15.111:0/3640693815 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:30.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffba5132640 1 --2- 172.21.15.111:0/3640693815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba015a2e0 0x7ffba015cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffba4931640 1 --2- 172.21.15.111:0/3640693815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba00cace0 0x7ffba0154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffb97fff640 1 -- 172.21.15.111:0/3640693815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba00cace0 msgr2=0x7ffba0154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffb97fff640 1 --2- 172.21.15.111:0/3640693815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba00cace0 0x7ffba0154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffb97fff640 1 -- 172.21.15.111:0/3640693815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba015a2e0 msgr2=0x7ffba015cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffb97fff640 1 --2- 172.21.15.111:0/3640693815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba015a2e0 0x7ffba015cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffb97fff640 1 -- 172.21.15.111:0/3640693815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb98009a70 con 0x7ffba00cb150 2024-09-13T17:58:30.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffb97fff640 1 --2- 172.21.15.111:0/3640693815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffba00cb150 0x7ffba00c7dc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000c780 tx=0x7ffb9000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:30.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffb957fa640 1 -- 172.21.15.111:0/3640693815 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb900072d0 con 0x7ffba00cb150 2024-09-13T17:58:30.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffb957fa640 1 -- 172.21.15.111:0/3640693815 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb90007430 con 0x7ffba00cb150 2024-09-13T17:58:30.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffba6b95640 1 -- 172.21.15.111:0/3640693815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba015bd70 con 0x7ffba00cb150 2024-09-13T17:58:30.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffb957fa640 1 -- 172.21.15.111:0/3640693815 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb900047e0 con 0x7ffba00cb150 2024-09-13T17:58:30.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.948+0000 7ffba6b95640 1 -- 172.21.15.111:0/3640693815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba015c330 con 0x7ffba00cb150 2024-09-13T17:58:30.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.950+0000 7ffba6b95640 1 -- 172.21.15.111:0/3640693815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba015c850 con 0x7ffba00cb150 2024-09-13T17:58:30.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.950+0000 7ffb957fa640 1 -- 172.21.15.111:0/3640693815 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ffb90004940 con 0x7ffba00cb150 2024-09-13T17:58:30.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.950+0000 7ffb957fa640 1 --2- 172.21.15.111:0/3640693815 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb8c06ac00 0x7ffb8c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:30.955 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.950+0000 7ffb957fa640 1 -- 172.21.15.111:0/3640693815 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ffb900889f0 con 0x7ffba00cb150 2024-09-13T17:58:30.955 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.950+0000 7ffba4931640 1 --2- 172.21.15.111:0/3640693815 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb8c06ac00 0x7ffb8c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:30.955 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.951+0000 7ffba4931640 1 --2- 172.21.15.111:0/3640693815 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb8c06ac00 0x7ffb8c06d0c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c000f60 tx=0x7ffb9c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:30.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:30.954+0000 7ffb957fa640 1 -- 172.21.15.111:0/3640693815 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ffba015c850 con 0x7ffba00cb150 2024-09-13T17:58:31.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.149+0000 7ffba6b95640 1 -- 172.21.15.111:0/3640693815 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffba00c9630 con 0x7ffba00cb150 2024-09-13T17:58:31.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.150+0000 7ffb957fa640 1 -- 172.21.15.111:0/3640693815 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4742 (secure 0 0 0) 0x7ffb90053840 con 0x7ffba00cb150 2024-09-13T17:58:31.152 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:31.152 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4001406804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":4001406804},{"type":"v1","addr":"172.21.15.188:6835","nonce":4001406804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2565466103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2565466103},{"type":"v1","addr":"172.21.15.111:6837","nonce":2565466103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/1252690799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1252690799},{"type":"v1","addr":"172.21.15.111:6836","nonce":1252690799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:57:25.702997+0000","modified":"2024-09-13T17:57:46.351493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/1266310843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1266310843},{"type":"v1","addr":"172.21.15.188:6834","nonce":1266310843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:58:31.152 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-09-13T17:58:31.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.152+0000 7ffba6b95640 1 -- 172.21.15.111:0/3640693815 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb8c06ac00 msgr2=0x7ffb8c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.152+0000 7ffba6b95640 1 --2- 172.21.15.111:0/3640693815 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb8c06ac00 0x7ffb8c06d0c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c000f60 tx=0x7ffb9c009040 comp rx=0 tx=0).stop 2024-09-13T17:58:31.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.152+0000 7ffba6b95640 1 -- 172.21.15.111:0/3640693815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffba00cb150 msgr2=0x7ffba00c7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.152+0000 7ffba6b95640 1 --2- 172.21.15.111:0/3640693815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffba00cb150 0x7ffba00c7dc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000c780 tx=0x7ffb9000cc40 comp rx=0 tx=0).stop 2024-09-13T17:58:31.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.152+0000 7ffba6b95640 1 -- 172.21.15.111:0/3640693815 shutdown_connections 2024-09-13T17:58:31.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.152+0000 7ffba6b95640 1 --2- 172.21.15.111:0/3640693815 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffb8c06ac00 0x7ffb8c06d0c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.152+0000 7ffba6b95640 1 --2- 172.21.15.111:0/3640693815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba015a2e0 0x7ffba015cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.152+0000 7ffba6b95640 1 --2- 172.21.15.111:0/3640693815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffba00cb150 0x7ffba00c7dc0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.152+0000 7ffba6b95640 1 --2- 172.21.15.111:0/3640693815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba00cace0 0x7ffba0154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.152+0000 7ffba6b95640 1 -- 172.21.15.111:0/3640693815 >> 172.21.15.111:0/3640693815 conn(0x7ffba0069ed0 msgr2=0x7ffba0070fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:31.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.152+0000 7ffba6b95640 1 -- 172.21.15.111:0/3640693815 shutdown_connections 2024-09-13T17:58:31.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.152+0000 7ffba6b95640 1 -- 172.21.15.111:0/3640693815 wait complete. 2024-09-13T17:58:31.165 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:31.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.261+0000 7ff3ea0e5640 1 Processor -- start 2024-09-13T17:58:31.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.261+0000 7ff3ea0e5640 1 -- start start 2024-09-13T17:58:31.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.262+0000 7ff3ea0e5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3e40cace0 0x7ff3e40cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.262+0000 7ff3ea0e5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e40c8920 con 0x7ff3e415a2e0 2024-09-13T17:58:31.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.262+0000 7ff3ea0e5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e415a6c0 con 0x7ff3e406d030 2024-09-13T17:58:31.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.262+0000 7ff3ea0e5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e40cb690 con 0x7ff3e40cace0 2024-09-13T17:58:31.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.262+0000 7ff3e37fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff3e415a2e0 0x7ff3e40c9a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56064/0 (socket says 172.21.15.111:56064) 2024-09-13T17:58:31.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.262+0000 7ff3e37fe640 1 -- 172.21.15.111:0/381717383 learned_addr learned my addr 172.21.15.111:0/381717383 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:31.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.262+0000 7ff3e2ffd640 1 --2- 172.21.15.111:0/381717383 >> v2:172.21.15.188:3300/0 conn(0x7ff3e40cace0 0x7ff3e40cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:31.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e27fc640 1 -- 172.21.15.111:0/381717383 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207556075 0 0) 0x7ff3e40c8920 con 0x7ff3e415a2e0 2024-09-13T17:58:31.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e27fc640 1 -- 172.21.15.111:0/381717383 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3d0003520 con 0x7ff3e415a2e0 2024-09-13T17:58:31.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e27fc640 1 -- 172.21.15.111:0/381717383 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311063764 0 0) 0x7ff3e415a6c0 con 0x7ff3e406d030 2024-09-13T17:58:31.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e27fc640 1 -- 172.21.15.111:0/381717383 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3e40c8920 con 0x7ff3e406d030 2024-09-13T17:58:31.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e27fc640 1 -- 172.21.15.111:0/381717383 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2636371758 0 0) 0x7ff3d0003520 con 0x7ff3e415a2e0 2024-09-13T17:58:31.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e27fc640 1 -- 172.21.15.111:0/381717383 >> v2:172.21.15.188:3300/0 conn(0x7ff3e40cace0 msgr2=0x7ff3e40cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:31.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e27fc640 1 --2- 172.21.15.111:0/381717383 >> v2:172.21.15.188:3300/0 conn(0x7ff3e40cace0 0x7ff3e40cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e27fc640 1 -- 172.21.15.111:0/381717383 >> v1:172.21.15.188:6789/0 conn(0x7ff3e406d030 legacy=0x7ff3e406d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e27fc640 1 -- 172.21.15.111:0/381717383 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e40ca500 con 0x7ff3e415a2e0 2024-09-13T17:58:31.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e27fc640 1 -- 172.21.15.111:0/381717383 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7ff3d4001670 con 0x7ff3e415a2e0 2024-09-13T17:58:31.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e27fc640 1 -- 172.21.15.111:0/381717383 >> v1:172.21.15.111:6789/0 conn(0x7ff3e415a2e0 legacy=0x7ff3e40c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e27fc640 1 --2- 172.21.15.111:0/381717383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d0003b50 0x7ff3d0003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e27fc640 1 --2- 172.21.15.111:0/381717383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d00047c0 0x7ff3d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e27fc640 1 --2- 172.21.15.111:0/381717383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d0005180 0x7ff3d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e27fc640 1 -- 172.21.15.111:0/381717383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e40cb690 con 0x7ff3d0005180 2024-09-13T17:58:31.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e27fc640 1 -- 172.21.15.111:0/381717383 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d000c050 con 0x7ff3d00047c0 2024-09-13T17:58:31.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e27fc640 1 -- 172.21.15.111:0/381717383 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d000c210 con 0x7ff3d0003b50 2024-09-13T17:58:31.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e3fff640 1 --2- 172.21.15.111:0/381717383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d0005180 0x7ff3d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:31.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e37fe640 1 --2- 172.21.15.111:0/381717383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d0003b50 0x7ff3d0003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:31.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e2ffd640 1 --2- 172.21.15.111:0/381717383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d00047c0 0x7ff3d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:31.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e3fff640 1 -- 172.21.15.111:0/381717383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d0003b50 msgr2=0x7ff3d0003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:31.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e3fff640 1 --2- 172.21.15.111:0/381717383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d0003b50 0x7ff3d0003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e3fff640 1 -- 172.21.15.111:0/381717383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d00047c0 msgr2=0x7ff3d0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:31.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e3fff640 1 --2- 172.21.15.111:0/381717383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d00047c0 0x7ff3d0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.263+0000 7ff3e3fff640 1 -- 172.21.15.111:0/381717383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d0003520 con 0x7ff3d0005180 2024-09-13T17:58:31.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.264+0000 7ff3e3fff640 1 -- 172.21.15.111:0/381717383 reap_dead start 2024-09-13T17:58:31.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.264+0000 7ff3e3fff640 1 --2- 172.21.15.111:0/381717383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d0005180 0x7ff3d000b980 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc001fd0 tx=0x7ff3d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b4ec72594b5c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:31.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.264+0000 7ff3e27fc640 1 -- 172.21.15.111:0/381717383 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc003b70 con 0x7ff3d0005180 2024-09-13T17:58:31.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.264+0000 7ff3e27fc640 1 -- 172.21.15.111:0/381717383 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc02d050 con 0x7ff3d0005180 2024-09-13T17:58:31.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.264+0000 7ff3e27fc640 1 -- 172.21.15.111:0/381717383 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0050d0 con 0x7ff3d0005180 2024-09-13T17:58:31.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.265+0000 7ff3ea0e5640 1 -- 172.21.15.111:0/381717383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d0005180 msgr2=0x7ff3d000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.265+0000 7ff3ea0e5640 1 --2- 172.21.15.111:0/381717383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d0005180 0x7ff3d000b980 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc001fd0 tx=0x7ff3d00057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.265+0000 7ff3ea0e5640 1 -- 172.21.15.111:0/381717383 shutdown_connections 2024-09-13T17:58:31.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.265+0000 7ff3ea0e5640 1 --2- 172.21.15.111:0/381717383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d0005180 0x7ff3d000b980 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.265+0000 7ff3ea0e5640 1 -- 172.21.15.111:0/381717383 >> 172.21.15.111:0/381717383 conn(0x7ff3e4069ed0 msgr2=0x7ff3e406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:31.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.265+0000 7ff3ea0e5640 1 -- 172.21.15.111:0/381717383 shutdown_connections 2024-09-13T17:58:31.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.265+0000 7ff3ea0e5640 1 -- 172.21.15.111:0/381717383 wait complete. 2024-09-13T17:58:31.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.266+0000 7ff3ea0e5640 1 Processor -- start 2024-09-13T17:58:31.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.266+0000 7ff3ea0e5640 1 -- start start 2024-09-13T17:58:31.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.267+0000 7ff3ea0e5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d0005180 0x7ff3e40c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.267+0000 7ff3ea0e5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3e40cbea0 0x7ff3e40c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.267+0000 7ff3ea0e5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e40716e0 0x7ff3e4153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.267+0000 7ff3ea0e5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e40cd480 con 0x7ff3e40716e0 2024-09-13T17:58:31.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.267+0000 7ff3ea0e5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e415bec0 con 0x7ff3d0005180 2024-09-13T17:58:31.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.267+0000 7ff3ea0e5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e40ccec0 con 0x7ff3e40cbea0 2024-09-13T17:58:31.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.267+0000 7ff3e3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e40716e0 0x7ff3e4153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:31.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.267+0000 7ff3e3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e40716e0 0x7ff3e4153d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34252/0 (socket says 172.21.15.111:34252) 2024-09-13T17:58:31.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.267+0000 7ff3e3fff640 1 -- 172.21.15.111:0/1412935279 learned_addr learned my addr 172.21.15.111:0/1412935279 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:31.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.267+0000 7ff3e37fe640 1 --2- 172.21.15.111:0/1412935279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d0005180 0x7ff3e40c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:31.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.267+0000 7ff3e2ffd640 1 --2- 172.21.15.111:0/1412935279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3e40cbea0 0x7ff3e40c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:31.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.267+0000 7ff3e3fff640 1 -- 172.21.15.111:0/1412935279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3e40cbea0 msgr2=0x7ff3e40c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.267+0000 7ff3e3fff640 1 --2- 172.21.15.111:0/1412935279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3e40cbea0 0x7ff3e40c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.267+0000 7ff3e3fff640 1 -- 172.21.15.111:0/1412935279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d0005180 msgr2=0x7ff3e40c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.267+0000 7ff3e3fff640 1 --2- 172.21.15.111:0/1412935279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d0005180 0x7ff3e40c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.267+0000 7ff3e3fff640 1 -- 172.21.15.111:0/1412935279 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e40cd1a0 con 0x7ff3e40716e0 2024-09-13T17:58:31.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.267+0000 7ff3e3fff640 1 --2- 172.21.15.111:0/1412935279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e40716e0 0x7ff3e4153d00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0020f0 tx=0x7ff3dc001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:31.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.268+0000 7ff3dbfff640 1 -- 172.21.15.111:0/1412935279 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0050d0 con 0x7ff3e40716e0 2024-09-13T17:58:31.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.268+0000 7ff3dbfff640 1 -- 172.21.15.111:0/1412935279 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc005250 con 0x7ff3e40716e0 2024-09-13T17:58:31.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.268+0000 7ff3dbfff640 1 -- 172.21.15.111:0/1412935279 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc005600 con 0x7ff3e40716e0 2024-09-13T17:58:31.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.268+0000 7ff3ea0e5640 1 -- 172.21.15.111:0/1412935279 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e4154240 con 0x7ff3e40716e0 2024-09-13T17:58:31.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.268+0000 7ff3ea0e5640 1 -- 172.21.15.111:0/1412935279 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e40c7350 con 0x7ff3e40716e0 2024-09-13T17:58:31.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.269+0000 7ff3ea0e5640 1 -- 172.21.15.111:0/1412935279 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e415bc20 con 0x7ff3e40716e0 2024-09-13T17:58:31.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.269+0000 7ff3dbfff640 1 -- 172.21.15.111:0/1412935279 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff3dc007040 con 0x7ff3e40716e0 2024-09-13T17:58:31.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.269+0000 7ff3dbfff640 1 --2- 172.21.15.111:0/1412935279 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff3c406ac30 0x7ff3c406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.270+0000 7ff3dbfff640 1 -- 172.21.15.111:0/1412935279 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff3dc0a8de0 con 0x7ff3e40716e0 2024-09-13T17:58:31.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.270+0000 7ff3e37fe640 1 --2- 172.21.15.111:0/1412935279 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff3c406ac30 0x7ff3c406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:31.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.270+0000 7ff3e37fe640 1 --2- 172.21.15.111:0/1412935279 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff3c406ac30 0x7ff3c406d0f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4001050 tx=0x7ff3d4001b00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:31.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.274+0000 7ff3dbfff640 1 -- 172.21.15.111:0/1412935279 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff3e415bc20 con 0x7ff3e40716e0 2024-09-13T17:58:31.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.434+0000 7ff3ea0e5640 1 -- 172.21.15.111:0/1412935279 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3e415c000 con 0x7ff3e40716e0 2024-09-13T17:58:31.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.435+0000 7ff3dbfff640 1 -- 172.21.15.111:0/1412935279 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18794 (secure 0 0 0) 0x7ff3e415c000 con 0x7ff3e40716e0 2024-09-13T17:58:31.437 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:31.437 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":26,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:07.749113+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:31.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.437+0000 7ff3ea0e5640 1 -- 172.21.15.111:0/1412935279 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff3c406ac30 msgr2=0x7ff3c406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.437+0000 7ff3ea0e5640 1 --2- 172.21.15.111:0/1412935279 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff3c406ac30 0x7ff3c406d0f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4001050 tx=0x7ff3d4001b00 comp rx=0 tx=0).stop 2024-09-13T17:58:31.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.437+0000 7ff3ea0e5640 1 -- 172.21.15.111:0/1412935279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e40716e0 msgr2=0x7ff3e4153d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.437+0000 7ff3ea0e5640 1 --2- 172.21.15.111:0/1412935279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e40716e0 0x7ff3e4153d00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0020f0 tx=0x7ff3dc001650 comp rx=0 tx=0).stop 2024-09-13T17:58:31.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.437+0000 7ff3ea0e5640 1 -- 172.21.15.111:0/1412935279 shutdown_connections 2024-09-13T17:58:31.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.437+0000 7ff3ea0e5640 1 --2- 172.21.15.111:0/1412935279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e40716e0 0x7ff3e4153d00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.437+0000 7ff3ea0e5640 1 --2- 172.21.15.111:0/1412935279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3e40cbea0 0x7ff3e40c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.437+0000 7ff3ea0e5640 1 --2- 172.21.15.111:0/1412935279 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff3c406ac30 0x7ff3c406d0f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.437+0000 7ff3ea0e5640 1 --2- 172.21.15.111:0/1412935279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d0005180 0x7ff3e40c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.437+0000 7ff3ea0e5640 1 -- 172.21.15.111:0/1412935279 >> 172.21.15.111:0/1412935279 conn(0x7ff3e4069ed0 msgr2=0x7ff3e4070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:31.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.437+0000 7ff3ea0e5640 1 -- 172.21.15.111:0/1412935279 shutdown_connections 2024-09-13T17:58:31.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.437+0000 7ff3ea0e5640 1 -- 172.21.15.111:0/1412935279 wait complete. 2024-09-13T17:58:31.451 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T17:58:31.451 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:58:31.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.547+0000 7f42b5b01640 1 Processor -- start 2024-09-13T17:58:31.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.547+0000 7f42b5b01640 1 -- start start 2024-09-13T17:58:31.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.548+0000 7f42b5b01640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f42b00c9030 0x7f42b0070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.548+0000 7f42b5b01640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b00abdf0 con 0x7f42b0071d30 2024-09-13T17:58:31.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.548+0000 7f42b5b01640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b00b2280 con 0x7f42b0072110 2024-09-13T17:58:31.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.548+0000 7f42b5b01640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b00ab1e0 con 0x7f42b00c9030 2024-09-13T17:58:31.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.548+0000 7f42af7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f42b0071d30 0x7f42b00b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56076/0 (socket says 172.21.15.111:56076) 2024-09-13T17:58:31.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.548+0000 7f42af7fe640 1 -- 172.21.15.111:0/193448286 learned_addr learned my addr 172.21.15.111:0/193448286 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:31.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.548+0000 7f42affff640 1 --2- 172.21.15.111:0/193448286 >> v2:172.21.15.111:3300/0 conn(0x7f42b00c9030 0x7f42b0070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:31.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.548+0000 7f42affff640 1 -- 172.21.15.111:0/193448286 >> v1:172.21.15.188:6789/0 conn(0x7f42b0072110 legacy=0x7f42b00c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:31.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.548+0000 7f42affff640 1 -- 172.21.15.111:0/193448286 >> v1:172.21.15.111:6789/0 conn(0x7f42b0071d30 legacy=0x7f42b00b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.548+0000 7f42affff640 1 -- 172.21.15.111:0/193448286 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b00ab4c0 con 0x7f42b00c9030 2024-09-13T17:58:31.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.549+0000 7f42affff640 1 --2- 172.21.15.111:0/193448286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b00c9030 0x7f42b0070190 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009d90 tx=0x7f42a802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444eb299042537e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:31.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.549+0000 7f42aeffd640 1 -- 172.21.15.111:0/193448286 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a8007740 con 0x7f42b00c9030 2024-09-13T17:58:31.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.549+0000 7f42aeffd640 1 -- 172.21.15.111:0/193448286 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a802d050 con 0x7f42b00c9030 2024-09-13T17:58:31.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.549+0000 7f42aeffd640 1 -- 172.21.15.111:0/193448286 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a8007ac0 con 0x7f42b00c9030 2024-09-13T17:58:31.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.550+0000 7f42b5b01640 1 -- 172.21.15.111:0/193448286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b00c9030 msgr2=0x7f42b0070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.550+0000 7f42b5b01640 1 --2- 172.21.15.111:0/193448286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b00c9030 0x7f42b0070190 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009d90 tx=0x7f42a802a6a0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.550+0000 7f42b5b01640 1 -- 172.21.15.111:0/193448286 shutdown_connections 2024-09-13T17:58:31.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.550+0000 7f42b5b01640 1 --2- 172.21.15.111:0/193448286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b00c9030 0x7f42b0070190 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.550+0000 7f42b5b01640 1 -- 172.21.15.111:0/193448286 >> 172.21.15.111:0/193448286 conn(0x7f42b0069ed0 msgr2=0x7f42b006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:31.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.550+0000 7f42b5b01640 1 -- 172.21.15.111:0/193448286 shutdown_connections 2024-09-13T17:58:31.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.550+0000 7f42b5b01640 1 -- 172.21.15.111:0/193448286 wait complete. 2024-09-13T17:58:31.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.551+0000 7f42b5b01640 1 Processor -- start 2024-09-13T17:58:31.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.551+0000 7f42b5b01640 1 -- start start 2024-09-13T17:58:31.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42b5b01640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42b0071d30 0x7f42b0153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42b5b01640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42b0072110 0x7f42b00cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42b5b01640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b00c9030 0x7f42b00c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42b5b01640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b00cbca0 con 0x7f42b00c9030 2024-09-13T17:58:31.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42b5b01640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b00b1b80 con 0x7f42b0071d30 2024-09-13T17:58:31.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42b5b01640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b00cd4a0 con 0x7f42b0072110 2024-09-13T17:58:31.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42affff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b00c9030 0x7f42b00c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:31.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42affff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b00c9030 0x7f42b00c53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34278/0 (socket says 172.21.15.111:34278) 2024-09-13T17:58:31.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42affff640 1 -- 172.21.15.111:0/1488378669 learned_addr learned my addr 172.21.15.111:0/1488378669 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:31.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42affff640 1 -- 172.21.15.111:0/1488378669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42b0072110 msgr2=0x7f42b00cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42affff640 1 --2- 172.21.15.111:0/1488378669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42b0072110 0x7f42b00cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42affff640 1 -- 172.21.15.111:0/1488378669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42b0071d30 msgr2=0x7f42b0153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42affff640 1 --2- 172.21.15.111:0/1488378669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42b0071d30 0x7f42b0153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42affff640 1 -- 172.21.15.111:0/1488378669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a8009a70 con 0x7f42b00c9030 2024-09-13T17:58:31.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42affff640 1 --2- 172.21.15.111:0/1488378669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b00c9030 0x7f42b00c53a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009d90 tx=0x7f42a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:31.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42acff9640 1 -- 172.21.15.111:0/1488378669 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a8031bc0 con 0x7f42b00c9030 2024-09-13T17:58:31.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42acff9640 1 -- 172.21.15.111:0/1488378669 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a8031d20 con 0x7f42b00c9030 2024-09-13T17:58:31.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42b5b01640 1 -- 172.21.15.111:0/1488378669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b00cd780 con 0x7f42b00c9030 2024-09-13T17:58:31.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.552+0000 7f42acff9640 1 -- 172.21.15.111:0/1488378669 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a803c620 con 0x7f42b00c9030 2024-09-13T17:58:31.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.553+0000 7f42b5b01640 1 -- 172.21.15.111:0/1488378669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b00cdd40 con 0x7f42b00c9030 2024-09-13T17:58:31.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.554+0000 7f42b5b01640 1 -- 172.21.15.111:0/1488378669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b0070190 con 0x7f42b00c9030 2024-09-13T17:58:31.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.554+0000 7f42acff9640 1 -- 172.21.15.111:0/1488378669 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f42a803c780 con 0x7f42b00c9030 2024-09-13T17:58:31.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.554+0000 7f42acff9640 1 --2- 172.21.15.111:0/1488378669 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f429006ac00 0x7f429006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.555+0000 7f42acff9640 1 -- 172.21.15.111:0/1488378669 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f42a80ab170 con 0x7f42b00c9030 2024-09-13T17:58:31.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.555+0000 7f42af7fe640 1 --2- 172.21.15.111:0/1488378669 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f429006ac00 0x7f429006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:31.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.555+0000 7f42af7fe640 1 --2- 172.21.15.111:0/1488378669 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f429006ac00 0x7f429006d0c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f42a0001550 tx=0x7f42a0007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:31.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.559+0000 7f42acff9640 1 -- 172.21.15.111:0/1488378669 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f42a8076050 con 0x7f42b00c9030 2024-09-13T17:58:31.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.753+0000 7f42b5b01640 1 -- 172.21.15.111:0/1488378669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42b00b1d20 con 0x7f42b00c9030 2024-09-13T17:58:31.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.753+0000 7f42acff9640 1 -- 172.21.15.111:0/1488378669 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4742 (secure 0 0 0) 0x7f42a8079670 con 0x7f42b00c9030 2024-09-13T17:58:31.755 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:31.755 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4001406804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":4001406804},{"type":"v1","addr":"172.21.15.188:6835","nonce":4001406804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2565466103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2565466103},{"type":"v1","addr":"172.21.15.111:6837","nonce":2565466103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/1252690799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1252690799},{"type":"v1","addr":"172.21.15.111:6836","nonce":1252690799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:57:25.702997+0000","modified":"2024-09-13T17:57:46.351493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/1266310843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1266310843},{"type":"v1","addr":"172.21.15.188:6834","nonce":1266310843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:58:31.756 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-09-13T17:58:31.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.755+0000 7f42b5b01640 1 -- 172.21.15.111:0/1488378669 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f429006ac00 msgr2=0x7f429006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.755+0000 7f42b5b01640 1 --2- 172.21.15.111:0/1488378669 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f429006ac00 0x7f429006d0c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f42a0001550 tx=0x7f42a0007a40 comp rx=0 tx=0).stop 2024-09-13T17:58:31.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.755+0000 7f42b5b01640 1 -- 172.21.15.111:0/1488378669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b00c9030 msgr2=0x7f42b00c53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.755+0000 7f42b5b01640 1 --2- 172.21.15.111:0/1488378669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b00c9030 0x7f42b00c53a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009d90 tx=0x7f42a8004a40 comp rx=0 tx=0).stop 2024-09-13T17:58:31.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.755+0000 7f42b5b01640 1 -- 172.21.15.111:0/1488378669 shutdown_connections 2024-09-13T17:58:31.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.755+0000 7f42b5b01640 1 --2- 172.21.15.111:0/1488378669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b00c9030 0x7f42b00c53a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.755+0000 7f42b5b01640 1 --2- 172.21.15.111:0/1488378669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42b0072110 0x7f42b00cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.755+0000 7f42b5b01640 1 --2- 172.21.15.111:0/1488378669 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f429006ac00 0x7f429006d0c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.755+0000 7f42b5b01640 1 --2- 172.21.15.111:0/1488378669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42b0071d30 0x7f42b0153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.755+0000 7f42b5b01640 1 -- 172.21.15.111:0/1488378669 >> 172.21.15.111:0/1488378669 conn(0x7f42b0069ed0 msgr2=0x7f42b00b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:31.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.755+0000 7f42b5b01640 1 -- 172.21.15.111:0/1488378669 shutdown_connections 2024-09-13T17:58:31.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.755+0000 7f42b5b01640 1 -- 172.21.15.111:0/1488378669 wait complete. 2024-09-13T17:58:31.768 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:58:31.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.864+0000 7f6b7e4f7640 1 Processor -- start 2024-09-13T17:58:31.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.864+0000 7f6b7e4f7640 1 -- start start 2024-09-13T17:58:31.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.865+0000 7f6b7e4f7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6b7815c6e0 0x7f6b780b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.865+0000 7f6b7e4f7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b780d07b0 0x7f6b780b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.865+0000 7f6b7e4f7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b780d1840 con 0x7f6b780ccb20 2024-09-13T17:58:31.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.865+0000 7f6b7e4f7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b7806bd20 con 0x7f6b7815c6e0 2024-09-13T17:58:31.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.865+0000 7f6b7e4f7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780d0c30 con 0x7f6b780d07b0 2024-09-13T17:58:31.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.865+0000 7f6b77fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6b7815c6e0 0x7f6b780b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:31.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.865+0000 7f6b7ca94640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6b780ccb20 0x7f6b78160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56082/0 (socket says 172.21.15.111:56082) 2024-09-13T17:58:31.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.865+0000 7f6b7ca94640 1 -- 172.21.15.111:0/3018177194 learned_addr learned my addr 172.21.15.111:0/3018177194 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:31.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.866+0000 7f6b777fe640 1 --2- 172.21.15.111:0/3018177194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b780d07b0 0x7f6b780b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:31.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.866+0000 7f6b76ffd640 1 -- 172.21.15.111:0/3018177194 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326071112 0 0) 0x7f6b780d1840 con 0x7f6b780ccb20 2024-09-13T17:58:31.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.866+0000 7f6b76ffd640 1 -- 172.21.15.111:0/3018177194 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b60003520 con 0x7f6b780ccb20 2024-09-13T17:58:31.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.866+0000 7f6b77fff640 1 -- 172.21.15.111:0/3018177194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b780d07b0 msgr2=0x7f6b780b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:31.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.866+0000 7f6b77fff640 1 --2- 172.21.15.111:0/3018177194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b780d07b0 0x7f6b780b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.866+0000 7f6b77fff640 1 -- 172.21.15.111:0/3018177194 >> v1:172.21.15.111:6789/0 conn(0x7f6b780ccb20 legacy=0x7f6b78160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.866+0000 7f6b77fff640 1 -- 172.21.15.111:0/3018177194 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b780d0f10 con 0x7f6b7815c6e0 2024-09-13T17:58:31.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.866+0000 7f6b777fe640 1 --2- 172.21.15.111:0/3018177194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b780d07b0 0x7f6b780b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:31.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.866+0000 7f6b77fff640 1 --2- 172.21.15.111:0/3018177194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7815c6e0 0x7f6b780b3f50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6b680097a0 tx=0x7f6b6802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c4efda6fab8650 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:31.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.867+0000 7f6b76ffd640 1 -- 172.21.15.111:0/3018177194 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6802ae50 con 0x7f6b7815c6e0 2024-09-13T17:58:31.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.867+0000 7f6b76ffd640 1 -- 172.21.15.111:0/3018177194 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b68032440 con 0x7f6b7815c6e0 2024-09-13T17:58:31.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.867+0000 7f6b76ffd640 1 -- 172.21.15.111:0/3018177194 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680327d0 con 0x7f6b7815c6e0 2024-09-13T17:58:31.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.867+0000 7f6b7e4f7640 1 -- 172.21.15.111:0/3018177194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7815c6e0 msgr2=0x7f6b780b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.867+0000 7f6b7e4f7640 1 --2- 172.21.15.111:0/3018177194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7815c6e0 0x7f6b780b3f50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6b680097a0 tx=0x7f6b6802a100 comp rx=0 tx=0).stop 2024-09-13T17:58:31.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.867+0000 7f6b7e4f7640 1 -- 172.21.15.111:0/3018177194 shutdown_connections 2024-09-13T17:58:31.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.867+0000 7f6b7e4f7640 1 --2- 172.21.15.111:0/3018177194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b780d07b0 0x7f6b780b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.867+0000 7f6b7e4f7640 1 --2- 172.21.15.111:0/3018177194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7815c6e0 0x7f6b780b3f50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.867+0000 7f6b7e4f7640 1 -- 172.21.15.111:0/3018177194 >> 172.21.15.111:0/3018177194 conn(0x7f6b78089f60 msgr2=0x7f6b780d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.867+0000 7f6b7e4f7640 1 -- 172.21.15.111:0/3018177194 shutdown_connections 2024-09-13T17:58:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.867+0000 7f6b7e4f7640 1 -- 172.21.15.111:0/3018177194 wait complete. 2024-09-13T17:58:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.868+0000 7f6b7e4f7640 1 Processor -- start 2024-09-13T17:58:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.869+0000 7f6b7e4f7640 1 -- start start 2024-09-13T17:58:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.869+0000 7f6b7e4f7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b780ccb20 0x7f6b7814a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.869+0000 7f6b7e4f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b780d07b0 0x7f6b7806ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.869+0000 7f6b7e4f7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b7815c6e0 0x7f6b7806b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.869+0000 7f6b7e4f7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780b6280 con 0x7f6b780d07b0 2024-09-13T17:58:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.869+0000 7f6b7e4f7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7814ae30 con 0x7f6b7815c6e0 2024-09-13T17:58:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.869+0000 7f6b7e4f7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780b5670 con 0x7f6b780ccb20 2024-09-13T17:58:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.869+0000 7f6b777fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b780d07b0 0x7f6b7806ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.869+0000 7f6b777fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b780d07b0 0x7f6b7806ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34286/0 (socket says 172.21.15.111:34286) 2024-09-13T17:58:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.869+0000 7f6b777fe640 1 -- 172.21.15.111:0/1733777604 learned_addr learned my addr 172.21.15.111:0/1733777604 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.870+0000 7f6b77fff640 1 --2- 172.21.15.111:0/1733777604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b780ccb20 0x7f6b7814a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.870+0000 7f6b7ca94640 1 --2- 172.21.15.111:0/1733777604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b7815c6e0 0x7f6b7806b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.870+0000 7f6b777fe640 1 -- 172.21.15.111:0/1733777604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b780ccb20 msgr2=0x7f6b7814a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.870+0000 7f6b777fe640 1 --2- 172.21.15.111:0/1733777604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b780ccb20 0x7f6b7814a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.870+0000 7f6b777fe640 1 -- 172.21.15.111:0/1733777604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b7815c6e0 msgr2=0x7f6b7806b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.870+0000 7f6b777fe640 1 --2- 172.21.15.111:0/1733777604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b7815c6e0 0x7f6b7806b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.870+0000 7f6b777fe640 1 -- 172.21.15.111:0/1733777604 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b68009400 con 0x7f6b780d07b0 2024-09-13T17:58:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.870+0000 7f6b777fe640 1 --2- 172.21.15.111:0/1733777604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b780d07b0 0x7f6b7806ad10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00d620 tx=0x7f6b6c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.870+0000 7f6b74ff9640 1 -- 172.21.15.111:0/1733777604 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c004030 con 0x7f6b780d07b0 2024-09-13T17:58:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.870+0000 7f6b74ff9640 1 -- 172.21.15.111:0/1733777604 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c004190 con 0x7f6b780d07b0 2024-09-13T17:58:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.870+0000 7f6b7e4f7640 1 -- 172.21.15.111:0/1733777604 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b780b5950 con 0x7f6b780d07b0 2024-09-13T17:58:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.870+0000 7f6b74ff9640 1 -- 172.21.15.111:0/1733777604 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c004520 con 0x7f6b780d07b0 2024-09-13T17:58:31.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.870+0000 7f6b7e4f7640 1 -- 172.21.15.111:0/1733777604 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b780b5f10 con 0x7f6b780d07b0 2024-09-13T17:58:31.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.871+0000 7f6b7e4f7640 1 -- 172.21.15.111:0/1733777604 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b7815cac0 con 0x7f6b780d07b0 2024-09-13T17:58:31.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.871+0000 7f6b74ff9640 1 -- 172.21.15.111:0/1733777604 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6b6c00b620 con 0x7f6b780d07b0 2024-09-13T17:58:31.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.872+0000 7f6b74ff9640 1 --2- 172.21.15.111:0/1733777604 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b6406ac00 0x7f6b6406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:31.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.872+0000 7f6b74ff9640 1 -- 172.21.15.111:0/1733777604 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6b6c089850 con 0x7f6b780d07b0 2024-09-13T17:58:31.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.872+0000 7f6b77fff640 1 --2- 172.21.15.111:0/1733777604 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b6406ac00 0x7f6b6406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:31.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.872+0000 7f6b77fff640 1 --2- 172.21.15.111:0/1733777604 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b6406ac00 0x7f6b6406d0c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6b68034000 tx=0x7f6b6802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:31.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:31.876+0000 7f6b74ff9640 1 -- 172.21.15.111:0/1733777604 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f6b6c054670 con 0x7f6b780d07b0 2024-09-13T17:58:32.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.069+0000 7f6b7e4f7640 1 -- 172.21.15.111:0/1733777604 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b780b4c80 con 0x7f6b780d07b0 2024-09-13T17:58:32.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.070+0000 7f6b74ff9640 1 -- 172.21.15.111:0/1733777604 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4742 (secure 0 0 0) 0x7f6b6c057c90 con 0x7f6b780d07b0 2024-09-13T17:58:32.072 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:32.072 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4001406804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":4001406804},{"type":"v1","addr":"172.21.15.188:6835","nonce":4001406804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2565466103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2565466103},{"type":"v1","addr":"172.21.15.111:6837","nonce":2565466103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/1252690799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1252690799},{"type":"v1","addr":"172.21.15.111:6836","nonce":1252690799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:57:25.702997+0000","modified":"2024-09-13T17:57:46.351493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/1266310843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1266310843},{"type":"v1","addr":"172.21.15.188:6834","nonce":1266310843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:58:32.072 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-09-13T17:58:32.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.072+0000 7f6b7e4f7640 1 -- 172.21.15.111:0/1733777604 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b6406ac00 msgr2=0x7f6b6406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.072+0000 7f6b7e4f7640 1 --2- 172.21.15.111:0/1733777604 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b6406ac00 0x7f6b6406d0c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6b68034000 tx=0x7f6b6802e000 comp rx=0 tx=0).stop 2024-09-13T17:58:32.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.072+0000 7f6b7e4f7640 1 -- 172.21.15.111:0/1733777604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b780d07b0 msgr2=0x7f6b7806ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.072+0000 7f6b7e4f7640 1 --2- 172.21.15.111:0/1733777604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b780d07b0 0x7f6b7806ad10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00d620 tx=0x7f6b6c00dae0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.072+0000 7f6b7e4f7640 1 -- 172.21.15.111:0/1733777604 shutdown_connections 2024-09-13T17:58:32.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.072+0000 7f6b7e4f7640 1 --2- 172.21.15.111:0/1733777604 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b6406ac00 0x7f6b6406d0c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.072+0000 7f6b7e4f7640 1 --2- 172.21.15.111:0/1733777604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b7815c6e0 0x7f6b7806b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.072+0000 7f6b7e4f7640 1 --2- 172.21.15.111:0/1733777604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b780d07b0 0x7f6b7806ad10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.072+0000 7f6b7e4f7640 1 --2- 172.21.15.111:0/1733777604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b780ccb20 0x7f6b7814a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.072+0000 7f6b7e4f7640 1 -- 172.21.15.111:0/1733777604 >> 172.21.15.111:0/1733777604 conn(0x7f6b78089f60 msgr2=0x7f6b780b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:32.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.072+0000 7f6b7e4f7640 1 -- 172.21.15.111:0/1733777604 shutdown_connections 2024-09-13T17:58:32.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.072+0000 7f6b7e4f7640 1 -- 172.21.15.111:0/1733777604 wait complete. 2024-09-13T17:58:32.084 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:32.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.179+0000 7f5f3323d640 1 Processor -- start 2024-09-13T17:58:32.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.179+0000 7f5f3323d640 1 -- start start 2024-09-13T17:58:32.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.180+0000 7f5f3323d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c08a370 0x7f5f2c0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:32.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.180+0000 7f5f3323d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2c0b3540 con 0x7f5f2c0b4510 2024-09-13T17:58:32.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.180+0000 7f5f3323d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2c0cdc70 con 0x7f5f2c0b4130 2024-09-13T17:58:32.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.180+0000 7f5f3323d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0b2ff0 con 0x7f5f2c08a370 2024-09-13T17:58:32.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.180+0000 7f5f317da640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5f2c0b4510 0x7f5f2c0d1450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56090/0 (socket says 172.21.15.111:56090) 2024-09-13T17:58:32.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.180+0000 7f5f317da640 1 -- 172.21.15.111:0/3407093920 learned_addr learned my addr 172.21.15.111:0/3407093920 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:32.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.181+0000 7f5f2bfff640 1 --2- 172.21.15.111:0/3407093920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c08a370 0x7f5f2c0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:32.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.181+0000 7f5f2b7fe640 1 -- 172.21.15.111:0/3407093920 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3905209688 0 0) 0x7f5f2c0b3540 con 0x7f5f2c0b4510 2024-09-13T17:58:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.181+0000 7f5f2b7fe640 1 -- 172.21.15.111:0/3407093920 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f14003520 con 0x7f5f2c0b4510 2024-09-13T17:58:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.181+0000 7f5f2b7fe640 1 -- 172.21.15.111:0/3407093920 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3834705376 0 0) 0x7f5f1c001110 con 0x7f5f2c0b4130 2024-09-13T17:58:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.181+0000 7f5f2b7fe640 1 -- 172.21.15.111:0/3407093920 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f2c0b3540 con 0x7f5f2c0b4130 2024-09-13T17:58:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.181+0000 7f5f2b7fe640 1 -- 172.21.15.111:0/3407093920 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1906872172 0 0) 0x7f5f14003520 con 0x7f5f2c0b4510 2024-09-13T17:58:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.181+0000 7f5f2bfff640 1 -- 172.21.15.111:0/3407093920 >> v1:172.21.15.188:6789/0 conn(0x7f5f2c0b4130 legacy=0x7f5f2c0d2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.181+0000 7f5f2bfff640 1 -- 172.21.15.111:0/3407093920 >> v1:172.21.15.111:6789/0 conn(0x7f5f2c0b4510 legacy=0x7f5f2c0d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.181+0000 7f5f2bfff640 1 -- 172.21.15.111:0/3407093920 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c0cd6f0 con 0x7f5f2c08a370 2024-09-13T17:58:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.182+0000 7f5f2bfff640 1 --2- 172.21.15.111:0/3407093920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c08a370 0x7f5f2c0d0f10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009dc0 tx=0x7f5f2002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=468ea601642fce42 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.182+0000 7f5f2b7fe640 1 -- 172.21.15.111:0/3407093920 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24001670 con 0x7f5f2c08a370 2024-09-13T17:58:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.182+0000 7f5f2b7fe640 1 -- 172.21.15.111:0/3407093920 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2002d050 con 0x7f5f2c08a370 2024-09-13T17:58:32.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.182+0000 7f5f2b7fe640 1 -- 172.21.15.111:0/3407093920 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20007740 con 0x7f5f2c08a370 2024-09-13T17:58:32.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.182+0000 7f5f3323d640 1 -- 172.21.15.111:0/3407093920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c08a370 msgr2=0x7f5f2c0d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.182+0000 7f5f3323d640 1 --2- 172.21.15.111:0/3407093920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c08a370 0x7f5f2c0d0f10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009dc0 tx=0x7f5f2002a5d0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.183+0000 7f5f3323d640 1 -- 172.21.15.111:0/3407093920 shutdown_connections 2024-09-13T17:58:32.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.183+0000 7f5f3323d640 1 --2- 172.21.15.111:0/3407093920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c08a370 0x7f5f2c0d0f10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.183+0000 7f5f3323d640 1 -- 172.21.15.111:0/3407093920 >> 172.21.15.111:0/3407093920 conn(0x7f5f2c089f60 msgr2=0x7f5f2c0d24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:32.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.183+0000 7f5f3323d640 1 -- 172.21.15.111:0/3407093920 shutdown_connections 2024-09-13T17:58:32.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.183+0000 7f5f3323d640 1 -- 172.21.15.111:0/3407093920 wait complete. 2024-09-13T17:58:32.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.184+0000 7f5f3323d640 1 Processor -- start 2024-09-13T17:58:32.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.184+0000 7f5f3323d640 1 -- start start 2024-09-13T17:58:32.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f3323d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c08a370 0x7f5f2c0c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:32.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f3323d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c0b4130 0x7f5f2c0af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:32.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f3323d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0b4510 0x7f5f2c1529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:32.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f3323d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0b0b60 con 0x7f5f2c0b4130 2024-09-13T17:58:32.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f3323d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0cc7f0 con 0x7f5f2c0b4510 2024-09-13T17:58:32.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f3323d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0cc230 con 0x7f5f2c08a370 2024-09-13T17:58:32.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f2bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c0b4130 0x7f5f2c0af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:32.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f2bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c0b4130 0x7f5f2c0af0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34288/0 (socket says 172.21.15.111:34288) 2024-09-13T17:58:32.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f2bfff640 1 -- 172.21.15.111:0/2368835305 learned_addr learned my addr 172.21.15.111:0/2368835305 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:32.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f30fd9640 1 --2- 172.21.15.111:0/2368835305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c08a370 0x7f5f2c0c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:32.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f317da640 1 --2- 172.21.15.111:0/2368835305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0b4510 0x7f5f2c1529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:32.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f2bfff640 1 -- 172.21.15.111:0/2368835305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c08a370 msgr2=0x7f5f2c0c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f2bfff640 1 --2- 172.21.15.111:0/2368835305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c08a370 0x7f5f2c0c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f2bfff640 1 -- 172.21.15.111:0/2368835305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0b4510 msgr2=0x7f5f2c1529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f2bfff640 1 --2- 172.21.15.111:0/2368835305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0b4510 0x7f5f2c1529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f2bfff640 1 -- 172.21.15.111:0/2368835305 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f20009a70 con 0x7f5f2c0b4130 2024-09-13T17:58:32.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f2bfff640 1 --2- 172.21.15.111:0/2368835305 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c0b4130 0x7f5f2c0af0e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009ee0 tx=0x7f5f20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:32.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f297fa640 1 -- 172.21.15.111:0/2368835305 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2003c830 con 0x7f5f2c0b4130 2024-09-13T17:58:32.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f297fa640 1 -- 172.21.15.111:0/2368835305 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2003c990 con 0x7f5f2c0b4130 2024-09-13T17:58:32.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f297fa640 1 -- 172.21.15.111:0/2368835305 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2003cd00 con 0x7f5f2c0b4130 2024-09-13T17:58:32.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.185+0000 7f5f3323d640 1 -- 172.21.15.111:0/2368835305 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c0cc510 con 0x7f5f2c0b4130 2024-09-13T17:58:32.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.186+0000 7f5f3323d640 1 -- 172.21.15.111:0/2368835305 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c0b1070 con 0x7f5f2c0b4130 2024-09-13T17:58:32.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.187+0000 7f5f3323d640 1 -- 172.21.15.111:0/2368835305 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c0d2ce0 con 0x7f5f2c0b4130 2024-09-13T17:58:32.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.187+0000 7f5f297fa640 1 -- 172.21.15.111:0/2368835305 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5f20031400 con 0x7f5f2c0b4130 2024-09-13T17:58:32.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.187+0000 7f5f297fa640 1 --2- 172.21.15.111:0/2368835305 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5f1806ac00 0x7f5f1806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:32.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.188+0000 7f5f297fa640 1 -- 172.21.15.111:0/2368835305 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5f200ab020 con 0x7f5f2c0b4130 2024-09-13T17:58:32.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.188+0000 7f5f30fd9640 1 --2- 172.21.15.111:0/2368835305 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5f1806ac00 0x7f5f1806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:32.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.188+0000 7f5f30fd9640 1 --2- 172.21.15.111:0/2368835305 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5f1806ac00 0x7f5f1806d0c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c007fd0 tx=0x7f5f1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:32.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.192+0000 7f5f297fa640 1 -- 172.21.15.111:0/2368835305 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f5f200760a0 con 0x7f5f2c0b4130 2024-09-13T17:58:32.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.355+0000 7f5f3323d640 1 -- 172.21.15.111:0/2368835305 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f2c069ed0 con 0x7f5f2c0b4130 2024-09-13T17:58:32.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.357+0000 7f5f297fa640 1 -- 172.21.15.111:0/2368835305 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18794 (secure 0 0 0) 0x7f5f2003ce60 con 0x7f5f2c0b4130 2024-09-13T17:58:32.359 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:32.359 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":26,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:07.749113+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:57:24.089739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:57:24.667768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:57:27.731417+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:32.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.359+0000 7f5f3323d640 1 -- 172.21.15.111:0/2368835305 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5f1806ac00 msgr2=0x7f5f1806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.359+0000 7f5f3323d640 1 --2- 172.21.15.111:0/2368835305 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5f1806ac00 0x7f5f1806d0c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c007fd0 tx=0x7f5f1c009040 comp rx=0 tx=0).stop 2024-09-13T17:58:32.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.359+0000 7f5f3323d640 1 -- 172.21.15.111:0/2368835305 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c0b4130 msgr2=0x7f5f2c0af0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.359+0000 7f5f3323d640 1 --2- 172.21.15.111:0/2368835305 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c0b4130 0x7f5f2c0af0e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009ee0 tx=0x7f5f20004a40 comp rx=0 tx=0).stop 2024-09-13T17:58:32.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.359+0000 7f5f3323d640 1 -- 172.21.15.111:0/2368835305 shutdown_connections 2024-09-13T17:58:32.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.359+0000 7f5f3323d640 1 --2- 172.21.15.111:0/2368835305 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5f1806ac00 0x7f5f1806d0c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.359+0000 7f5f3323d640 1 --2- 172.21.15.111:0/2368835305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0b4510 0x7f5f2c1529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.359+0000 7f5f3323d640 1 --2- 172.21.15.111:0/2368835305 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c0b4130 0x7f5f2c0af0e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.359+0000 7f5f3323d640 1 --2- 172.21.15.111:0/2368835305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c08a370 0x7f5f2c0c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.359+0000 7f5f3323d640 1 -- 172.21.15.111:0/2368835305 >> 172.21.15.111:0/2368835305 conn(0x7f5f2c089f60 msgr2=0x7f5f2c0b7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:32.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.359+0000 7f5f3323d640 1 -- 172.21.15.111:0/2368835305 shutdown_connections 2024-09-13T17:58:32.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.359+0000 7f5f3323d640 1 -- 172.21.15.111:0/2368835305 wait complete. 2024-09-13T17:58:32.372 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T17:58:32.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.468+0000 7fd3690c5640 1 Processor -- start 2024-09-13T17:58:32.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.468+0000 7fd3690c5640 1 -- start start 2024-09-13T17:58:32.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.469+0000 7fd3690c5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3640d07b0 0x7fd36406ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:32.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.469+0000 7fd3690c5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd3640b4660 0x7fd36406b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:32.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.469+0000 7fd3690c5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3640d1840 con 0x7fd36415c6e0 2024-09-13T17:58:32.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.469+0000 7fd3690c5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3640d0c30 con 0x7fd3640b4660 2024-09-13T17:58:32.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.469+0000 7fd3690c5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640d0f10 con 0x7fd3640d07b0 2024-09-13T17:58:32.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.470+0000 7fd36259c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3640d07b0 0x7fd36406ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:32.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.470+0000 7fd36359e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd3640b4660 0x7fd36406b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:32.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.470+0000 7fd362d9d640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fd36415c6e0 0x7fd3640b3f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:44616/0 (socket says 172.21.15.111:44616) 2024-09-13T17:58:32.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.470+0000 7fd362d9d640 1 -- 172.21.15.111:0/927381359 learned_addr learned my addr 172.21.15.111:0/927381359 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:32.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.470+0000 7fd36359e640 1 -- 172.21.15.111:0/927381359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3640d07b0 msgr2=0x7fd36406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:32.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.470+0000 7fd36359e640 1 --2- 172.21.15.111:0/927381359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3640d07b0 0x7fd36406ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.470+0000 7fd36359e640 1 -- 172.21.15.111:0/927381359 >> v1:172.21.15.188:6789/0 conn(0x7fd36415c6e0 legacy=0x7fd3640b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.470+0000 7fd36359e640 1 -- 172.21.15.111:0/927381359 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3640d11f0 con 0x7fd3640b4660 2024-09-13T17:58:32.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.470+0000 7fd361d9b640 1 -- 172.21.15.111:0/927381359 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2108509406 0 0) 0x7fd3640d1840 con 0x7fd36415c6e0 2024-09-13T17:58:32.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.470+0000 7fd36259c640 1 --2- 172.21.15.111:0/927381359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3640d07b0 0x7fd36406ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:32.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.471+0000 7fd36359e640 1 --2- 172.21.15.111:0/927381359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3640b4660 0x7fd36406b250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0097a0 tx=0x7fd35c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5486a1815391513c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:32.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.471+0000 7fd361d9b640 1 -- 172.21.15.111:0/927381359 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c02ae50 con 0x7fd3640b4660 2024-09-13T17:58:32.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.471+0000 7fd361d9b640 1 -- 172.21.15.111:0/927381359 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35c032440 con 0x7fd3640b4660 2024-09-13T17:58:32.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.471+0000 7fd361d9b640 1 -- 172.21.15.111:0/927381359 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0327d0 con 0x7fd3640b4660 2024-09-13T17:58:32.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.471+0000 7fd3690c5640 1 -- 172.21.15.111:0/927381359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3640b4660 msgr2=0x7fd36406b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.471+0000 7fd3690c5640 1 --2- 172.21.15.111:0/927381359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3640b4660 0x7fd36406b250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0097a0 tx=0x7fd35c02a100 comp rx=0 tx=0).stop 2024-09-13T17:58:32.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.471+0000 7fd3690c5640 1 -- 172.21.15.111:0/927381359 shutdown_connections 2024-09-13T17:58:32.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.471+0000 7fd3690c5640 1 --2- 172.21.15.111:0/927381359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3640b4660 0x7fd36406b250 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.471+0000 7fd3690c5640 1 --2- 172.21.15.111:0/927381359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3640d07b0 0x7fd36406ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.471+0000 7fd3690c5640 1 -- 172.21.15.111:0/927381359 >> 172.21.15.111:0/927381359 conn(0x7fd364089f60 msgr2=0x7fd3640d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:32.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.471+0000 7fd3690c5640 1 -- 172.21.15.111:0/927381359 shutdown_connections 2024-09-13T17:58:32.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.471+0000 7fd3690c5640 1 -- 172.21.15.111:0/927381359 wait complete. 2024-09-13T17:58:32.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.473+0000 7fd3690c5640 1 Processor -- start 2024-09-13T17:58:32.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.473+0000 7fd3690c5640 1 -- start start 2024-09-13T17:58:32.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.474+0000 7fd3690c5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3640b4660 0x7fd364159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:32.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.474+0000 7fd3690c5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3640d07b0 0x7fd36415f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:32.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.474+0000 7fd3690c5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd36415c6e0 0x7fd36414b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:32.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.474+0000 7fd3690c5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640b57b0 con 0x7fd36415c6e0 2024-09-13T17:58:32.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.474+0000 7fd3690c5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640b5a90 con 0x7fd3640d07b0 2024-09-13T17:58:32.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.474+0000 7fd3690c5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640b5d70 con 0x7fd3640b4660 2024-09-13T17:58:32.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.474+0000 7fd36359e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd36415c6e0 0x7fd36414b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:32.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.474+0000 7fd36359e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd36415c6e0 0x7fd36414b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58910/0 (socket says 172.21.15.111:58910) 2024-09-13T17:58:32.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.474+0000 7fd36359e640 1 -- 172.21.15.111:0/2424951214 learned_addr learned my addr 172.21.15.111:0/2424951214 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:32.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.474+0000 7fd36259c640 1 --2- 172.21.15.111:0/2424951214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3640d07b0 0x7fd36415f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:32.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.474+0000 7fd362d9d640 1 --2- 172.21.15.111:0/2424951214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3640b4660 0x7fd364159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:32.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.474+0000 7fd36359e640 1 -- 172.21.15.111:0/2424951214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3640b4660 msgr2=0x7fd364159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.474+0000 7fd36359e640 1 --2- 172.21.15.111:0/2424951214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3640b4660 0x7fd364159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.474+0000 7fd36359e640 1 -- 172.21.15.111:0/2424951214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3640d07b0 msgr2=0x7fd36415f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.474+0000 7fd36359e640 1 --2- 172.21.15.111:0/2424951214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3640d07b0 0x7fd36415f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.474+0000 7fd36359e640 1 -- 172.21.15.111:0/2424951214 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd358009400 con 0x7fd36415c6e0 2024-09-13T17:58:32.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.474+0000 7fd36359e640 1 --2- 172.21.15.111:0/2424951214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd36415c6e0 0x7fd36414b4d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0097a0 tx=0x7fd35c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:32.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.475+0000 7fd34b7fe640 1 -- 172.21.15.111:0/2424951214 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c004230 con 0x7fd36415c6e0 2024-09-13T17:58:32.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.475+0000 7fd34b7fe640 1 -- 172.21.15.111:0/2424951214 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35c0043b0 con 0x7fd36415c6e0 2024-09-13T17:58:32.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.475+0000 7fd3690c5640 1 -- 172.21.15.111:0/2424951214 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35c009400 con 0x7fd36415c6e0 2024-09-13T17:58:32.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.475+0000 7fd34b7fe640 1 -- 172.21.15.111:0/2424951214 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c004520 con 0x7fd36415c6e0 2024-09-13T17:58:32.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.475+0000 7fd3690c5640 1 -- 172.21.15.111:0/2424951214 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3640b6330 con 0x7fd36415c6e0 2024-09-13T17:58:32.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.476+0000 7fd3690c5640 1 -- 172.21.15.111:0/2424951214 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd36415c040 con 0x7fd36415c6e0 2024-09-13T17:58:32.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.476+0000 7fd34b7fe640 1 -- 172.21.15.111:0/2424951214 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd35c039050 con 0x7fd36415c6e0 2024-09-13T17:58:32.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.476+0000 7fd34b7fe640 1 --2- 172.21.15.111:0/2424951214 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd35006ac30 0x7fd35006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:32.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.477+0000 7fd34b7fe640 1 -- 172.21.15.111:0/2424951214 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd35c0a9c10 con 0x7fd36415c6e0 2024-09-13T17:58:32.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.477+0000 7fd362d9d640 1 --2- 172.21.15.111:0/2424951214 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd35006ac30 0x7fd35006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:32.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.477+0000 7fd362d9d640 1 --2- 172.21.15.111:0/2424951214 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd35006ac30 0x7fd35006d0f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd3540012e0 tx=0x7fd354001700 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:32.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.481+0000 7fd34b7fe640 1 -- 172.21.15.111:0/2424951214 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fd35c074c90 con 0x7fd36415c6e0 2024-09-13T17:58:32.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.677+0000 7fd3690c5640 1 -- 172.21.15.111:0/2424951214 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd36408c610 con 0x7fd36415c6e0 2024-09-13T17:58:32.680 INFO:tasks.ceph.mon.a.smithi111.stderr:2024-09-13T17:58:32.678+0000 7f3bedaae640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T17:58:32.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.690+0000 7fd34b7fe640 1 -- 172.21.15.111:0/2424951214 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd35c04b090 con 0x7fd36415c6e0 2024-09-13T17:58:32.692 INFO:teuthology.orchestra.run.smithi111.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T17:58:32.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.692+0000 7fd3690c5640 1 -- 172.21.15.111:0/2424951214 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd35006ac30 msgr2=0x7fd35006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.692+0000 7fd3690c5640 1 --2- 172.21.15.111:0/2424951214 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd35006ac30 0x7fd35006d0f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd3540012e0 tx=0x7fd354001700 comp rx=0 tx=0).stop 2024-09-13T17:58:32.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.692+0000 7fd3690c5640 1 -- 172.21.15.111:0/2424951214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd36415c6e0 msgr2=0x7fd36414b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.692+0000 7fd3690c5640 1 --2- 172.21.15.111:0/2424951214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd36415c6e0 0x7fd36414b4d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0097a0 tx=0x7fd35c0046c0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.692+0000 7fd3690c5640 1 -- 172.21.15.111:0/2424951214 shutdown_connections 2024-09-13T17:58:32.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.692+0000 7fd3690c5640 1 --2- 172.21.15.111:0/2424951214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd36415c6e0 0x7fd36414b4d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.692+0000 7fd3690c5640 1 --2- 172.21.15.111:0/2424951214 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd35006ac30 0x7fd35006d0f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.692+0000 7fd3690c5640 1 --2- 172.21.15.111:0/2424951214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3640d07b0 0x7fd36415f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.692+0000 7fd3690c5640 1 --2- 172.21.15.111:0/2424951214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3640b4660 0x7fd364159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.692+0000 7fd3690c5640 1 -- 172.21.15.111:0/2424951214 >> 172.21.15.111:0/2424951214 conn(0x7fd364089f60 msgr2=0x7fd3640b2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:32.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.692+0000 7fd3690c5640 1 -- 172.21.15.111:0/2424951214 shutdown_connections 2024-09-13T17:58:32.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.692+0000 7fd3690c5640 1 -- 172.21.15.111:0/2424951214 wait complete. 2024-09-13T17:58:32.695 INFO:tasks.ceph.mds.b.smithi188.stderr: -4089> 2024-09-13T17:57:30.806+0000 7f1289ce4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:58:32.704 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T17:58:32.726 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-09-13T17:58:32.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.804+0000 7fcb6f7db640 1 Processor -- start 2024-09-13T17:58:32.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.804+0000 7fcb6f7db640 1 -- start start 2024-09-13T17:58:32.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.805+0000 7fcb6f7db640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcb680b2b40 0x7fcb680ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:32.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.805+0000 7fcb6f7db640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcb680ce4f0 0x7fcb680ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:32.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.805+0000 7fcb6f7db640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb680ccbe0 con 0x7fcb68159310 2024-09-13T17:58:32.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.805+0000 7fcb6f7db640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb680ad940 con 0x7fcb680ce4f0 2024-09-13T17:58:32.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.805+0000 7fcb6f7db640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb680b1810 con 0x7fcb680b2b40 2024-09-13T17:58:32.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.805+0000 7fcb6dd78640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcb680ce4f0 0x7fcb680ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:32.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.805+0000 7fcb6dd78640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcb680ce4f0 0x7fcb680ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58916/0 (socket says 172.21.15.111:58916) 2024-09-13T17:58:32.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.805+0000 7fcb6dd78640 1 -- 172.21.15.111:0/2633933812 learned_addr learned my addr 172.21.15.111:0/2633933812 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:32.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.805+0000 7fcb6dd78640 1 -- 172.21.15.111:0/2633933812 >> v2:172.21.15.188:3300/0 conn(0x7fcb680b2b40 msgr2=0x7fcb680ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:32.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.805+0000 7fcb6dd78640 1 --2- 172.21.15.111:0/2633933812 >> v2:172.21.15.188:3300/0 conn(0x7fcb680b2b40 0x7fcb680ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.805+0000 7fcb6dd78640 1 -- 172.21.15.111:0/2633933812 >> v1:172.21.15.188:6789/0 conn(0x7fcb68159310 legacy=0x7fcb680ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:32.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.805+0000 7fcb6dd78640 1 -- 172.21.15.111:0/2633933812 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb680cee10 con 0x7fcb680ce4f0 2024-09-13T17:58:32.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.806+0000 7fcb6dd78640 1 --2- 172.21.15.111:0/2633933812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb680ce4f0 0x7fcb680ce8d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcb680b16a0 tx=0x7fcb6002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d00cafb0a1cf34 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:32.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.806+0000 7fcb5ffff640 1 -- 172.21.15.111:0/2633933812 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb60031600 con 0x7fcb680ce4f0 2024-09-13T17:58:32.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.806+0000 7fcb5ffff640 1 -- 172.21.15.111:0/2633933812 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6002d050 con 0x7fcb680ce4f0 2024-09-13T17:58:32.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.806+0000 7fcb5ffff640 1 -- 172.21.15.111:0/2633933812 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb60031980 con 0x7fcb680ce4f0 2024-09-13T17:58:32.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.807+0000 7fcb6f7db640 1 -- 172.21.15.111:0/2633933812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb680ce4f0 msgr2=0x7fcb680ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.807+0000 7fcb6f7db640 1 --2- 172.21.15.111:0/2633933812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb680ce4f0 0x7fcb680ce8d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcb680b16a0 tx=0x7fcb6002a660 comp rx=0 tx=0).stop 2024-09-13T17:58:32.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.807+0000 7fcb6f7db640 1 -- 172.21.15.111:0/2633933812 shutdown_connections 2024-09-13T17:58:32.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.807+0000 7fcb6f7db640 1 --2- 172.21.15.111:0/2633933812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb680ce4f0 0x7fcb680ce8d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.807+0000 7fcb6f7db640 1 --2- 172.21.15.111:0/2633933812 >> v2:172.21.15.188:3300/0 conn(0x7fcb680b2b40 0x7fcb680ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.807+0000 7fcb6f7db640 1 -- 172.21.15.111:0/2633933812 >> 172.21.15.111:0/2633933812 conn(0x7fcb6806a0b0 msgr2=0x7fcb6806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:32.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.807+0000 7fcb6f7db640 1 -- 172.21.15.111:0/2633933812 shutdown_connections 2024-09-13T17:58:32.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.807+0000 7fcb6f7db640 1 -- 172.21.15.111:0/2633933812 wait complete. 2024-09-13T17:58:32.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.808+0000 7fcb6f7db640 1 Processor -- start 2024-09-13T17:58:32.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.808+0000 7fcb6f7db640 1 -- start start 2024-09-13T17:58:32.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb6f7db640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb680b2b40 0x7fcb680aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:32.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb6f7db640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb680ce4f0 0x7fcb680ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:32.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb6f7db640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb68159310 0x7fcb681e2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:32.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb6f7db640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680ae8e0 con 0x7fcb680b2b40 2024-09-13T17:58:32.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb6f7db640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb681f2340 con 0x7fcb680ce4f0 2024-09-13T17:58:32.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb6f7db640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680afe00 con 0x7fcb68159310 2024-09-13T17:58:32.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb6d577640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb680b2b40 0x7fcb680aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:32.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb6d577640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb680b2b40 0x7fcb680aeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58918/0 (socket says 172.21.15.111:58918) 2024-09-13T17:58:32.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb6d577640 1 -- 172.21.15.111:0/789017321 learned_addr learned my addr 172.21.15.111:0/789017321 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:32.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb6d577640 1 -- 172.21.15.111:0/789017321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb68159310 msgr2=0x7fcb681e2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb6d577640 1 --2- 172.21.15.111:0/789017321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb68159310 0x7fcb681e2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb6d577640 1 -- 172.21.15.111:0/789017321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb680ce4f0 msgr2=0x7fcb680ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:32.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb6d577640 1 --2- 172.21.15.111:0/789017321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb680ce4f0 0x7fcb680ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:32.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb6d577640 1 -- 172.21.15.111:0/789017321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb60009b30 con 0x7fcb680b2b40 2024-09-13T17:58:32.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb6d577640 1 --2- 172.21.15.111:0/789017321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb680b2b40 0x7fcb680aeb80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400b410 tx=0x7fcb6400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:32.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb5dffb640 1 -- 172.21.15.111:0/789017321 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64011830 con 0x7fcb680b2b40 2024-09-13T17:58:32.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb5dffb640 1 -- 172.21.15.111:0/789017321 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb640119b0 con 0x7fcb680b2b40 2024-09-13T17:58:32.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb6f7db640 1 -- 172.21.15.111:0/789017321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb681e3090 con 0x7fcb680b2b40 2024-09-13T17:58:32.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.809+0000 7fcb5dffb640 1 -- 172.21.15.111:0/789017321 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64011d60 con 0x7fcb680b2b40 2024-09-13T17:58:32.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.810+0000 7fcb6f7db640 1 -- 172.21.15.111:0/789017321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb680af560 con 0x7fcb680b2b40 2024-09-13T17:58:32.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.811+0000 7fcb6f7db640 1 -- 172.21.15.111:0/789017321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6806dfa0 con 0x7fcb680b2b40 2024-09-13T17:58:32.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.811+0000 7fcb5dffb640 1 -- 172.21.15.111:0/789017321 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcb6401a400 con 0x7fcb680b2b40 2024-09-13T17:58:32.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.811+0000 7fcb5dffb640 1 --2- 172.21.15.111:0/789017321 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcb4c06ac00 0x7fcb4c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:32.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.811+0000 7fcb5dffb640 1 -- 172.21.15.111:0/789017321 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fcb64089770 con 0x7fcb680b2b40 2024-09-13T17:58:32.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.811+0000 7fcb6cd76640 1 --2- 172.21.15.111:0/789017321 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcb4c06ac00 0x7fcb4c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:32.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.812+0000 7fcb6cd76640 1 --2- 172.21.15.111:0/789017321 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcb4c06ac00 0x7fcb4c06d0c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcb6809c3d0 tx=0x7fcb58006c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:32.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:32.816+0000 7fcb5dffb640 1 -- 172.21.15.111:0/789017321 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fcb640545d0 con 0x7fcb680b2b40 2024-09-13T17:58:33.015 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.013+0000 7fcb6f7db640 1 -- 172.21.15.111:0/789017321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcb6806e4b0 con 0x7fcb680b2b40 2024-09-13T17:58:33.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.692+0000 7fcb5dffb640 1 -- 172.21.15.111:0/789017321 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcb64057bf0 con 0x7fcb680b2b40 2024-09-13T17:58:33.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.694+0000 7fcb6f7db640 1 -- 172.21.15.111:0/789017321 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcb4c06ac00 msgr2=0x7fcb4c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:33.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.694+0000 7fcb6f7db640 1 --2- 172.21.15.111:0/789017321 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcb4c06ac00 0x7fcb4c06d0c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcb6809c3d0 tx=0x7fcb58006c90 comp rx=0 tx=0).stop 2024-09-13T17:58:33.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.694+0000 7fcb6f7db640 1 -- 172.21.15.111:0/789017321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb680b2b40 msgr2=0x7fcb680aeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:33.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.694+0000 7fcb6f7db640 1 --2- 172.21.15.111:0/789017321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb680b2b40 0x7fcb680aeb80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400b410 tx=0x7fcb6400b8d0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.694+0000 7fcb6f7db640 1 -- 172.21.15.111:0/789017321 shutdown_connections 2024-09-13T17:58:33.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.694+0000 7fcb6f7db640 1 --2- 172.21.15.111:0/789017321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb68159310 0x7fcb681e2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.694+0000 7fcb6f7db640 1 --2- 172.21.15.111:0/789017321 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcb4c06ac00 0x7fcb4c06d0c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.694+0000 7fcb6f7db640 1 --2- 172.21.15.111:0/789017321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb680ce4f0 0x7fcb680ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.694+0000 7fcb6f7db640 1 --2- 172.21.15.111:0/789017321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb680b2b40 0x7fcb680aeb80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.694+0000 7fcb6f7db640 1 -- 172.21.15.111:0/789017321 >> 172.21.15.111:0/789017321 conn(0x7fcb6806a0b0 msgr2=0x7fcb680cb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:33.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.694+0000 7fcb6f7db640 1 -- 172.21.15.111:0/789017321 shutdown_connections 2024-09-13T17:58:33.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.694+0000 7fcb6f7db640 1 -- 172.21.15.111:0/789017321 wait complete. 2024-09-13T17:58:33.708 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T17:58:33.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.812+0000 7f244151f640 1 Processor -- start 2024-09-13T17:58:33.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.812+0000 7f244151f640 1 -- start start 2024-09-13T17:58:33.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.813+0000 7f244151f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c163220 0x7f243c165610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:33.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.813+0000 7f244151f640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243c0cc300 con 0x7f243c0ce3e0 2024-09-13T17:58:33.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.813+0000 7f244151f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243c0b3d20 con 0x7f243c15a7f0 2024-09-13T17:58:33.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.813+0000 7f244151f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0997f0 con 0x7f243c163220 2024-09-13T17:58:33.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.813+0000 7f243a7fc640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f243c0ce3e0 0x7f243c0990e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48686/0 (socket says 172.21.15.111:48686) 2024-09-13T17:58:33.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.813+0000 7f243a7fc640 1 -- 172.21.15.111:0/1972055306 learned_addr learned my addr 172.21.15.111:0/1972055306 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:33.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.813+0000 7f2439ffb640 1 -- 172.21.15.111:0/1972055306 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956098703 0 0) 0x7f243c0cc300 con 0x7f243c0ce3e0 2024-09-13T17:58:33.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.813+0000 7f2439ffb640 1 -- 172.21.15.111:0/1972055306 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2424003520 con 0x7f243c0ce3e0 2024-09-13T17:58:33.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.813+0000 7f243b7fe640 1 --2- 172.21.15.111:0/1972055306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c163220 0x7f243c165610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:33.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f2439ffb640 1 -- 172.21.15.111:0/1972055306 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4201984933 0 0) 0x7f2424003520 con 0x7f243c0ce3e0 2024-09-13T17:58:33.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f2439ffb640 1 -- 172.21.15.111:0/1972055306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c163220 msgr2=0x7f243c165610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:33.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f2439ffb640 1 --2- 172.21.15.111:0/1972055306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c163220 0x7f243c165610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f2439ffb640 1 -- 172.21.15.111:0/1972055306 >> v1:172.21.15.188:6789/0 conn(0x7f243c15a7f0 legacy=0x7f243c0989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:33.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f2439ffb640 1 -- 172.21.15.111:0/1972055306 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c0b33f0 con 0x7f243c0ce3e0 2024-09-13T17:58:33.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f2439ffb640 1 -- 172.21.15.111:0/1972055306 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f2430001670 con 0x7f243c0ce3e0 2024-09-13T17:58:33.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f2439ffb640 1 -- 172.21.15.111:0/1972055306 >> v1:172.21.15.111:6789/0 conn(0x7f243c0ce3e0 legacy=0x7f243c0990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:33.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f2439ffb640 1 --2- 172.21.15.111:0/1972055306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24240039a0 0x7f2424003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f2439ffb640 1 --2- 172.21.15.111:0/1972055306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24240047c0 0x7f2424004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f2439ffb640 1 --2- 172.21.15.111:0/1972055306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c163220 0x7f242400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f2439ffb640 1 -- 172.21.15.111:0/1972055306 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0997f0 con 0x7f24240047c0 2024-09-13T17:58:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f2439ffb640 1 -- 172.21.15.111:0/1972055306 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24240054e0 con 0x7f24240039a0 2024-09-13T17:58:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f2439ffb640 1 -- 172.21.15.111:0/1972055306 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242400bec0 con 0x7f243c163220 2024-09-13T17:58:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f243a7fc640 1 --2- 172.21.15.111:0/1972055306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24240047c0 0x7f2424004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f243affd640 1 --2- 172.21.15.111:0/1972055306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24240039a0 0x7f2424003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f243b7fe640 1 --2- 172.21.15.111:0/1972055306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c163220 0x7f242400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f243a7fc640 1 -- 172.21.15.111:0/1972055306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c163220 msgr2=0x7f242400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f243a7fc640 1 --2- 172.21.15.111:0/1972055306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c163220 0x7f242400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f243a7fc640 1 -- 172.21.15.111:0/1972055306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24240039a0 msgr2=0x7f2424003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f243a7fc640 1 --2- 172.21.15.111:0/1972055306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24240039a0 0x7f2424003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.814+0000 7f243a7fc640 1 -- 172.21.15.111:0/1972055306 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2424003520 con 0x7f24240047c0 2024-09-13T17:58:33.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.815+0000 7f243a7fc640 1 --2- 172.21.15.111:0/1972055306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24240047c0 0x7f2424004c40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f243000c210 tx=0x7f24240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9ba5a6bec31acb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:33.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.815+0000 7f2439ffb640 1 -- 172.21.15.111:0/1972055306 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243002e070 con 0x7f24240047c0 2024-09-13T17:58:33.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.815+0000 7f2439ffb640 1 -- 172.21.15.111:0/1972055306 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24300058d0 con 0x7f24240047c0 2024-09-13T17:58:33.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.815+0000 7f2439ffb640 1 -- 172.21.15.111:0/1972055306 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430005ca0 con 0x7f24240047c0 2024-09-13T17:58:33.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.815+0000 7f244151f640 1 -- 172.21.15.111:0/1972055306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24240047c0 msgr2=0x7f2424004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:33.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.815+0000 7f244151f640 1 --2- 172.21.15.111:0/1972055306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24240047c0 0x7f2424004c40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f243000c210 tx=0x7f24240057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.815+0000 7f243b7fe640 1 -- 172.21.15.111:0/1972055306 reap_dead start 2024-09-13T17:58:33.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.816+0000 7f244151f640 1 -- 172.21.15.111:0/1972055306 shutdown_connections 2024-09-13T17:58:33.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.816+0000 7f244151f640 1 -- 172.21.15.111:0/1972055306 >> 172.21.15.111:0/1972055306 conn(0x7f243c0b1f60 msgr2=0x7f243c0b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:33.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.816+0000 7f244151f640 1 -- 172.21.15.111:0/1972055306 shutdown_connections 2024-09-13T17:58:33.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.816+0000 7f244151f640 1 -- 172.21.15.111:0/1972055306 wait complete. 2024-09-13T17:58:33.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.817+0000 7f244151f640 1 Processor -- start 2024-09-13T17:58:33.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.817+0000 7f244151f640 1 -- start start 2024-09-13T17:58:33.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f244151f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24240047c0 0x7f243c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:33.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f244151f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c0ca340 0x7f243c153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:33.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f244151f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f243c0c3cd0 0x7f243c154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:33.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f244151f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0c8a60 con 0x7f24240047c0 2024-09-13T17:58:33.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f244151f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0c4640 con 0x7f243c0c3cd0 2024-09-13T17:58:33.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f244151f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0c4e50 con 0x7f243c0ca340 2024-09-13T17:58:33.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f243affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24240047c0 0x7f243c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:33.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f243affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24240047c0 0x7f243c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58928/0 (socket says 172.21.15.111:58928) 2024-09-13T17:58:33.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f243affd640 1 -- 172.21.15.111:0/1164956221 learned_addr learned my addr 172.21.15.111:0/1164956221 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:33.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f243a7fc640 1 --2- 172.21.15.111:0/1164956221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c0ca340 0x7f243c153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:33.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f243b7fe640 1 --2- 172.21.15.111:0/1164956221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f243c0c3cd0 0x7f243c154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:33.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f243affd640 1 -- 172.21.15.111:0/1164956221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c0ca340 msgr2=0x7f243c153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:33.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f243affd640 1 --2- 172.21.15.111:0/1164956221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c0ca340 0x7f243c153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f243affd640 1 -- 172.21.15.111:0/1164956221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f243c0c3cd0 msgr2=0x7f243c154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:33.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f243affd640 1 --2- 172.21.15.111:0/1164956221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f243c0c3cd0 0x7f243c154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f243affd640 1 -- 172.21.15.111:0/1164956221 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c0c4890 con 0x7f24240047c0 2024-09-13T17:58:33.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f243a7fc640 1 --2- 172.21.15.111:0/1164956221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c0ca340 0x7f243c153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:33.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f243b7fe640 1 --2- 172.21.15.111:0/1164956221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f243c0c3cd0 0x7f243c154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:33.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f243affd640 1 --2- 172.21.15.111:0/1164956221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24240047c0 0x7f243c15abd0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f242c00fc10 tx=0x7f242c00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:33.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f24237fe640 1 -- 172.21.15.111:0/1164956221 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c005160 con 0x7f24240047c0 2024-09-13T17:58:33.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f24237fe640 1 -- 172.21.15.111:0/1164956221 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242c00dbd0 con 0x7f24240047c0 2024-09-13T17:58:33.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f244151f640 1 -- 172.21.15.111:0/1164956221 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243c0c4b70 con 0x7f24240047c0 2024-09-13T17:58:33.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f24237fe640 1 -- 172.21.15.111:0/1164956221 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c014470 con 0x7f24240047c0 2024-09-13T17:58:33.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.818+0000 7f244151f640 1 -- 172.21.15.111:0/1164956221 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243c06e780 con 0x7f24240047c0 2024-09-13T17:58:33.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.819+0000 7f244151f640 1 -- 172.21.15.111:0/1164956221 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f243c0989d0 con 0x7f24240047c0 2024-09-13T17:58:33.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.820+0000 7f24237fe640 1 -- 172.21.15.111:0/1164956221 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f242c0145d0 con 0x7f24240047c0 2024-09-13T17:58:33.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.820+0000 7f24237fe640 1 --2- 172.21.15.111:0/1164956221 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f242806ac30 0x7f242806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:33.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.820+0000 7f24237fe640 1 -- 172.21.15.111:0/1164956221 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f242c08a0d0 con 0x7f24240047c0 2024-09-13T17:58:33.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.820+0000 7f243a7fc640 1 --2- 172.21.15.111:0/1164956221 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f242806ac30 0x7f242806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:33.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.821+0000 7f243a7fc640 1 --2- 172.21.15.111:0/1164956221 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f242806ac30 0x7f242806d0f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2430033000 tx=0x7f243002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:33.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.824+0000 7f24237fe640 1 -- 172.21.15.111:0/1164956221 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f242c054ea0 con 0x7f24240047c0 2024-09-13T17:58:33.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:33.987+0000 7f244151f640 1 -- 172.21.15.111:0/1164956221 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f243c0ccc60 con 0x7f24240047c0 2024-09-13T17:58:34.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.850+0000 7f24237fe640 1 -- 172.21.15.111:0/1164956221 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7f242c0180d0 con 0x7f24240047c0 2024-09-13T17:58:34.852 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' removed 2024-09-13T17:58:34.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.854+0000 7f244151f640 1 -- 172.21.15.111:0/1164956221 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f242806ac30 msgr2=0x7f242806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:34.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.854+0000 7f244151f640 1 --2- 172.21.15.111:0/1164956221 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f242806ac30 0x7f242806d0f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2430033000 tx=0x7f243002e000 comp rx=0 tx=0).stop 2024-09-13T17:58:34.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.854+0000 7f244151f640 1 -- 172.21.15.111:0/1164956221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24240047c0 msgr2=0x7f243c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:34.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.854+0000 7f244151f640 1 --2- 172.21.15.111:0/1164956221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24240047c0 0x7f243c15abd0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f242c00fc10 tx=0x7f242c00d520 comp rx=0 tx=0).stop 2024-09-13T17:58:34.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.854+0000 7f244151f640 1 -- 172.21.15.111:0/1164956221 shutdown_connections 2024-09-13T17:58:34.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.854+0000 7f244151f640 1 --2- 172.21.15.111:0/1164956221 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f242806ac30 0x7f242806d0f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:34.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.854+0000 7f244151f640 1 --2- 172.21.15.111:0/1164956221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f243c0c3cd0 0x7f243c154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:34.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.854+0000 7f244151f640 1 --2- 172.21.15.111:0/1164956221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c0ca340 0x7f243c153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:34.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.854+0000 7f244151f640 1 --2- 172.21.15.111:0/1164956221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24240047c0 0x7f243c15abd0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:34.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.854+0000 7f244151f640 1 -- 172.21.15.111:0/1164956221 >> 172.21.15.111:0/1164956221 conn(0x7f243c0b1f60 msgr2=0x7f243c098be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:34.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.854+0000 7f244151f640 1 -- 172.21.15.111:0/1164956221 shutdown_connections 2024-09-13T17:58:34.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.854+0000 7f244151f640 1 -- 172.21.15.111:0/1164956221 wait complete. 2024-09-13T17:58:34.870 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T17:58:34.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.976+0000 7fa419896640 1 Processor -- start 2024-09-13T17:58:34.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.976+0000 7fa419896640 1 -- start start 2024-09-13T17:58:34.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.977+0000 7fa419896640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa41415fbc0 0x7fa4140cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:34.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.977+0000 7fa419896640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa4140d29e0 0x7fa41406bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:34.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.977+0000 7fa419896640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41406b6c0 con 0x7fa4140cd6c0 2024-09-13T17:58:34.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.977+0000 7fa419896640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4140ce500 con 0x7fa4140d29e0 2024-09-13T17:58:34.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.977+0000 7fa419896640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4140cdc60 con 0x7fa41415fbc0 2024-09-13T17:58:34.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.977+0000 7fa4137fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa4140d29e0 0x7fa41406bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:34.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.977+0000 7fa4137fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa4140d29e0 0x7fa41406bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58938/0 (socket says 172.21.15.111:58938) 2024-09-13T17:58:34.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.977+0000 7fa4137fe640 1 -- 172.21.15.111:0/1705333515 learned_addr learned my addr 172.21.15.111:0/1705333515 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:34.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.978+0000 7fa4137fe640 1 -- 172.21.15.111:0/1705333515 >> v2:172.21.15.188:3300/0 conn(0x7fa41415fbc0 msgr2=0x7fa4140cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:34.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.978+0000 7fa412ffd640 1 --2- 172.21.15.111:0/1705333515 >> v2:172.21.15.188:3300/0 conn(0x7fa41415fbc0 0x7fa4140cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:34.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.978+0000 7fa4137fe640 1 --2- 172.21.15.111:0/1705333515 >> v2:172.21.15.188:3300/0 conn(0x7fa41415fbc0 0x7fa4140cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:34.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.978+0000 7fa4137fe640 1 -- 172.21.15.111:0/1705333515 >> v1:172.21.15.188:6789/0 conn(0x7fa4140cd6c0 legacy=0x7fa4140d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:34.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.978+0000 7fa4137fe640 1 -- 172.21.15.111:0/1705333515 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4140cdf40 con 0x7fa4140d29e0 2024-09-13T17:58:34.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.978+0000 7fa412ffd640 1 --2- 172.21.15.111:0/1705333515 >> v2:172.21.15.188:3300/0 conn(0x7fa41415fbc0 0x7fa4140cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:34.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.978+0000 7fa4137fe640 1 --2- 172.21.15.111:0/1705333515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4140d29e0 0x7fa41406bb50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa41408c4c0 tx=0x7fa40c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78670c158936c78 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:34.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.979+0000 7fa4127fc640 1 -- 172.21.15.111:0/1705333515 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c031600 con 0x7fa4140d29e0 2024-09-13T17:58:34.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.979+0000 7fa4127fc640 1 -- 172.21.15.111:0/1705333515 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c02d050 con 0x7fa4140d29e0 2024-09-13T17:58:34.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.979+0000 7fa4127fc640 1 -- 172.21.15.111:0/1705333515 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c031980 con 0x7fa4140d29e0 2024-09-13T17:58:34.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.979+0000 7fa419896640 1 -- 172.21.15.111:0/1705333515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4140d29e0 msgr2=0x7fa41406bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:34.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.979+0000 7fa419896640 1 --2- 172.21.15.111:0/1705333515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4140d29e0 0x7fa41406bb50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa41408c4c0 tx=0x7fa40c02a6f0 comp rx=0 tx=0).stop 2024-09-13T17:58:34.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.979+0000 7fa419896640 1 -- 172.21.15.111:0/1705333515 shutdown_connections 2024-09-13T17:58:34.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.979+0000 7fa419896640 1 --2- 172.21.15.111:0/1705333515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4140d29e0 0x7fa41406bb50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:34.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.979+0000 7fa419896640 1 --2- 172.21.15.111:0/1705333515 >> v2:172.21.15.188:3300/0 conn(0x7fa41415fbc0 0x7fa4140cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:34.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.979+0000 7fa419896640 1 -- 172.21.15.111:0/1705333515 >> 172.21.15.111:0/1705333515 conn(0x7fa41409b830 msgr2=0x7fa414089f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:34.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.979+0000 7fa419896640 1 -- 172.21.15.111:0/1705333515 shutdown_connections 2024-09-13T17:58:34.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.979+0000 7fa419896640 1 -- 172.21.15.111:0/1705333515 wait complete. 2024-09-13T17:58:34.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.980+0000 7fa419896640 1 Processor -- start 2024-09-13T17:58:34.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.981+0000 7fa419896640 1 -- start start 2024-09-13T17:58:34.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.981+0000 7fa419896640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4140cd6c0 0x7fa4140b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:34.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.981+0000 7fa419896640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4140d29e0 0x7fa414156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:34.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.981+0000 7fa419896640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa41415fbc0 0x7fa4140b69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:34.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.981+0000 7fa419896640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa414156780 con 0x7fa4140cd6c0 2024-09-13T17:58:34.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.981+0000 7fa419896640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140b7c70 con 0x7fa41415fbc0 2024-09-13T17:58:34.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.981+0000 7fa419896640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa414159120 con 0x7fa4140d29e0 2024-09-13T17:58:34.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.981+0000 7fa412ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4140cd6c0 0x7fa4140b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:34.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.981+0000 7fa412ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4140cd6c0 0x7fa4140b7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58948/0 (socket says 172.21.15.111:58948) 2024-09-13T17:58:34.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.981+0000 7fa412ffd640 1 -- 172.21.15.111:0/587416068 learned_addr learned my addr 172.21.15.111:0/587416068 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:34.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.981+0000 7fa4137fe640 1 --2- 172.21.15.111:0/587416068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa41415fbc0 0x7fa4140b69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:34.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.981+0000 7fa40a7fc640 1 --2- 172.21.15.111:0/587416068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4140d29e0 0x7fa414156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:34.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.982+0000 7fa412ffd640 1 -- 172.21.15.111:0/587416068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4140d29e0 msgr2=0x7fa414156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:34.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.982+0000 7fa412ffd640 1 --2- 172.21.15.111:0/587416068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4140d29e0 0x7fa414156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:34.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.982+0000 7fa412ffd640 1 -- 172.21.15.111:0/587416068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa41415fbc0 msgr2=0x7fa4140b69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:34.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.982+0000 7fa412ffd640 1 --2- 172.21.15.111:0/587416068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa41415fbc0 0x7fa4140b69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:34.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.982+0000 7fa412ffd640 1 -- 172.21.15.111:0/587416068 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c009b30 con 0x7fa4140cd6c0 2024-09-13T17:58:34.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.982+0000 7fa412ffd640 1 --2- 172.21.15.111:0/587416068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4140cd6c0 0x7fa4140b7e50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa40400c7f0 tx=0x7fa40400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:34.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.982+0000 7fa410ff9640 1 -- 172.21.15.111:0/587416068 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404007a30 con 0x7fa4140cd6c0 2024-09-13T17:58:34.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.982+0000 7fa410ff9640 1 -- 172.21.15.111:0/587416068 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa404007b90 con 0x7fa4140cd6c0 2024-09-13T17:58:34.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.982+0000 7fa419896640 1 -- 172.21.15.111:0/587416068 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4141587f0 con 0x7fa4140cd6c0 2024-09-13T17:58:34.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.982+0000 7fa410ff9640 1 -- 172.21.15.111:0/587416068 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40400ce40 con 0x7fa4140cd6c0 2024-09-13T17:58:34.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.982+0000 7fa419896640 1 -- 172.21.15.111:0/587416068 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa414158db0 con 0x7fa4140cd6c0 2024-09-13T17:58:34.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.983+0000 7fa419896640 1 -- 172.21.15.111:0/587416068 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4141592b0 con 0x7fa4140cd6c0 2024-09-13T17:58:34.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.983+0000 7fa410ff9640 1 -- 172.21.15.111:0/587416068 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa4040113f0 con 0x7fa4140cd6c0 2024-09-13T17:58:34.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.984+0000 7fa410ff9640 1 --2- 172.21.15.111:0/587416068 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa3f406ac00 0x7fa3f406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:34.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.984+0000 7fa410ff9640 1 -- 172.21.15.111:0/587416068 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6311+0+0 (secure 0 0 0) 0x7fa404089720 con 0x7fa4140cd6c0 2024-09-13T17:58:34.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.984+0000 7fa40a7fc640 1 --2- 172.21.15.111:0/587416068 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa3f406ac00 0x7fa3f406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:34.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.984+0000 7fa40a7fc640 1 --2- 172.21.15.111:0/587416068 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa3f406ac00 0x7fa3f406d0c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa4140b76c0 tx=0x7fa3fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:34.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:34.988+0000 7fa410ff9640 1 -- 172.21.15.111:0/587416068 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fa4040546a0 con 0x7fa4140cd6c0 2024-09-13T17:58:35.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.158+0000 7fa419896640 1 -- 172.21.15.111:0/587416068 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa4140af090 con 0x7fa4140cd6c0 2024-09-13T17:58:35.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.852+0000 7fa410ff9640 1 -- 172.21.15.111:0/587416068 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa404057cc0 con 0x7fa4140cd6c0 2024-09-13T17:58:35.854 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' removed 2024-09-13T17:58:35.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.855+0000 7fa419896640 1 -- 172.21.15.111:0/587416068 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa3f406ac00 msgr2=0x7fa3f406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:35.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.855+0000 7fa419896640 1 --2- 172.21.15.111:0/587416068 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa3f406ac00 0x7fa3f406d0c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa4140b76c0 tx=0x7fa3fc008040 comp rx=0 tx=0).stop 2024-09-13T17:58:35.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.855+0000 7fa419896640 1 -- 172.21.15.111:0/587416068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4140cd6c0 msgr2=0x7fa4140b7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.855+0000 7fa419896640 1 --2- 172.21.15.111:0/587416068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4140cd6c0 0x7fa4140b7e50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa40400c7f0 tx=0x7fa40400cbf0 comp rx=0 tx=0).stop 2024-09-13T17:58:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.855+0000 7fa419896640 1 -- 172.21.15.111:0/587416068 shutdown_connections 2024-09-13T17:58:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.855+0000 7fa419896640 1 --2- 172.21.15.111:0/587416068 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa3f406ac00 0x7fa3f406d0c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.855+0000 7fa419896640 1 --2- 172.21.15.111:0/587416068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa41415fbc0 0x7fa4140b69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.855+0000 7fa419896640 1 --2- 172.21.15.111:0/587416068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4140d29e0 0x7fa414156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.855+0000 7fa419896640 1 --2- 172.21.15.111:0/587416068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4140cd6c0 0x7fa4140b7e50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.855+0000 7fa419896640 1 -- 172.21.15.111:0/587416068 >> 172.21.15.111:0/587416068 conn(0x7fa41409b830 msgr2=0x7fa4140c8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.856+0000 7fa419896640 1 -- 172.21.15.111:0/587416068 shutdown_connections 2024-09-13T17:58:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.856+0000 7fa419896640 1 -- 172.21.15.111:0/587416068 wait complete. 2024-09-13T17:58:35.870 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-13T17:58:35.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.974+0000 7f8f9c183640 1 Processor -- start 2024-09-13T17:58:35.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.974+0000 7f8f9c183640 1 -- start start 2024-09-13T17:58:35.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.975+0000 7f8f9c183640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f940b2b40 0x7f8f940ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:35.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.975+0000 7f8f9c183640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8f940ce4f0 0x7f8f940ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:35.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.975+0000 7f8f9c183640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f940ccbe0 con 0x7f8f94159310 2024-09-13T17:58:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.975+0000 7f8f9c183640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f940ad940 con 0x7f8f940ce4f0 2024-09-13T17:58:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.975+0000 7f8f9c183640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f940ceea0 con 0x7f8f940b2b40 2024-09-13T17:58:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.975+0000 7f8f9a720640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8f940ce4f0 0x7f8f940ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.975+0000 7f8f9971e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f940b2b40 0x7f8f940ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.975+0000 7f8f99f1f640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f8f94159310 0x7f8f940ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:44650/0 (socket says 172.21.15.111:44650) 2024-09-13T17:58:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.975+0000 7f8f99f1f640 1 -- 172.21.15.111:0/3582694322 learned_addr learned my addr 172.21.15.111:0/3582694322 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.975+0000 7f8f9971e640 1 -- 172.21.15.111:0/3582694322 >> v2:172.21.15.188:3300/0 conn(0x7f8f940ce4f0 msgr2=0x7f8f940ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.975+0000 7f8f9971e640 1 --2- 172.21.15.111:0/3582694322 >> v2:172.21.15.188:3300/0 conn(0x7f8f940ce4f0 0x7f8f940ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.975+0000 7f8f9971e640 1 -- 172.21.15.111:0/3582694322 >> v1:172.21.15.188:6789/0 conn(0x7f8f94159310 legacy=0x7f8f940ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.975+0000 7f8f9971e640 1 -- 172.21.15.111:0/3582694322 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f940b1810 con 0x7f8f940b2b40 2024-09-13T17:58:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.975+0000 7f8f9a720640 1 --2- 172.21.15.111:0/3582694322 >> v2:172.21.15.188:3300/0 conn(0x7f8f940ce4f0 0x7f8f940ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:35.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.976+0000 7f8f9971e640 1 --2- 172.21.15.111:0/3582694322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f940b2b40 0x7f8f940ad400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f840097a0 tx=0x7f8f8402a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f9d59ac485fe581 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:35.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.976+0000 7f8f98f1d640 1 -- 172.21.15.111:0/3582694322 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8402adb0 con 0x7f8f940b2b40 2024-09-13T17:58:35.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.976+0000 7f8f98f1d640 1 -- 172.21.15.111:0/3582694322 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f84032440 con 0x7f8f940b2b40 2024-09-13T17:58:35.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.976+0000 7f8f98f1d640 1 -- 172.21.15.111:0/3582694322 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f840327d0 con 0x7f8f940b2b40 2024-09-13T17:58:35.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.977+0000 7f8f9c183640 1 -- 172.21.15.111:0/3582694322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f940b2b40 msgr2=0x7f8f940ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:35.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.977+0000 7f8f9c183640 1 --2- 172.21.15.111:0/3582694322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f940b2b40 0x7f8f940ad400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f840097a0 tx=0x7f8f8402a060 comp rx=0 tx=0).stop 2024-09-13T17:58:35.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.977+0000 7f8f9c183640 1 -- 172.21.15.111:0/3582694322 shutdown_connections 2024-09-13T17:58:35.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.977+0000 7f8f9c183640 1 --2- 172.21.15.111:0/3582694322 >> v2:172.21.15.188:3300/0 conn(0x7f8f940ce4f0 0x7f8f940ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:35.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.977+0000 7f8f9c183640 1 --2- 172.21.15.111:0/3582694322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f940b2b40 0x7f8f940ad400 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:35.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.977+0000 7f8f9c183640 1 -- 172.21.15.111:0/3582694322 >> 172.21.15.111:0/3582694322 conn(0x7f8f9406a0b0 msgr2=0x7f8f9406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:35.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.977+0000 7f8f9c183640 1 -- 172.21.15.111:0/3582694322 shutdown_connections 2024-09-13T17:58:35.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.977+0000 7f8f9c183640 1 -- 172.21.15.111:0/3582694322 wait complete. 2024-09-13T17:58:35.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.978+0000 7f8f9c183640 1 Processor -- start 2024-09-13T17:58:35.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.978+0000 7f8f9c183640 1 -- start start 2024-09-13T17:58:35.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.979+0000 7f8f9c183640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f940b2b40 0x7f8f941eed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:35.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.979+0000 7f8f9c183640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f940ce4f0 0x7f8f94099700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:35.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.979+0000 7f8f9c183640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f94159310 0x7f8f9409b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:35.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.979+0000 7f8f9c183640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f941ef270 con 0x7f8f940ce4f0 2024-09-13T17:58:35.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.979+0000 7f8f9c183640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9409a1b0 con 0x7f8f940b2b40 2024-09-13T17:58:35.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.979+0000 7f8f9c183640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9409a490 con 0x7f8f94159310 2024-09-13T17:58:35.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.979+0000 7f8f9971e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f940ce4f0 0x7f8f94099700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:35.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.979+0000 7f8f9971e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f940ce4f0 0x7f8f94099700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58950/0 (socket says 172.21.15.111:58950) 2024-09-13T17:58:35.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.979+0000 7f8f9971e640 1 -- 172.21.15.111:0/2554362182 learned_addr learned my addr 172.21.15.111:0/2554362182 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:35.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.979+0000 7f8f9a720640 1 --2- 172.21.15.111:0/2554362182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f94159310 0x7f8f9409b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:35.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.979+0000 7f8f99f1f640 1 --2- 172.21.15.111:0/2554362182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f940b2b40 0x7f8f941eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:35.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.979+0000 7f8f9971e640 1 -- 172.21.15.111:0/2554362182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f94159310 msgr2=0x7f8f9409b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:35.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.979+0000 7f8f9971e640 1 --2- 172.21.15.111:0/2554362182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f94159310 0x7f8f9409b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:35.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.979+0000 7f8f9971e640 1 -- 172.21.15.111:0/2554362182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f940b2b40 msgr2=0x7f8f941eed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:35.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.979+0000 7f8f9971e640 1 --2- 172.21.15.111:0/2554362182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f940b2b40 0x7f8f941eed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:35.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.979+0000 7f8f9971e640 1 -- 172.21.15.111:0/2554362182 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8c009400 con 0x7f8f940ce4f0 2024-09-13T17:58:35.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.980+0000 7f8f9971e640 1 --2- 172.21.15.111:0/2554362182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f940ce4f0 0x7f8f94099700 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8f84000c00 tx=0x7f8f840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:35.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.980+0000 7f8f8a7fc640 1 -- 172.21.15.111:0/2554362182 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8403d850 con 0x7f8f940ce4f0 2024-09-13T17:58:35.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.980+0000 7f8f8a7fc640 1 -- 172.21.15.111:0/2554362182 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8403d9d0 con 0x7f8f940ce4f0 2024-09-13T17:58:35.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.980+0000 7f8f9c183640 1 -- 172.21.15.111:0/2554362182 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f84009400 con 0x7f8f940ce4f0 2024-09-13T17:58:35.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.980+0000 7f8f8a7fc640 1 -- 172.21.15.111:0/2554362182 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8403dd80 con 0x7f8f940ce4f0 2024-09-13T17:58:35.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.980+0000 7f8f9c183640 1 -- 172.21.15.111:0/2554362182 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9409aa50 con 0x7f8f940ce4f0 2024-09-13T17:58:35.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.981+0000 7f8f9c183640 1 -- 172.21.15.111:0/2554362182 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f94066f30 con 0x7f8f940ce4f0 2024-09-13T17:58:35.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.981+0000 7f8f8a7fc640 1 -- 172.21.15.111:0/2554362182 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8f84039040 con 0x7f8f940ce4f0 2024-09-13T17:58:35.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.981+0000 7f8f8a7fc640 1 --2- 172.21.15.111:0/2554362182 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8f7806ac00 0x7f8f7806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:35.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.982+0000 7f8f8a7fc640 1 -- 172.21.15.111:0/2554362182 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5781+0+0 (secure 0 0 0) 0x7f8f840a9d70 con 0x7f8f940ce4f0 2024-09-13T17:58:35.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.982+0000 7f8f99f1f640 1 --2- 172.21.15.111:0/2554362182 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8f7806ac00 0x7f8f7806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:35.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.982+0000 7f8f99f1f640 1 --2- 172.21.15.111:0/2554362182 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8f7806ac00 0x7f8f7806d0c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8f90005c00 tx=0x7f8f90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:35.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:35.985+0000 7f8f8a7fc640 1 -- 172.21.15.111:0/2554362182 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f8f84074e70 con 0x7f8f940ce4f0 2024-09-13T17:58:36.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:36.149+0000 7f8f9c183640 1 -- 172.21.15.111:0/2554362182 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8f940cdfb0 con 0x7f8f940ce4f0 2024-09-13T17:58:36.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:36.864+0000 7f8f8a7fc640 1 -- 172.21.15.111:0/2554362182 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7f8f840370d0 con 0x7f8f940ce4f0 2024-09-13T17:58:36.866 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data_ec' removed 2024-09-13T17:58:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:36.869+0000 7f8f9c183640 1 -- 172.21.15.111:0/2554362182 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8f7806ac00 msgr2=0x7f8f7806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:36.869+0000 7f8f9c183640 1 --2- 172.21.15.111:0/2554362182 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8f7806ac00 0x7f8f7806d0c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8f90005c00 tx=0x7f8f90009040 comp rx=0 tx=0).stop 2024-09-13T17:58:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:36.869+0000 7f8f9c183640 1 -- 172.21.15.111:0/2554362182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f940ce4f0 msgr2=0x7f8f94099700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:36.869+0000 7f8f9c183640 1 --2- 172.21.15.111:0/2554362182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f940ce4f0 0x7f8f94099700 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8f84000c00 tx=0x7f8f840046c0 comp rx=0 tx=0).stop 2024-09-13T17:58:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:36.869+0000 7f8f9c183640 1 -- 172.21.15.111:0/2554362182 shutdown_connections 2024-09-13T17:58:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:36.869+0000 7f8f9c183640 1 --2- 172.21.15.111:0/2554362182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f94159310 0x7f8f9409b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:36.869+0000 7f8f9c183640 1 --2- 172.21.15.111:0/2554362182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f940ce4f0 0x7f8f94099700 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:36.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:36.869+0000 7f8f9c183640 1 --2- 172.21.15.111:0/2554362182 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f8f7806ac00 0x7f8f7806d0c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:36.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:36.869+0000 7f8f9c183640 1 --2- 172.21.15.111:0/2554362182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f940b2b40 0x7f8f941eed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:36.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:36.869+0000 7f8f9c183640 1 -- 172.21.15.111:0/2554362182 >> 172.21.15.111:0/2554362182 conn(0x7f8f9406a0b0 msgr2=0x7f8f940aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:36.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:36.869+0000 7f8f9c183640 1 -- 172.21.15.111:0/2554362182 shutdown_connections 2024-09-13T17:58:36.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:36.869+0000 7f8f9c183640 1 -- 172.21.15.111:0/2554362182 wait complete. 2024-09-13T17:58:36.898 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:58:36.898 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T17:58:36.898 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:58:36.898 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:58:36.898 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T17:58:36.898 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T17:58:36.899 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T17:58:36.899 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:58:36.899 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T17:58:36.899 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T17:58:36.899 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T17:58:36.899 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:58:36.899 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T17:58:36.899 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T17:58:36.900 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T17:58:36.900 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:58:36.908 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:58:36.908 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:58:36.909 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T17:58:36.909 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T17:58:36.910 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T17:58:36.910 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T17:58:36.911 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T17:58:36.911 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T17:58:36.951 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:58:36.952 DEBUG:teuthology.parallel:result is None 2024-09-13T17:58:36.953 INFO:tasks.ceph.mds.c:Started 2024-09-13T17:58:36.953 DEBUG:teuthology.parallel:result is None 2024-09-13T17:58:36.954 INFO:tasks.ceph.mds.b:Started 2024-09-13T17:58:36.954 DEBUG:teuthology.parallel:result is None 2024-09-13T17:58:36.955 INFO:tasks.ceph.mds.d:Started 2024-09-13T17:58:36.955 DEBUG:teuthology.parallel:result is None 2024-09-13T17:58:36.955 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T17:58:37.030 INFO:tasks.ceph.mds.c.smithi111.stdout:starting mds.c at 2024-09-13T17:58:37.031 INFO:tasks.ceph.mds.a.smithi111.stdout:starting mds.a at 2024-09-13T17:58:37.034 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-09-13T17:58:37.034 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-09-13T17:58:37.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.063+0000 7f582355a640 1 Processor -- start 2024-09-13T17:58:37.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.063+0000 7f582355a640 1 -- start start 2024-09-13T17:58:37.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.064+0000 7f582355a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c15c6e0 0x7f581c0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:37.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.064+0000 7f582355a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f581c0d07b0 0x7f581c0b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:37.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.064+0000 7f582355a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581c0d1840 con 0x7f581c0ccb20 2024-09-13T17:58:37.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.064+0000 7f5820af5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f581c0d07b0 0x7f581c0b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:37.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.064+0000 7f5820af5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f581c0d07b0 0x7f581c0b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58992/0 (socket says 172.21.15.111:58992) 2024-09-13T17:58:37.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.064+0000 7f5820af5640 1 -- 172.21.15.111:0/3657649798 learned_addr learned my addr 172.21.15.111:0/3657649798 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:37.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.064+0000 7f582355a640 1 -- 172.21.15.111:0/3657649798 --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581c06bd20 con 0x7f581c0d07b0 2024-09-13T17:58:37.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.064+0000 7f58212f6640 1 --2- 172.21.15.111:0/3657649798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c15c6e0 0x7f581c0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:37.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.064+0000 7f582355a640 1 -- 172.21.15.111:0/3657649798 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0d0c30 con 0x7f581c15c6e0 2024-09-13T17:58:37.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.064+0000 7f5813fff640 1 -- 172.21.15.111:0/3657649798 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535814800 0 0) 0x7f581c0d1840 con 0x7f581c0ccb20 2024-09-13T17:58:37.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.064+0000 7f5813fff640 1 -- 172.21.15.111:0/3657649798 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5808003520 con 0x7f581c0ccb20 2024-09-13T17:58:37.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.064+0000 7f5820af5640 1 -- 172.21.15.111:0/3657649798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c15c6e0 msgr2=0x7f581c0b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:37.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.064+0000 7f5820af5640 1 --2- 172.21.15.111:0/3657649798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c15c6e0 0x7f581c0b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:37.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.064+0000 7f5820af5640 1 -- 172.21.15.111:0/3657649798 >> v1:172.21.15.111:6789/0 conn(0x7f581c0ccb20 legacy=0x7f581c160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:37.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.064+0000 7f5820af5640 1 -- 172.21.15.111:0/3657649798 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c0d0f10 con 0x7f581c0d07b0 2024-09-13T17:58:37.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.065+0000 7f5820af5640 1 --2- 172.21.15.111:0/3657649798 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c0d07b0 0x7f581c0b4490 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f580c009870 tx=0x7f580c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb95949ee44db447 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:37.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.065+0000 7f5813fff640 1 -- 172.21.15.111:0/3657649798 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c032440 con 0x7f581c0d07b0 2024-09-13T17:58:37.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.065+0000 7f5813fff640 1 -- 172.21.15.111:0/3657649798 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c0325a0 con 0x7f581c0d07b0 2024-09-13T17:58:37.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.065+0000 7f5813fff640 1 -- 172.21.15.111:0/3657649798 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c032930 con 0x7f581c0d07b0 2024-09-13T17:58:37.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.066+0000 7f582355a640 1 -- 172.21.15.111:0/3657649798 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c0d07b0 msgr2=0x7f581c0b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:37.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.066+0000 7f582355a640 1 --2- 172.21.15.111:0/3657649798 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c0d07b0 0x7f581c0b4490 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f580c009870 tx=0x7f580c02a1c0 comp rx=0 tx=0).stop 2024-09-13T17:58:37.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.066+0000 7f582355a640 1 -- 172.21.15.111:0/3657649798 shutdown_connections 2024-09-13T17:58:37.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.066+0000 7f582355a640 1 --2- 172.21.15.111:0/3657649798 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c0d07b0 0x7f581c0b4490 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:37.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.066+0000 7f582355a640 1 --2- 172.21.15.111:0/3657649798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c15c6e0 0x7f581c0b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:37.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.066+0000 7f582355a640 1 -- 172.21.15.111:0/3657649798 >> 172.21.15.111:0/3657649798 conn(0x7f581c089f60 msgr2=0x7f581c0d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:37.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.066+0000 7f582355a640 1 -- 172.21.15.111:0/3657649798 shutdown_connections 2024-09-13T17:58:37.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.066+0000 7f582355a640 1 -- 172.21.15.111:0/3657649798 wait complete. 2024-09-13T17:58:37.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.067+0000 7f582355a640 1 Processor -- start 2024-09-13T17:58:37.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.067+0000 7f582355a640 1 -- start start 2024-09-13T17:58:37.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f582355a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c0ccb20 0x7f581c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:37.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f582355a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f581c15c6e0 0x7f581c153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:37.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f582355a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c158260 0x7f581c0b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:37.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f582355a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c1586f0 con 0x7f581c158260 2024-09-13T17:58:37.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f582355a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0b7380 con 0x7f581c15c6e0 2024-09-13T17:58:37.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f582355a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0b7660 con 0x7f581c0ccb20 2024-09-13T17:58:37.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f5821af7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c158260 0x7f581c0b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:37.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f5821af7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c158260 0x7f581c0b0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59002/0 (socket says 172.21.15.111:59002) 2024-09-13T17:58:37.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f5821af7640 1 -- 172.21.15.111:0/2302031418 learned_addr learned my addr 172.21.15.111:0/2302031418 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:37.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f58212f6640 1 --2- 172.21.15.111:0/2302031418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c0ccb20 0x7f581c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:37.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f5820af5640 1 --2- 172.21.15.111:0/2302031418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f581c15c6e0 0x7f581c153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:37.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f5821af7640 1 -- 172.21.15.111:0/2302031418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c0ccb20 msgr2=0x7f581c154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:37.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f5821af7640 1 --2- 172.21.15.111:0/2302031418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c0ccb20 0x7f581c154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:37.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f5821af7640 1 -- 172.21.15.111:0/2302031418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f581c15c6e0 msgr2=0x7f581c153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:37.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f5821af7640 1 --2- 172.21.15.111:0/2302031418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f581c15c6e0 0x7f581c153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:37.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f5821af7640 1 -- 172.21.15.111:0/2302031418 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c0094d0 con 0x7f581c158260 2024-09-13T17:58:37.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f5821af7640 1 --2- 172.21.15.111:0/2302031418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c158260 0x7f581c0b0490 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f581400c6b0 tx=0x7f581400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:37.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f5811ffb640 1 -- 172.21.15.111:0/2302031418 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814003b70 con 0x7f581c158260 2024-09-13T17:58:37.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f5811ffb640 1 -- 172.21.15.111:0/2302031418 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5814003cd0 con 0x7f581c158260 2024-09-13T17:58:37.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f582355a640 1 -- 172.21.15.111:0/2302031418 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581c0b7940 con 0x7f581c158260 2024-09-13T17:58:37.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.068+0000 7f5811ffb640 1 -- 172.21.15.111:0/2302031418 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814005310 con 0x7f581c158260 2024-09-13T17:58:37.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.069+0000 7f582355a640 1 -- 172.21.15.111:0/2302031418 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581c153fc0 con 0x7f581c158260 2024-09-13T17:58:37.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.070+0000 7f582355a640 1 -- 172.21.15.111:0/2302031418 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581c0b7e60 con 0x7f581c158260 2024-09-13T17:58:37.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.070+0000 7f5811ffb640 1 -- 172.21.15.111:0/2302031418 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5814005470 con 0x7f581c158260 2024-09-13T17:58:37.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.070+0000 7f5811ffb640 1 --2- 172.21.15.111:0/2302031418 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f580006ac00 0x7f580006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:37.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.070+0000 7f5811ffb640 1 -- 172.21.15.111:0/2302031418 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f5814088db0 con 0x7f581c158260 2024-09-13T17:58:37.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.070+0000 7f58212f6640 1 --2- 172.21.15.111:0/2302031418 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f580006ac00 0x7f580006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:37.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.071+0000 7f58212f6640 1 --2- 172.21.15.111:0/2302031418 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f580006ac00 0x7f580006d0c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f581c1f60a0 tx=0x7f5818009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:37.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.075+0000 7f5811ffb640 1 -- 172.21.15.111:0/2302031418 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f581c0b7e60 con 0x7f581c158260 2024-09-13T17:58:37.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:37.247+0000 7f582355a640 1 -- 172.21.15.111:0/2302031418 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f581c0b7e60 con 0x7f581c158260 2024-09-13T17:58:38.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.065+0000 7f5811ffb640 1 -- 172.21.15.111:0/2302031418 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) v1 ==== 97+0+0 (secure 0 0 0) 0x7f581c0b7e60 con 0x7f581c158260 2024-09-13T17:58:38.067 INFO:teuthology.orchestra.run.smithi111.stderr: removed all blocklist entries 2024-09-13T17:58:38.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.067+0000 7f582355a640 1 -- 172.21.15.111:0/2302031418 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f580006ac00 msgr2=0x7f580006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.067+0000 7f582355a640 1 --2- 172.21.15.111:0/2302031418 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f580006ac00 0x7f580006d0c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f581c1f60a0 tx=0x7f5818009190 comp rx=0 tx=0).stop 2024-09-13T17:58:38.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.067+0000 7f582355a640 1 -- 172.21.15.111:0/2302031418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c158260 msgr2=0x7f581c0b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.067+0000 7f582355a640 1 --2- 172.21.15.111:0/2302031418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c158260 0x7f581c0b0490 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f581400c6b0 tx=0x7f581400cb70 comp rx=0 tx=0).stop 2024-09-13T17:58:38.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.067+0000 7f582355a640 1 -- 172.21.15.111:0/2302031418 shutdown_connections 2024-09-13T17:58:38.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.067+0000 7f582355a640 1 --2- 172.21.15.111:0/2302031418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f581c158260 0x7f581c0b0490 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.067+0000 7f582355a640 1 --2- 172.21.15.111:0/2302031418 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f580006ac00 0x7f580006d0c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.067+0000 7f582355a640 1 --2- 172.21.15.111:0/2302031418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f581c15c6e0 0x7f581c153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.067+0000 7f582355a640 1 --2- 172.21.15.111:0/2302031418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c0ccb20 0x7f581c154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.067+0000 7f582355a640 1 -- 172.21.15.111:0/2302031418 >> 172.21.15.111:0/2302031418 conn(0x7f581c089f60 msgr2=0x7f581c06b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:38.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.067+0000 7f582355a640 1 -- 172.21.15.111:0/2302031418 shutdown_connections 2024-09-13T17:58:38.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.067+0000 7f582355a640 1 -- 172.21.15.111:0/2302031418 wait complete. 2024-09-13T17:58:38.083 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T17:58:38.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.184+0000 7f6c6a8a3640 1 Processor -- start 2024-09-13T17:58:38.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.184+0000 7f6c6a8a3640 1 -- start start 2024-09-13T17:58:38.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c6a8a3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c640cace0 0x7f6c640cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c6a8a3640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c640c8920 con 0x7f6c6406d030 2024-09-13T17:58:38.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c6a8a3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6415a6c0 con 0x7f6c6415a2e0 2024-09-13T17:58:38.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c6a8a3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c640cb690 con 0x7f6c640cace0 2024-09-13T17:58:38.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c68e40640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6c6406d030 0x7f6c6406d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48710/0 (socket says 172.21.15.111:48710) 2024-09-13T17:58:38.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c68e40640 1 -- 172.21.15.111:0/4117832905 learned_addr learned my addr 172.21.15.111:0/4117832905 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:38.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c62ffd640 1 -- 172.21.15.111:0/4117832905 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645596572 0 0) 0x7f6c640c8920 con 0x7f6c6406d030 2024-09-13T17:58:38.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c637fe640 1 --2- 172.21.15.111:0/4117832905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c640cace0 0x7f6c640cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c62ffd640 1 -- 172.21.15.111:0/4117832905 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c50003520 con 0x7f6c6406d030 2024-09-13T17:58:38.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c62ffd640 1 -- 172.21.15.111:0/4117832905 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 714659879 0 0) 0x7f6c50003520 con 0x7f6c6406d030 2024-09-13T17:58:38.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c62ffd640 1 -- 172.21.15.111:0/4117832905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c640cace0 msgr2=0x7f6c640cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:38.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c62ffd640 1 --2- 172.21.15.111:0/4117832905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c640cace0 0x7f6c640cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c62ffd640 1 -- 172.21.15.111:0/4117832905 >> v1:172.21.15.188:6789/0 conn(0x7f6c6415a2e0 legacy=0x7f6c640c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c62ffd640 1 -- 172.21.15.111:0/4117832905 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c640ca500 con 0x7f6c6406d030 2024-09-13T17:58:38.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c62ffd640 1 -- 172.21.15.111:0/4117832905 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f6c5c0019f0 con 0x7f6c6406d030 2024-09-13T17:58:38.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c62ffd640 1 -- 172.21.15.111:0/4117832905 >> v1:172.21.15.111:6789/0 conn(0x7f6c6406d030 legacy=0x7f6c6406d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c62ffd640 1 --2- 172.21.15.111:0/4117832905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c500039a0 0x7f6c50003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c62ffd640 1 --2- 172.21.15.111:0/4117832905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c50004350 0x7f6c50004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c62ffd640 1 --2- 172.21.15.111:0/4117832905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c640cace0 0x7f6c50005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c62ffd640 1 -- 172.21.15.111:0/4117832905 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c640cb690 con 0x7f6c50004350 2024-09-13T17:58:38.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c62ffd640 1 -- 172.21.15.111:0/4117832905 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5000b9b0 con 0x7f6c500039a0 2024-09-13T17:58:38.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c62ffd640 1 -- 172.21.15.111:0/4117832905 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5000bb70 con 0x7f6c640cace0 2024-09-13T17:58:38.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.185+0000 7f6c637fe640 1 --2- 172.21.15.111:0/4117832905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c50004350 0x7f6c50004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.186+0000 7f6c68e40640 1 --2- 172.21.15.111:0/4117832905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c640cace0 0x7f6c50005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.186+0000 7f6c63fff640 1 --2- 172.21.15.111:0/4117832905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c500039a0 0x7f6c50003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.186+0000 7f6c637fe640 1 -- 172.21.15.111:0/4117832905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c640cace0 msgr2=0x7f6c50005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.186+0000 7f6c637fe640 1 --2- 172.21.15.111:0/4117832905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c640cace0 0x7f6c50005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.186+0000 7f6c637fe640 1 -- 172.21.15.111:0/4117832905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c500039a0 msgr2=0x7f6c50003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.186+0000 7f6c637fe640 1 --2- 172.21.15.111:0/4117832905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c500039a0 0x7f6c50003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.186+0000 7f6c637fe640 1 -- 172.21.15.111:0/4117832905 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c50003520 con 0x7f6c50004350 2024-09-13T17:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.186+0000 7f6c637fe640 1 --2- 172.21.15.111:0/4117832905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c50004350 0x7f6c50004820 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009ac0 tx=0x7f6c500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e285a9f0e09ceb82 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.187+0000 7f6c62ffd640 1 -- 172.21.15.111:0/4117832905 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c031400 con 0x7f6c50004350 2024-09-13T17:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.187+0000 7f6c62ffd640 1 -- 172.21.15.111:0/4117832905 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c02d050 con 0x7f6c50004350 2024-09-13T17:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.187+0000 7f6c62ffd640 1 -- 172.21.15.111:0/4117832905 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c031780 con 0x7f6c50004350 2024-09-13T17:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.187+0000 7f6c6a8a3640 1 -- 172.21.15.111:0/4117832905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c50004350 msgr2=0x7f6c50004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.187+0000 7f6c6a8a3640 1 --2- 172.21.15.111:0/4117832905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c50004350 0x7f6c50004820 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009ac0 tx=0x7f6c500057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.187+0000 7f6c68e40640 1 -- 172.21.15.111:0/4117832905 reap_dead start 2024-09-13T17:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.187+0000 7f6c6a8a3640 1 -- 172.21.15.111:0/4117832905 shutdown_connections 2024-09-13T17:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.187+0000 7f6c6a8a3640 1 -- 172.21.15.111:0/4117832905 >> 172.21.15.111:0/4117832905 conn(0x7f6c64069ed0 msgr2=0x7f6c6406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:38.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.187+0000 7f6c6a8a3640 1 -- 172.21.15.111:0/4117832905 shutdown_connections 2024-09-13T17:58:38.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.187+0000 7f6c6a8a3640 1 -- 172.21.15.111:0/4117832905 wait complete. 2024-09-13T17:58:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.188+0000 7f6c6a8a3640 1 Processor -- start 2024-09-13T17:58:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.189+0000 7f6c6a8a3640 1 -- start start 2024-09-13T17:58:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.189+0000 7f6c6a8a3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c50004350 0x7f6c64156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.189+0000 7f6c6a8a3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c640c7df0 0x7f6c64098370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.189+0000 7f6c6a8a3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c640988b0 0x7f6c6409a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.189+0000 7f6c6a8a3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6415d020 con 0x7f6c640988b0 2024-09-13T17:58:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.189+0000 7f6c6a8a3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c64156ad0 con 0x7f6c50004350 2024-09-13T17:58:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.189+0000 7f6c6a8a3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c64098d60 con 0x7f6c640c7df0 2024-09-13T17:58:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.189+0000 7f6c68e40640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c640988b0 0x7f6c6409a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.189+0000 7f6c63fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c50004350 0x7f6c64156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.189+0000 7f6c68e40640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c640988b0 0x7f6c6409a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59030/0 (socket says 172.21.15.111:59030) 2024-09-13T17:58:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.189+0000 7f6c68e40640 1 -- 172.21.15.111:0/3845517267 learned_addr learned my addr 172.21.15.111:0/3845517267 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.190+0000 7f6c637fe640 1 --2- 172.21.15.111:0/3845517267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c640c7df0 0x7f6c64098370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.190+0000 7f6c68e40640 1 -- 172.21.15.111:0/3845517267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c640c7df0 msgr2=0x7f6c64098370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.190+0000 7f6c68e40640 1 --2- 172.21.15.111:0/3845517267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c640c7df0 0x7f6c64098370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.190+0000 7f6c68e40640 1 -- 172.21.15.111:0/3845517267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c50004350 msgr2=0x7f6c64156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.190+0000 7f6c68e40640 1 --2- 172.21.15.111:0/3845517267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c50004350 0x7f6c64156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.190+0000 7f6c68e40640 1 -- 172.21.15.111:0/3845517267 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6415c3a0 con 0x7f6c640988b0 2024-09-13T17:58:38.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.190+0000 7f6c68e40640 1 --2- 172.21.15.111:0/3845517267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c640988b0 0x7f6c6409a390 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6c640b1630 tx=0x7f6c5c00a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:38.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.190+0000 7f6c60ff9640 1 -- 172.21.15.111:0/3845517267 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c00ad30 con 0x7f6c640988b0 2024-09-13T17:58:38.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.190+0000 7f6c60ff9640 1 -- 172.21.15.111:0/3845517267 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5c0133f0 con 0x7f6c640988b0 2024-09-13T17:58:38.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.190+0000 7f6c6a8a3640 1 -- 172.21.15.111:0/3845517267 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6415c680 con 0x7f6c640988b0 2024-09-13T17:58:38.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.190+0000 7f6c60ff9640 1 -- 172.21.15.111:0/3845517267 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c0137a0 con 0x7f6c640988b0 2024-09-13T17:58:38.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.190+0000 7f6c6a8a3640 1 -- 172.21.15.111:0/3845517267 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6415cc40 con 0x7f6c640988b0 2024-09-13T17:58:38.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.191+0000 7f6c6a8a3640 1 -- 172.21.15.111:0/3845517267 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6415d160 con 0x7f6c640988b0 2024-09-13T17:58:38.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.192+0000 7f6c60ff9640 1 -- 172.21.15.111:0/3845517267 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6c5c013900 con 0x7f6c640988b0 2024-09-13T17:58:38.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.192+0000 7f6c60ff9640 1 --2- 172.21.15.111:0/3845517267 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6c4406ac30 0x7f6c4406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.192+0000 7f6c63fff640 1 --2- 172.21.15.111:0/3845517267 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6c4406ac30 0x7f6c4406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.192+0000 7f6c60ff9640 1 -- 172.21.15.111:0/3845517267 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f6c5c089820 con 0x7f6c640988b0 2024-09-13T17:58:38.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.193+0000 7f6c63fff640 1 --2- 172.21.15.111:0/3845517267 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6c4406ac30 0x7f6c4406d0f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6c54000c00 tx=0x7f6c54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:38.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.196+0000 7f6c60ff9640 1 -- 172.21.15.111:0/3845517267 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f6c6415d160 con 0x7f6c640988b0 2024-09-13T17:58:38.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.418+0000 7f6c6a8a3640 1 -- 172.21.15.111:0/3845517267 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6c6415d160 con 0x7f6c640988b0 2024-09-13T17:58:38.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.418+0000 7f6c60ff9640 1 -- 172.21.15.111:0/3845517267 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f6c6415d160 con 0x7f6c640988b0 2024-09-13T17:58:38.420 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:38.420 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T17:58:38.421 INFO:teuthology.orchestra.run.smithi111.stdout: "auth_dump": [ 2024-09-13T17:58:38.421 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.421 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.a", 2024-09-13T17:58:38.421 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDMfORmGhsAKhAACDZXT4HDQrOTTulTPDDwBQ==", 2024-09-13T17:58:38.421 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.421 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2024-09-13T17:58:38.421 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2024-09-13T17:58:38.421 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2024-09-13T17:58:38.421 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-09-13T17:58:38.421 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.421 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.421 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.422 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.b", 2024-09-13T17:58:38.422 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDMfORmlVqTNRAA/v5yZiPCYiORGmfwU5DqmQ==", 2024-09-13T17:58:38.422 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.422 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2024-09-13T17:58:38.422 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2024-09-13T17:58:38.422 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2024-09-13T17:58:38.422 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-09-13T17:58:38.422 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.422 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.422 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.422 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.c", 2024-09-13T17:58:38.422 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDMfORmab3yMBAAVENS+Dd1iEHaLhSJs0+dgQ==", 2024-09-13T17:58:38.422 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.422 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2024-09-13T17:58:38.423 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2024-09-13T17:58:38.423 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2024-09-13T17:58:38.423 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-09-13T17:58:38.423 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.423 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.423 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.423 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.d", 2024-09-13T17:58:38.423 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDNfORmP/oEARAABOBxvI5yMvBLW27cxx/dTg==", 2024-09-13T17:58:38.423 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.423 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2024-09-13T17:58:38.423 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2024-09-13T17:58:38.423 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2024-09-13T17:58:38.423 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-09-13T17:58:38.423 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.423 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.424 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.424 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.0", 2024-09-13T17:58:38.424 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDQfORm2zZEIRAAZVyIO+oraTl7fReOjRPFIg==", 2024-09-13T17:58:38.424 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.424 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2024-09-13T17:58:38.424 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2024-09-13T17:58:38.424 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-09-13T17:58:38.424 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.424 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.424 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.424 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.1", 2024-09-13T17:58:38.424 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDTfORm2Cw+HxAAEblep5UnrnIjGE0ne/uhbA==", 2024-09-13T17:58:38.425 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.425 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2024-09-13T17:58:38.425 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2024-09-13T17:58:38.425 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-09-13T17:58:38.425 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.425 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.425 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.425 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.2", 2024-09-13T17:58:38.425 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDWfORm60erHRAAznKL8gIlxUuPRJs71ibQgQ==", 2024-09-13T17:58:38.425 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.425 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2024-09-13T17:58:38.425 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2024-09-13T17:58:38.425 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-09-13T17:58:38.426 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.426 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.426 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.426 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.3", 2024-09-13T17:58:38.426 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDZfORmXg3iHRAA1CDnJjVEw3IHgQbwO4X9kw==", 2024-09-13T17:58:38.426 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.426 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2024-09-13T17:58:38.426 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2024-09-13T17:58:38.426 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-09-13T17:58:38.426 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.426 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.426 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.426 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.4", 2024-09-13T17:58:38.426 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDffORm7eaQIhAAyZXNz0WPphUV7UgUgXEQVA==", 2024-09-13T17:58:38.427 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.427 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2024-09-13T17:58:38.427 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2024-09-13T17:58:38.427 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-09-13T17:58:38.427 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.427 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.427 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.427 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.5", 2024-09-13T17:58:38.427 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDifORmT4/lIBAAtD/epRe+TthTciIrVS1QPw==", 2024-09-13T17:58:38.427 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.428 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2024-09-13T17:58:38.428 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2024-09-13T17:58:38.428 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-09-13T17:58:38.428 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.428 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.428 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.428 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.6", 2024-09-13T17:58:38.428 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDlfORmhdihHhAAiskdva4E09iEu5trygpobQ==", 2024-09-13T17:58:38.428 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.428 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2024-09-13T17:58:38.428 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2024-09-13T17:58:38.428 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-09-13T17:58:38.428 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.428 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.428 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.429 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.7", 2024-09-13T17:58:38.429 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDofORm4CluHBAAQRzgHsJ2GEeReXRun+slyg==", 2024-09-13T17:58:38.429 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.429 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2024-09-13T17:58:38.429 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2024-09-13T17:58:38.429 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-09-13T17:58:38.429 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.429 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.429 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.429 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.0", 2024-09-13T17:58:38.429 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDNfORmzMgJBRAAEOeFOXxEvbtjqaxzRhfWtw==", 2024-09-13T17:58:38.429 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.430 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2024-09-13T17:58:38.430 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2024-09-13T17:58:38.430 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2024-09-13T17:58:38.430 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2024-09-13T17:58:38.430 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.430 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.430 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.430 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.1", 2024-09-13T17:58:38.430 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDNfORm1XzaDRAAyfCz1uhL9jnJ8WlZGbaspQ==", 2024-09-13T17:58:38.430 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.430 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2024-09-13T17:58:38.430 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2024-09-13T17:58:38.430 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2024-09-13T17:58:38.430 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2024-09-13T17:58:38.431 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.431 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.431 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.431 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.2", 2024-09-13T17:58:38.431 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDNfORmx5GlChAAH81QLf1bQUV2pIXYoRAwJg==", 2024-09-13T17:58:38.431 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.431 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2024-09-13T17:58:38.431 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2024-09-13T17:58:38.431 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2024-09-13T17:58:38.431 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2024-09-13T17:58:38.431 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.431 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.432 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.432 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.3", 2024-09-13T17:58:38.432 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDNfORmm7hpExAAQtZIpkmyDzo/cqH6N+nhZQ==", 2024-09-13T17:58:38.432 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.432 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2024-09-13T17:58:38.432 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2024-09-13T17:58:38.432 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2024-09-13T17:58:38.432 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2024-09-13T17:58:38.432 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.432 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.432 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.432 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.admin", 2024-09-13T17:58:38.432 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDMfORmHNSjDRAASFdMwwFTL71Ra2Hr+RNkzQ==", 2024-09-13T17:58:38.433 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.433 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow *", 2024-09-13T17:58:38.433 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2024-09-13T17:58:38.433 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2024-09-13T17:58:38.433 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-09-13T17:58:38.433 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.433 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.433 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.433 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-mds", 2024-09-13T17:58:38.433 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDufORm5S/7DxAAlM1V7kXKDXO5OSBQEZIJlw==", 2024-09-13T17:58:38.433 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.433 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-mds" 2024-09-13T17:58:38.433 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.433 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.434 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.434 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-mgr", 2024-09-13T17:58:38.434 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDufORm+T37DxAAkVJPaaxkzXsMeg4QW0IBww==", 2024-09-13T17:58:38.434 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.434 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-13T17:58:38.434 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.434 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.434 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.434 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-osd", 2024-09-13T17:58:38.434 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDufORmN0r7DxAAjzqzV4NNm68/WUSRjxnRPA==", 2024-09-13T17:58:38.434 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.434 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-osd" 2024-09-13T17:58:38.434 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.434 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.435 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.435 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-rbd", 2024-09-13T17:58:38.435 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDufORmBpT7DxAA038L6WhmN/905wk9mPmoMg==", 2024-09-13T17:58:38.435 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.435 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-13T17:58:38.435 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.435 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.435 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.435 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-13T17:58:38.435 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDufORmIp37DxAA6WpkUMARb7vUfX1BTbEaCg==", 2024-09-13T17:58:38.435 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.435 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-13T17:58:38.435 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.436 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.436 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.436 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-rgw", 2024-09-13T17:58:38.436 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDufORm6qP7DxAAd0kfSDARmf4vCaFkgl+OMA==", 2024-09-13T17:58:38.436 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.436 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-13T17:58:38.436 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.436 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.436 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.436 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mgr.x", 2024-09-13T17:58:38.436 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDMfORmHLHQJhAAVOix4lyIfaxaQb/3LUO5Kw==", 2024-09-13T17:58:38.436 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.436 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow *", 2024-09-13T17:58:38.436 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile mgr", 2024-09-13T17:58:38.437 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-09-13T17:58:38.437 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.437 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:38.437 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:38.437 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mgr.y", 2024-09-13T17:58:38.437 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDMfORmMfQwIhAAnGHtvQPIerUgf+C7YMXBrA==", 2024-09-13T17:58:38.437 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-09-13T17:58:38.437 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow *", 2024-09-13T17:58:38.437 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile mgr", 2024-09-13T17:58:38.437 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-09-13T17:58:38.437 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.437 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:38.437 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-13T17:58:38.438 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T17:58:38.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.420+0000 7f6c6a8a3640 1 -- 172.21.15.111:0/3845517267 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6c4406ac30 msgr2=0x7f6c4406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.420+0000 7f6c6a8a3640 1 --2- 172.21.15.111:0/3845517267 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6c4406ac30 0x7f6c4406d0f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6c54000c00 tx=0x7f6c54009040 comp rx=0 tx=0).stop 2024-09-13T17:58:38.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.420+0000 7f6c6a8a3640 1 -- 172.21.15.111:0/3845517267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c640988b0 msgr2=0x7f6c6409a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.420+0000 7f6c6a8a3640 1 --2- 172.21.15.111:0/3845517267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c640988b0 0x7f6c6409a390 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6c640b1630 tx=0x7f6c5c00a730 comp rx=0 tx=0).stop 2024-09-13T17:58:38.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.420+0000 7f6c6a8a3640 1 -- 172.21.15.111:0/3845517267 shutdown_connections 2024-09-13T17:58:38.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.420+0000 7f6c6a8a3640 1 --2- 172.21.15.111:0/3845517267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c640988b0 0x7f6c6409a390 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.420+0000 7f6c6a8a3640 1 --2- 172.21.15.111:0/3845517267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c640c7df0 0x7f6c64098370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.420+0000 7f6c6a8a3640 1 --2- 172.21.15.111:0/3845517267 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6c4406ac30 0x7f6c4406d0f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.420+0000 7f6c6a8a3640 1 --2- 172.21.15.111:0/3845517267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c50004350 0x7f6c64156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.420+0000 7f6c6a8a3640 1 -- 172.21.15.111:0/3845517267 >> 172.21.15.111:0/3845517267 conn(0x7f6c64069ed0 msgr2=0x7f6c640ad150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:38.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.420+0000 7f6c6a8a3640 1 -- 172.21.15.111:0/3845517267 shutdown_connections 2024-09-13T17:58:38.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.420+0000 7f6c6a8a3640 1 -- 172.21.15.111:0/3845517267 wait complete. 2024-09-13T17:58:38.440 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-13T17:58:38.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.537+0000 7f0d5b0e9640 1 Processor -- start 2024-09-13T17:58:38.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.537+0000 7f0d5b0e9640 1 -- start start 2024-09-13T17:58:38.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.538+0000 7f0d5b0e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d54159310 0x7f0d540ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.538+0000 7f0d5b0e9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d540ccbe0 con 0x7f0d540ce4f0 2024-09-13T17:58:38.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.538+0000 7f0d5b0e9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d540b1810 con 0x7f0d540b2b40 2024-09-13T17:58:38.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.538+0000 7f0d5b0e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d540ad940 con 0x7f0d54159310 2024-09-13T17:58:38.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.538+0000 7f0d59686640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f0d540ce4f0 0x7f0d540b1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48718/0 (socket says 172.21.15.111:48718) 2024-09-13T17:58:38.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.538+0000 7f0d59686640 1 -- 172.21.15.111:0/2611206190 learned_addr learned my addr 172.21.15.111:0/2611206190 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:38.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.538+0000 7f0d4b7fe640 1 -- 172.21.15.111:0/2611206190 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009681827 0 0) 0x7f0d540ccbe0 con 0x7f0d540ce4f0 2024-09-13T17:58:38.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4b7fe640 1 -- 172.21.15.111:0/2611206190 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d3c003520 con 0x7f0d540ce4f0 2024-09-13T17:58:38.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d58e85640 1 --2- 172.21.15.111:0/2611206190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d54159310 0x7f0d540ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4b7fe640 1 -- 172.21.15.111:0/2611206190 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1743934436 0 0) 0x7f0d3c003520 con 0x7f0d540ce4f0 2024-09-13T17:58:38.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4b7fe640 1 -- 172.21.15.111:0/2611206190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d54159310 msgr2=0x7f0d540ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:38.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4b7fe640 1 --2- 172.21.15.111:0/2611206190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d54159310 0x7f0d540ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4b7fe640 1 -- 172.21.15.111:0/2611206190 >> v1:172.21.15.188:6789/0 conn(0x7f0d540b2b40 legacy=0x7f0d540ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4b7fe640 1 -- 172.21.15.111:0/2611206190 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d5406d9e0 con 0x7f0d540ce4f0 2024-09-13T17:58:38.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4b7fe640 1 -- 172.21.15.111:0/2611206190 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f0d4c0019f0 con 0x7f0d540ce4f0 2024-09-13T17:58:38.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4b7fe640 1 -- 172.21.15.111:0/2611206190 >> v1:172.21.15.111:6789/0 conn(0x7f0d540ce4f0 legacy=0x7f0d540b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4b7fe640 1 --2- 172.21.15.111:0/2611206190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d54159310 0x7f0d3c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4b7fe640 1 --2- 172.21.15.111:0/2611206190 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c003f20 0x7f0d3c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4b7fe640 1 --2- 172.21.15.111:0/2611206190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d3c004d00 0x7f0d3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4b7fe640 1 -- 172.21.15.111:0/2611206190 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c00bb10 con 0x7f0d3c003f20 2024-09-13T17:58:38.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4b7fe640 1 -- 172.21.15.111:0/2611206190 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c00bc50 con 0x7f0d3c004d00 2024-09-13T17:58:38.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4b7fe640 1 -- 172.21.15.111:0/2611206190 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c00be10 con 0x7f0d54159310 2024-09-13T17:58:38.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4bfff640 1 --2- 172.21.15.111:0/2611206190 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c003f20 0x7f0d3c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d58e85640 1 --2- 172.21.15.111:0/2611206190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d54159310 0x7f0d3c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d59686640 1 --2- 172.21.15.111:0/2611206190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d3c004d00 0x7f0d3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4bfff640 1 -- 172.21.15.111:0/2611206190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d54159310 msgr2=0x7f0d3c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:38.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4bfff640 1 --2- 172.21.15.111:0/2611206190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d54159310 0x7f0d3c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4bfff640 1 -- 172.21.15.111:0/2611206190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d3c004d00 msgr2=0x7f0d3c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:38.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4bfff640 1 --2- 172.21.15.111:0/2611206190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d3c004d00 0x7f0d3c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.539+0000 7f0d4bfff640 1 -- 172.21.15.111:0/2611206190 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3c003520 con 0x7f0d3c003f20 2024-09-13T17:58:38.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.540+0000 7f0d4bfff640 1 --2- 172.21.15.111:0/2611206190 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c003f20 0x7f0d3c0047c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0d44001750 tx=0x7f0d3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df6514a14cdb3f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:38.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.540+0000 7f0d4b7fe640 1 -- 172.21.15.111:0/2611206190 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d440052d0 con 0x7f0d3c003f20 2024-09-13T17:58:38.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.540+0000 7f0d4b7fe640 1 -- 172.21.15.111:0/2611206190 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d44005430 con 0x7f0d3c003f20 2024-09-13T17:58:38.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.540+0000 7f0d4b7fe640 1 -- 172.21.15.111:0/2611206190 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4402abe0 con 0x7f0d3c003f20 2024-09-13T17:58:38.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.541+0000 7f0d5b0e9640 1 -- 172.21.15.111:0/2611206190 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c003f20 msgr2=0x7f0d3c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.541+0000 7f0d5b0e9640 1 --2- 172.21.15.111:0/2611206190 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c003f20 0x7f0d3c0047c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0d44001750 tx=0x7f0d3c0057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.541+0000 7f0d59686640 1 -- 172.21.15.111:0/2611206190 reap_dead start 2024-09-13T17:58:38.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.541+0000 7f0d5b0e9640 1 -- 172.21.15.111:0/2611206190 shutdown_connections 2024-09-13T17:58:38.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.541+0000 7f0d5b0e9640 1 -- 172.21.15.111:0/2611206190 >> 172.21.15.111:0/2611206190 conn(0x7f0d5406a0b0 msgr2=0x7f0d5406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:38.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.541+0000 7f0d5b0e9640 1 -- 172.21.15.111:0/2611206190 shutdown_connections 2024-09-13T17:58:38.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.541+0000 7f0d5b0e9640 1 -- 172.21.15.111:0/2611206190 wait complete. 2024-09-13T17:58:38.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.542+0000 7f0d5b0e9640 1 Processor -- start 2024-09-13T17:58:38.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.542+0000 7f0d5b0e9640 1 -- start start 2024-09-13T17:58:38.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d5b0e9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c003f20 0x7f0d54154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d5b0e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d540c54c0 0x7f0d541ef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d5b0e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d54071210 0x7f0d54071690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d5b0e9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d54071bd0 con 0x7f0d3c003f20 2024-09-13T17:58:38.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d5b0e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d54155a30 con 0x7f0d540c54c0 2024-09-13T17:58:38.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d5b0e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d54154690 con 0x7f0d54071210 2024-09-13T17:58:38.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d58e85640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c003f20 0x7f0d54154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d58e85640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c003f20 0x7f0d54154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59048/0 (socket says 172.21.15.111:59048) 2024-09-13T17:58:38.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d58e85640 1 -- 172.21.15.111:0/2530885327 learned_addr learned my addr 172.21.15.111:0/2530885327 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:38.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d4bfff640 1 --2- 172.21.15.111:0/2530885327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d540c54c0 0x7f0d541ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d59686640 1 --2- 172.21.15.111:0/2530885327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d54071210 0x7f0d54071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d58e85640 1 -- 172.21.15.111:0/2530885327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d54071210 msgr2=0x7f0d54071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d58e85640 1 --2- 172.21.15.111:0/2530885327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d54071210 0x7f0d54071690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d58e85640 1 -- 172.21.15.111:0/2530885327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d540c54c0 msgr2=0x7f0d541ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d58e85640 1 --2- 172.21.15.111:0/2530885327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d540c54c0 0x7f0d541ef9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d58e85640 1 -- 172.21.15.111:0/2530885327 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d54154970 con 0x7f0d3c003f20 2024-09-13T17:58:38.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d4bfff640 1 --2- 172.21.15.111:0/2530885327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d540c54c0 0x7f0d541ef9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:38.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d59686640 1 --2- 172.21.15.111:0/2530885327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d54071210 0x7f0d54071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:38.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d58e85640 1 --2- 172.21.15.111:0/2530885327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c003f20 0x7f0d54154020 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0d500119a0 tx=0x7f0d50011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:38.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d497fa640 1 -- 172.21.15.111:0/2530885327 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5000faf0 con 0x7f0d3c003f20 2024-09-13T17:58:38.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d497fa640 1 -- 172.21.15.111:0/2530885327 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5000fc50 con 0x7f0d3c003f20 2024-09-13T17:58:38.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d5b0e9640 1 -- 172.21.15.111:0/2530885327 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d54154c50 con 0x7f0d3c003f20 2024-09-13T17:58:38.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.543+0000 7f0d497fa640 1 -- 172.21.15.111:0/2530885327 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50014440 con 0x7f0d3c003f20 2024-09-13T17:58:38.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.544+0000 7f0d5b0e9640 1 -- 172.21.15.111:0/2530885327 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d540c5940 con 0x7f0d3c003f20 2024-09-13T17:58:38.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.544+0000 7f0d5b0e9640 1 -- 172.21.15.111:0/2530885327 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d540cd010 con 0x7f0d3c003f20 2024-09-13T17:58:38.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.545+0000 7f0d497fa640 1 -- 172.21.15.111:0/2530885327 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0d500145a0 con 0x7f0d3c003f20 2024-09-13T17:58:38.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.545+0000 7f0d497fa640 1 --2- 172.21.15.111:0/2530885327 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0d4006ac30 0x7f0d4006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.545+0000 7f0d497fa640 1 -- 172.21.15.111:0/2530885327 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f0d50089990 con 0x7f0d3c003f20 2024-09-13T17:58:38.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.545+0000 7f0d4bfff640 1 --2- 172.21.15.111:0/2530885327 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0d4006ac30 0x7f0d4006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.546+0000 7f0d4bfff640 1 --2- 172.21.15.111:0/2530885327 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0d4006ac30 0x7f0d4006d0f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0d44001750 tx=0x7f0d4402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:38.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.549+0000 7f0d497fa640 1 -- 172.21.15.111:0/2530885327 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f0d50054e10 con 0x7f0d3c003f20 2024-09-13T17:58:38.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.773+0000 7f0d5b0e9640 1 -- 172.21.15.111:0/2530885327 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f0d54066f60 con 0x7f0d3c003f20 2024-09-13T17:58:38.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.777+0000 7f0d497fa640 1 -- 172.21.15.111:0/2530885327 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0d50058430 con 0x7f0d3c003f20 2024-09-13T17:58:38.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.779+0000 7f0d5b0e9640 1 -- 172.21.15.111:0/2530885327 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0d4006ac30 msgr2=0x7f0d4006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.779+0000 7f0d5b0e9640 1 --2- 172.21.15.111:0/2530885327 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0d4006ac30 0x7f0d4006d0f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0d44001750 tx=0x7f0d4402e000 comp rx=0 tx=0).stop 2024-09-13T17:58:38.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.779+0000 7f0d5b0e9640 1 -- 172.21.15.111:0/2530885327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c003f20 msgr2=0x7f0d54154020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.779+0000 7f0d5b0e9640 1 --2- 172.21.15.111:0/2530885327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c003f20 0x7f0d54154020 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0d500119a0 tx=0x7f0d50011e60 comp rx=0 tx=0).stop 2024-09-13T17:58:38.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.779+0000 7f0d5b0e9640 1 -- 172.21.15.111:0/2530885327 shutdown_connections 2024-09-13T17:58:38.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.779+0000 7f0d5b0e9640 1 --2- 172.21.15.111:0/2530885327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d54071210 0x7f0d54071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.779+0000 7f0d5b0e9640 1 --2- 172.21.15.111:0/2530885327 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0d4006ac30 0x7f0d4006d0f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.779+0000 7f0d5b0e9640 1 --2- 172.21.15.111:0/2530885327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d540c54c0 0x7f0d541ef9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.779+0000 7f0d5b0e9640 1 --2- 172.21.15.111:0/2530885327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c003f20 0x7f0d54154020 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.779+0000 7f0d5b0e9640 1 -- 172.21.15.111:0/2530885327 >> 172.21.15.111:0/2530885327 conn(0x7f0d5406a0b0 msgr2=0x7f0d54158290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:38.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.779+0000 7f0d5b0e9640 1 -- 172.21.15.111:0/2530885327 shutdown_connections 2024-09-13T17:58:38.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.779+0000 7f0d5b0e9640 1 -- 172.21.15.111:0/2530885327 wait complete. 2024-09-13T17:58:38.791 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-13T17:58:38.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.888+0000 7ff465326640 1 Processor -- start 2024-09-13T17:58:38.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.888+0000 7ff465326640 1 -- start start 2024-09-13T17:58:38.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff465326640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff4600b1370 0x7ff4600b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff465326640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4600d1f40 con 0x7ff4600d1650 2024-09-13T17:58:38.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff465326640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4600d2190 con 0x7ff4600cd500 2024-09-13T17:58:38.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff465326640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4600ccb20 con 0x7ff4600b1370 2024-09-13T17:58:38.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45f7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff4600d1650 0x7ff4600b1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48724/0 (socket says 172.21.15.111:48724) 2024-09-13T17:58:38.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45f7fe640 1 -- 172.21.15.111:0/1398711574 learned_addr learned my addr 172.21.15.111:0/1398711574 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:38.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff4567fc640 1 --2- 172.21.15.111:0/1398711574 >> v2:172.21.15.188:3300/0 conn(0x7ff4600b1370 0x7ff4600b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45e7fc640 1 -- 172.21.15.111:0/1398711574 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398909523 0 0) 0x7ff4600d1f40 con 0x7ff4600d1650 2024-09-13T17:58:38.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45e7fc640 1 -- 172.21.15.111:0/1398711574 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff44c003520 con 0x7ff4600d1650 2024-09-13T17:58:38.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45e7fc640 1 -- 172.21.15.111:0/1398711574 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232398552 0 0) 0x7ff4600d2190 con 0x7ff4600cd500 2024-09-13T17:58:38.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45e7fc640 1 -- 172.21.15.111:0/1398711574 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4600d1f40 con 0x7ff4600cd500 2024-09-13T17:58:38.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45e7fc640 1 -- 172.21.15.111:0/1398711574 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3378475397 0 0) 0x7ff44c003520 con 0x7ff4600d1650 2024-09-13T17:58:38.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45e7fc640 1 -- 172.21.15.111:0/1398711574 >> v2:172.21.15.188:3300/0 conn(0x7ff4600b1370 msgr2=0x7ff4600b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:38.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45e7fc640 1 --2- 172.21.15.111:0/1398711574 >> v2:172.21.15.188:3300/0 conn(0x7ff4600b1370 0x7ff4600b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45e7fc640 1 -- 172.21.15.111:0/1398711574 >> v1:172.21.15.188:6789/0 conn(0x7ff4600cd500 legacy=0x7ff4600b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45e7fc640 1 -- 172.21.15.111:0/1398711574 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4600cce00 con 0x7ff4600d1650 2024-09-13T17:58:38.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45e7fc640 1 -- 172.21.15.111:0/1398711574 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7ff4580019f0 con 0x7ff4600d1650 2024-09-13T17:58:38.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45e7fc640 1 -- 172.21.15.111:0/1398711574 >> v1:172.21.15.111:6789/0 conn(0x7ff4600d1650 legacy=0x7ff4600b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45e7fc640 1 --2- 172.21.15.111:0/1398711574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff44c003b50 0x7ff44c003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45e7fc640 1 --2- 172.21.15.111:0/1398711574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff44c0047c0 0x7ff44c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45e7fc640 1 --2- 172.21.15.111:0/1398711574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff44c005180 0x7ff44c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45e7fc640 1 -- 172.21.15.111:0/1398711574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600ccb20 con 0x7ff44c005180 2024-09-13T17:58:38.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45e7fc640 1 -- 172.21.15.111:0/1398711574 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c00c050 con 0x7ff44c003b50 2024-09-13T17:58:38.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.889+0000 7ff45e7fc640 1 -- 172.21.15.111:0/1398711574 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c00c210 con 0x7ff44c0047c0 2024-09-13T17:58:38.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.890+0000 7ff45f7fe640 1 --2- 172.21.15.111:0/1398711574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff44c005180 0x7ff44c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.890+0000 7ff45effd640 1 --2- 172.21.15.111:0/1398711574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff44c003b50 0x7ff44c003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.890+0000 7ff4567fc640 1 --2- 172.21.15.111:0/1398711574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff44c0047c0 0x7ff44c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.890+0000 7ff45f7fe640 1 -- 172.21.15.111:0/1398711574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff44c0047c0 msgr2=0x7ff44c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:38.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.890+0000 7ff45f7fe640 1 --2- 172.21.15.111:0/1398711574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff44c0047c0 0x7ff44c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.890+0000 7ff45f7fe640 1 -- 172.21.15.111:0/1398711574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff44c003b50 msgr2=0x7ff44c003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:38.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.890+0000 7ff45f7fe640 1 --2- 172.21.15.111:0/1398711574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff44c003b50 0x7ff44c003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.890+0000 7ff45f7fe640 1 -- 172.21.15.111:0/1398711574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c003520 con 0x7ff44c005180 2024-09-13T17:58:38.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.890+0000 7ff4567fc640 1 --2- 172.21.15.111:0/1398711574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff44c0047c0 0x7ff44c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:38.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.890+0000 7ff45f7fe640 1 -- 172.21.15.111:0/1398711574 reap_dead start 2024-09-13T17:58:38.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.890+0000 7ff45f7fe640 1 --2- 172.21.15.111:0/1398711574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff44c005180 0x7ff44c00b980 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff4600b1850 tx=0x7ff44c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1001fe87a609994 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:38.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.891+0000 7ff45e7fc640 1 -- 172.21.15.111:0/1398711574 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45802b8d0 con 0x7ff44c005180 2024-09-13T17:58:38.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.891+0000 7ff45e7fc640 1 -- 172.21.15.111:0/1398711574 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45802ba30 con 0x7ff44c005180 2024-09-13T17:58:38.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.891+0000 7ff45e7fc640 1 -- 172.21.15.111:0/1398711574 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45802bde0 con 0x7ff44c005180 2024-09-13T17:58:38.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.891+0000 7ff465326640 1 -- 172.21.15.111:0/1398711574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff44c005180 msgr2=0x7ff44c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.891+0000 7ff465326640 1 --2- 172.21.15.111:0/1398711574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff44c005180 0x7ff44c00b980 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff4600b1850 tx=0x7ff44c0057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.891+0000 7ff465326640 1 -- 172.21.15.111:0/1398711574 shutdown_connections 2024-09-13T17:58:38.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.891+0000 7ff465326640 1 --2- 172.21.15.111:0/1398711574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff44c005180 0x7ff44c00b980 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.891+0000 7ff465326640 1 -- 172.21.15.111:0/1398711574 >> 172.21.15.111:0/1398711574 conn(0x7ff460089f60 msgr2=0x7ff4600d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:38.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.891+0000 7ff465326640 1 -- 172.21.15.111:0/1398711574 shutdown_connections 2024-09-13T17:58:38.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.891+0000 7ff465326640 1 -- 172.21.15.111:0/1398711574 wait complete. 2024-09-13T17:58:38.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.892+0000 7ff465326640 1 Processor -- start 2024-09-13T17:58:38.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.893+0000 7ff465326640 1 -- start start 2024-09-13T17:58:38.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.893+0000 7ff465326640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff44c005180 0x7ff460152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.893+0000 7ff465326640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4600c5c60 0x7ff46006c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.893+0000 7ff465326640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff46006c600 0x7ff4600b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.893+0000 7ff465326640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46006ca80 con 0x7ff4600c5c60 2024-09-13T17:58:38.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.893+0000 7ff465326640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff460152b00 con 0x7ff46006c600 2024-09-13T17:58:38.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.893+0000 7ff465326640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600c82d0 con 0x7ff44c005180 2024-09-13T17:58:38.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.893+0000 7ff4567fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4600c5c60 0x7ff46006c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.893+0000 7ff4567fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4600c5c60 0x7ff46006c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59070/0 (socket says 172.21.15.111:59070) 2024-09-13T17:58:38.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.893+0000 7ff4567fc640 1 -- 172.21.15.111:0/2664930157 learned_addr learned my addr 172.21.15.111:0/2664930157 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:38.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.894+0000 7ff45effd640 1 --2- 172.21.15.111:0/2664930157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff44c005180 0x7ff460152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.894+0000 7ff45f7fe640 1 --2- 172.21.15.111:0/2664930157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff46006c600 0x7ff4600b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.894+0000 7ff4567fc640 1 -- 172.21.15.111:0/2664930157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff44c005180 msgr2=0x7ff460152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.894+0000 7ff4567fc640 1 --2- 172.21.15.111:0/2664930157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff44c005180 0x7ff460152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.894+0000 7ff4567fc640 1 -- 172.21.15.111:0/2664930157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff46006c600 msgr2=0x7ff4600b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:38.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.894+0000 7ff4567fc640 1 --2- 172.21.15.111:0/2664930157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff46006c600 0x7ff4600b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:38.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.894+0000 7ff4567fc640 1 -- 172.21.15.111:0/2664930157 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff460152d50 con 0x7ff4600c5c60 2024-09-13T17:58:38.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.894+0000 7ff45f7fe640 1 --2- 172.21.15.111:0/2664930157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff46006c600 0x7ff4600b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:38.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.894+0000 7ff4567fc640 1 --2- 172.21.15.111:0/2664930157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4600c5c60 0x7ff46006c0c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff448011570 tx=0x7ff448011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:38.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.894+0000 7ff457fff640 1 -- 172.21.15.111:0/2664930157 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff448015870 con 0x7ff4600c5c60 2024-09-13T17:58:38.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.894+0000 7ff457fff640 1 -- 172.21.15.111:0/2664930157 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4480159d0 con 0x7ff4600c5c60 2024-09-13T17:58:38.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.894+0000 7ff465326640 1 -- 172.21.15.111:0/2664930157 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff460153030 con 0x7ff4600c5c60 2024-09-13T17:58:38.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.894+0000 7ff457fff640 1 -- 172.21.15.111:0/2664930157 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff448015d40 con 0x7ff4600c5c60 2024-09-13T17:58:38.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.894+0000 7ff465326640 1 -- 172.21.15.111:0/2664930157 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4600c8800 con 0x7ff4600c5c60 2024-09-13T17:58:38.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.895+0000 7ff465326640 1 -- 172.21.15.111:0/2664930157 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4600d2190 con 0x7ff4600c5c60 2024-09-13T17:58:38.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.895+0000 7ff457fff640 1 -- 172.21.15.111:0/2664930157 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff44800f460 con 0x7ff4600c5c60 2024-09-13T17:58:38.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.896+0000 7ff457fff640 1 --2- 172.21.15.111:0/2664930157 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff44006ac30 0x7ff44006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:38.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.896+0000 7ff457fff640 1 -- 172.21.15.111:0/2664930157 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ff448089960 con 0x7ff4600c5c60 2024-09-13T17:58:38.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.896+0000 7ff45effd640 1 --2- 172.21.15.111:0/2664930157 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff44006ac30 0x7ff44006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:38.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.896+0000 7ff45effd640 1 --2- 172.21.15.111:0/2664930157 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff44006ac30 0x7ff44006d0f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff4500018c0 tx=0x7ff450009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:38.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:38.900+0000 7ff457fff640 1 -- 172.21.15.111:0/2664930157 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff448054cc0 con 0x7ff4600c5c60 2024-09-13T17:58:39.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.122+0000 7ff465326640 1 -- 172.21.15.111:0/2664930157 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7ff460160560 con 0x7ff4600c5c60 2024-09-13T17:58:39.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.127+0000 7ff457fff640 1 -- 172.21.15.111:0/2664930157 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff4480582e0 con 0x7ff4600c5c60 2024-09-13T17:58:39.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.129+0000 7ff465326640 1 -- 172.21.15.111:0/2664930157 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff44006ac30 msgr2=0x7ff44006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:39.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.129+0000 7ff465326640 1 --2- 172.21.15.111:0/2664930157 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff44006ac30 0x7ff44006d0f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff4500018c0 tx=0x7ff450009040 comp rx=0 tx=0).stop 2024-09-13T17:58:39.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.129+0000 7ff465326640 1 -- 172.21.15.111:0/2664930157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4600c5c60 msgr2=0x7ff46006c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:39.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.129+0000 7ff465326640 1 --2- 172.21.15.111:0/2664930157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4600c5c60 0x7ff46006c0c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff448011570 tx=0x7ff448011a30 comp rx=0 tx=0).stop 2024-09-13T17:58:39.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.129+0000 7ff465326640 1 -- 172.21.15.111:0/2664930157 shutdown_connections 2024-09-13T17:58:39.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.129+0000 7ff465326640 1 --2- 172.21.15.111:0/2664930157 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff44006ac30 0x7ff44006d0f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.129+0000 7ff465326640 1 --2- 172.21.15.111:0/2664930157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff46006c600 0x7ff4600b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.129+0000 7ff465326640 1 --2- 172.21.15.111:0/2664930157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4600c5c60 0x7ff46006c0c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.129+0000 7ff465326640 1 --2- 172.21.15.111:0/2664930157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff44c005180 0x7ff460152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.129+0000 7ff465326640 1 -- 172.21.15.111:0/2664930157 >> 172.21.15.111:0/2664930157 conn(0x7ff460089f60 msgr2=0x7ff4600d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:39.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.129+0000 7ff465326640 1 -- 172.21.15.111:0/2664930157 shutdown_connections 2024-09-13T17:58:39.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.129+0000 7ff465326640 1 -- 172.21.15.111:0/2664930157 wait complete. 2024-09-13T17:58:39.142 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-13T17:58:39.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.239+0000 7f62150f7640 1 Processor -- start 2024-09-13T17:58:39.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.239+0000 7f62150f7640 1 -- start start 2024-09-13T17:58:39.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.240+0000 7f62150f7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62100b2b40 0x7f62100ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.240+0000 7f62150f7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f62100ce4f0 0x7f62100ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.240+0000 7f62150f7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62100ccbe0 con 0x7f6210159310 2024-09-13T17:58:39.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.240+0000 7f62150f7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62100ad940 con 0x7f62100ce4f0 2024-09-13T17:58:39.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.240+0000 7f62150f7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62100ceea0 con 0x7f62100b2b40 2024-09-13T17:58:39.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.240+0000 7f620f59e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f62100ce4f0 0x7f62100ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:39.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.240+0000 7f620f59e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f62100ce4f0 0x7f62100ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59080/0 (socket says 172.21.15.111:59080) 2024-09-13T17:58:39.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.240+0000 7f620f59e640 1 -- 172.21.15.111:0/276086684 learned_addr learned my addr 172.21.15.111:0/276086684 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:39.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.240+0000 7f620e59c640 1 --2- 172.21.15.111:0/276086684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62100b2b40 0x7f62100ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:39.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.240+0000 7f620f59e640 1 -- 172.21.15.111:0/276086684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62100b2b40 msgr2=0x7f62100ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:39.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.240+0000 7f620f59e640 1 --2- 172.21.15.111:0/276086684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62100b2b40 0x7f62100ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.240+0000 7f620f59e640 1 -- 172.21.15.111:0/276086684 >> v1:172.21.15.188:6789/0 conn(0x7f6210159310 legacy=0x7f62100ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:39.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.240+0000 7f620f59e640 1 -- 172.21.15.111:0/276086684 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62100b1810 con 0x7f62100ce4f0 2024-09-13T17:58:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.241+0000 7f620f59e640 1 --2- 172.21.15.111:0/276086684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62100ce4f0 0x7f62100ce960 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f620800d180 tx=0x7f620802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa71491e5d2d9158 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.241+0000 7f620dd9b640 1 -- 172.21.15.111:0/276086684 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6208009600 con 0x7f62100ce4f0 2024-09-13T17:58:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.241+0000 7f620dd9b640 1 -- 172.21.15.111:0/276086684 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620802f050 con 0x7f62100ce4f0 2024-09-13T17:58:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.241+0000 7f620dd9b640 1 -- 172.21.15.111:0/276086684 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6208009980 con 0x7f62100ce4f0 2024-09-13T17:58:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.242+0000 7f62150f7640 1 -- 172.21.15.111:0/276086684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62100ce4f0 msgr2=0x7f62100ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.242+0000 7f62150f7640 1 --2- 172.21.15.111:0/276086684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62100ce4f0 0x7f62100ce960 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f620800d180 tx=0x7f620802c7e0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.242+0000 7f62150f7640 1 -- 172.21.15.111:0/276086684 shutdown_connections 2024-09-13T17:58:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.242+0000 7f62150f7640 1 --2- 172.21.15.111:0/276086684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62100ce4f0 0x7f62100ce960 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.242+0000 7f62150f7640 1 --2- 172.21.15.111:0/276086684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62100b2b40 0x7f62100ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.242+0000 7f62150f7640 1 -- 172.21.15.111:0/276086684 >> 172.21.15.111:0/276086684 conn(0x7f621006a0b0 msgr2=0x7f621006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.242+0000 7f62150f7640 1 -- 172.21.15.111:0/276086684 shutdown_connections 2024-09-13T17:58:39.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.242+0000 7f62150f7640 1 -- 172.21.15.111:0/276086684 wait complete. 2024-09-13T17:58:39.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.243+0000 7f62150f7640 1 Processor -- start 2024-09-13T17:58:39.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.243+0000 7f62150f7640 1 -- start start 2024-09-13T17:58:39.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f62150f7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62100b2b40 0x7f62100c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f62150f7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62100ce4f0 0x7f6210154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f62150f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6210159310 0x7f62100addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f62150f7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62101e0970 con 0x7f6210159310 2024-09-13T17:58:39.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f62150f7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62100ae820 con 0x7f62100b2b40 2024-09-13T17:58:39.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f62150f7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62100aeb00 con 0x7f62100ce4f0 2024-09-13T17:58:39.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f620f59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6210159310 0x7f62100addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:39.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f620f59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6210159310 0x7f62100addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59092/0 (socket says 172.21.15.111:59092) 2024-09-13T17:58:39.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f620f59e640 1 -- 172.21.15.111:0/2757253894 learned_addr learned my addr 172.21.15.111:0/2757253894 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:39.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f620e59c640 1 --2- 172.21.15.111:0/2757253894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62100ce4f0 0x7f6210154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:39.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f620ed9d640 1 --2- 172.21.15.111:0/2757253894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62100b2b40 0x7f62100c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:39.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f620f59e640 1 -- 172.21.15.111:0/2757253894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62100ce4f0 msgr2=0x7f6210154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:39.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f620f59e640 1 --2- 172.21.15.111:0/2757253894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62100ce4f0 0x7f6210154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f620f59e640 1 -- 172.21.15.111:0/2757253894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62100b2b40 msgr2=0x7f62100c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:39.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f620f59e640 1 --2- 172.21.15.111:0/2757253894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62100b2b40 0x7f62100c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f620f59e640 1 -- 172.21.15.111:0/2757253894 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620800bd40 con 0x7f6210159310 2024-09-13T17:58:39.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f620ed9d640 1 --2- 172.21.15.111:0/2757253894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62100b2b40 0x7f62100c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:39.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.244+0000 7f620f59e640 1 --2- 172.21.15.111:0/2757253894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6210159310 0x7f62100addd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f620802cdd0 tx=0x7f6208004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:39.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.245+0000 7f61f77fe640 1 -- 172.21.15.111:0/2757253894 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6208003b90 con 0x7f6210159310 2024-09-13T17:58:39.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.245+0000 7f61f77fe640 1 -- 172.21.15.111:0/2757253894 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6208003cf0 con 0x7f6210159310 2024-09-13T17:58:39.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.245+0000 7f62150f7640 1 -- 172.21.15.111:0/2757253894 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62100aede0 con 0x7f6210159310 2024-09-13T17:58:39.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.245+0000 7f61f77fe640 1 -- 172.21.15.111:0/2757253894 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620803d840 con 0x7f6210159310 2024-09-13T17:58:39.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.245+0000 7f62150f7640 1 -- 172.21.15.111:0/2757253894 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62100afcc0 con 0x7f6210159310 2024-09-13T17:58:39.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.246+0000 7f62150f7640 1 -- 172.21.15.111:0/2757253894 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621009c4f0 con 0x7f6210159310 2024-09-13T17:58:39.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.246+0000 7f61f77fe640 1 -- 172.21.15.111:0/2757253894 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f620803a070 con 0x7f6210159310 2024-09-13T17:58:39.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.246+0000 7f61f77fe640 1 --2- 172.21.15.111:0/2757253894 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f61fc06ac00 0x7f61fc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.247+0000 7f61f77fe640 1 -- 172.21.15.111:0/2757253894 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f62080abd40 con 0x7f6210159310 2024-09-13T17:58:39.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.247+0000 7f620ed9d640 1 --2- 172.21.15.111:0/2757253894 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f61fc06ac00 0x7f61fc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:39.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.247+0000 7f620ed9d640 1 --2- 172.21.15.111:0/2757253894 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f61fc06ac00 0x7f61fc06d0c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f62100caf20 tx=0x7f6200008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:39.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.251+0000 7f61f77fe640 1 -- 172.21.15.111:0/2757253894 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f6208077140 con 0x7f6210159310 2024-09-13T17:58:39.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.474+0000 7f62150f7640 1 -- 172.21.15.111:0/2757253894 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f62100ab820 con 0x7f6210159310 2024-09-13T17:58:39.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.480+0000 7f61f77fe640 1 -- 172.21.15.111:0/2757253894 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f620807a760 con 0x7f6210159310 2024-09-13T17:58:39.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.482+0000 7f62150f7640 1 -- 172.21.15.111:0/2757253894 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f61fc06ac00 msgr2=0x7f61fc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:39.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.482+0000 7f62150f7640 1 --2- 172.21.15.111:0/2757253894 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f61fc06ac00 0x7f61fc06d0c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f62100caf20 tx=0x7f6200008040 comp rx=0 tx=0).stop 2024-09-13T17:58:39.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.482+0000 7f62150f7640 1 -- 172.21.15.111:0/2757253894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6210159310 msgr2=0x7f62100addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:39.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.482+0000 7f62150f7640 1 --2- 172.21.15.111:0/2757253894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6210159310 0x7f62100addd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f620802cdd0 tx=0x7f6208004a40 comp rx=0 tx=0).stop 2024-09-13T17:58:39.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.482+0000 7f62150f7640 1 -- 172.21.15.111:0/2757253894 shutdown_connections 2024-09-13T17:58:39.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.482+0000 7f62150f7640 1 --2- 172.21.15.111:0/2757253894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6210159310 0x7f62100addd0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.482+0000 7f62150f7640 1 --2- 172.21.15.111:0/2757253894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62100ce4f0 0x7f6210154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.482+0000 7f62150f7640 1 --2- 172.21.15.111:0/2757253894 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f61fc06ac00 0x7f61fc06d0c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.482+0000 7f62150f7640 1 --2- 172.21.15.111:0/2757253894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62100b2b40 0x7f62100c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.482+0000 7f62150f7640 1 -- 172.21.15.111:0/2757253894 >> 172.21.15.111:0/2757253894 conn(0x7f621006a0b0 msgr2=0x7f621006a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:39.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.482+0000 7f62150f7640 1 -- 172.21.15.111:0/2757253894 shutdown_connections 2024-09-13T17:58:39.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.482+0000 7f62150f7640 1 -- 172.21.15.111:0/2757253894 wait complete. 2024-09-13T17:58:39.495 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-13T17:58:39.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.591+0000 7f4ae2f65640 1 Processor -- start 2024-09-13T17:58:39.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.591+0000 7f4ae2f65640 1 -- start start 2024-09-13T17:58:39.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.592+0000 7f4ae2f65640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4adc159310 0x7f4adc0ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.592+0000 7f4ae2f65640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4adc0b2b40 0x7f4adc0ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.592+0000 7f4ae2f65640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4adc0ccbe0 con 0x7f4adc0ab680 2024-09-13T17:58:39.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.592+0000 7f4ae2f65640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4adc0ad940 con 0x7f4adc0b2b40 2024-09-13T17:58:39.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.592+0000 7f4ae2f65640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4adc0b1810 con 0x7f4adc159310 2024-09-13T17:58:39.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.592+0000 7f4ad3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4adc0b2b40 0x7f4adc0ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:39.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.592+0000 7f4ad3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4adc0b2b40 0x7f4adc0ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59094/0 (socket says 172.21.15.111:59094) 2024-09-13T17:58:39.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.592+0000 7f4ad3fff640 1 -- 172.21.15.111:0/1936885276 learned_addr learned my addr 172.21.15.111:0/1936885276 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:39.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.592+0000 7f4ad3fff640 1 -- 172.21.15.111:0/1936885276 >> v2:172.21.15.188:3300/0 conn(0x7f4adc159310 msgr2=0x7f4adc0ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:39.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.592+0000 7f4ad3fff640 1 --2- 172.21.15.111:0/1936885276 >> v2:172.21.15.188:3300/0 conn(0x7f4adc159310 0x7f4adc0ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.592+0000 7f4ad3fff640 1 -- 172.21.15.111:0/1936885276 >> v1:172.21.15.188:6789/0 conn(0x7f4adc0ab680 legacy=0x7f4adc0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:39.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.592+0000 7f4ad3fff640 1 -- 172.21.15.111:0/1936885276 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc06d9e0 con 0x7f4adc0b2b40 2024-09-13T17:58:39.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.593+0000 7f4ad3fff640 1 --2- 172.21.15.111:0/1936885276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4adc0b2b40 0x7f4adc0ab140 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0cac90 tx=0x7f4acc02c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85fdc2ff96593e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:39.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.593+0000 7f4ad37fe640 1 -- 172.21.15.111:0/1936885276 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc032710 con 0x7f4adc0b2b40 2024-09-13T17:58:39.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.593+0000 7f4ad37fe640 1 -- 172.21.15.111:0/1936885276 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4acc030050 con 0x7f4adc0b2b40 2024-09-13T17:58:39.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.593+0000 7f4ad37fe640 1 -- 172.21.15.111:0/1936885276 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc032ab0 con 0x7f4adc0b2b40 2024-09-13T17:58:39.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.593+0000 7f4ae2f65640 1 -- 172.21.15.111:0/1936885276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4adc0b2b40 msgr2=0x7f4adc0ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:39.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.593+0000 7f4ae2f65640 1 --2- 172.21.15.111:0/1936885276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4adc0b2b40 0x7f4adc0ab140 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0cac90 tx=0x7f4acc02c850 comp rx=0 tx=0).stop 2024-09-13T17:58:39.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.593+0000 7f4ae2f65640 1 -- 172.21.15.111:0/1936885276 shutdown_connections 2024-09-13T17:58:39.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.593+0000 7f4ae2f65640 1 --2- 172.21.15.111:0/1936885276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4adc0b2b40 0x7f4adc0ab140 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.593+0000 7f4ae2f65640 1 --2- 172.21.15.111:0/1936885276 >> v2:172.21.15.188:3300/0 conn(0x7f4adc159310 0x7f4adc0ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.593+0000 7f4ae2f65640 1 -- 172.21.15.111:0/1936885276 >> 172.21.15.111:0/1936885276 conn(0x7f4adc06a0b0 msgr2=0x7f4adc06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:39.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.593+0000 7f4ae2f65640 1 -- 172.21.15.111:0/1936885276 shutdown_connections 2024-09-13T17:58:39.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.593+0000 7f4ae2f65640 1 -- 172.21.15.111:0/1936885276 wait complete. 2024-09-13T17:58:39.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.595+0000 7f4ae2f65640 1 Processor -- start 2024-09-13T17:58:39.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.595+0000 7f4ae2f65640 1 -- start start 2024-09-13T17:58:39.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.595+0000 7f4ae2f65640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4adc0ab680 0x7f4adc0c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.595+0000 7f4ae2f65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4adc0b2b40 0x7f4adc150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.595+0000 7f4ae2f65640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4adc159310 0x7f4adc0c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.595+0000 7f4ae2f65640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc0c4910 con 0x7f4adc0ab680 2024-09-13T17:58:39.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.595+0000 7f4ae2f65640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc070590 con 0x7f4adc159310 2024-09-13T17:58:39.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.595+0000 7f4ae2f65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc070870 con 0x7f4adc0b2b40 2024-09-13T17:58:39.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.595+0000 7f4ae0d01640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4adc0ab680 0x7f4adc0c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:39.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.596+0000 7f4ae0d01640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4adc0ab680 0x7f4adc0c4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59102/0 (socket says 172.21.15.111:59102) 2024-09-13T17:58:39.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.596+0000 7f4ae0d01640 1 -- 172.21.15.111:0/3158395775 learned_addr learned my addr 172.21.15.111:0/3158395775 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:39.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.596+0000 7f4ad3fff640 1 --2- 172.21.15.111:0/3158395775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4adc0b2b40 0x7f4adc150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:39.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.596+0000 7f4ae1502640 1 --2- 172.21.15.111:0/3158395775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4adc159310 0x7f4adc0c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:39.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.596+0000 7f4ae0d01640 1 -- 172.21.15.111:0/3158395775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4adc0b2b40 msgr2=0x7f4adc150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:39.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.596+0000 7f4ae0d01640 1 --2- 172.21.15.111:0/3158395775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4adc0b2b40 0x7f4adc150ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.596+0000 7f4ae0d01640 1 -- 172.21.15.111:0/3158395775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4adc159310 msgr2=0x7f4adc0c43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:39.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.596+0000 7f4ae0d01640 1 --2- 172.21.15.111:0/3158395775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4adc159310 0x7f4adc0c43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.596+0000 7f4ae0d01640 1 -- 172.21.15.111:0/3158395775 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4acc00b770 con 0x7f4adc0ab680 2024-09-13T17:58:39.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.596+0000 7f4ae0d01640 1 --2- 172.21.15.111:0/3158395775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4adc0ab680 0x7f4adc0c4b40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800ba50 tx=0x7f4ad800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:39.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.596+0000 7f4ad17fa640 1 -- 172.21.15.111:0/3158395775 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad800cbe0 con 0x7f4adc0ab680 2024-09-13T17:58:39.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.596+0000 7f4ad17fa640 1 -- 172.21.15.111:0/3158395775 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad800cd60 con 0x7f4adc0ab680 2024-09-13T17:58:39.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.596+0000 7f4ae2f65640 1 -- 172.21.15.111:0/3158395775 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4adc070b50 con 0x7f4adc0ab680 2024-09-13T17:58:39.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.596+0000 7f4ad17fa640 1 -- 172.21.15.111:0/3158395775 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8014680 con 0x7f4adc0ab680 2024-09-13T17:58:39.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.596+0000 7f4ae2f65640 1 -- 172.21.15.111:0/3158395775 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4adc071110 con 0x7f4adc0ab680 2024-09-13T17:58:39.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.597+0000 7f4ae2f65640 1 -- 172.21.15.111:0/3158395775 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4adc0cebe0 con 0x7f4adc0ab680 2024-09-13T17:58:39.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.597+0000 7f4ad17fa640 1 -- 172.21.15.111:0/3158395775 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4ad80147e0 con 0x7f4adc0ab680 2024-09-13T17:58:39.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.598+0000 7f4ad17fa640 1 --2- 172.21.15.111:0/3158395775 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4ac806ac30 0x7f4ac806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.598+0000 7f4ad3fff640 1 --2- 172.21.15.111:0/3158395775 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4ac806ac30 0x7f4ac806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:39.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.598+0000 7f4ad17fa640 1 -- 172.21.15.111:0/3158395775 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4ad80890c0 con 0x7f4adc0ab680 2024-09-13T17:58:39.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.598+0000 7f4ad3fff640 1 --2- 172.21.15.111:0/3158395775 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4ac806ac30 0x7f4ac806d0f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4acc037000 tx=0x7f4acc032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:39.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.602+0000 7f4ad17fa640 1 -- 172.21.15.111:0/3158395775 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4adc0cebe0 con 0x7f4adc0ab680 2024-09-13T17:58:39.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.824+0000 7f4ae2f65640 1 -- 172.21.15.111:0/3158395775 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f4adc0cebe0 con 0x7f4adc0ab680 2024-09-13T17:58:39.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.830+0000 7f4ad17fa640 1 -- 172.21.15.111:0/3158395775 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4adc0cebe0 con 0x7f4adc0ab680 2024-09-13T17:58:39.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.832+0000 7f4ae2f65640 1 -- 172.21.15.111:0/3158395775 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4ac806ac30 msgr2=0x7f4ac806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:39.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.832+0000 7f4ae2f65640 1 --2- 172.21.15.111:0/3158395775 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4ac806ac30 0x7f4ac806d0f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4acc037000 tx=0x7f4acc032000 comp rx=0 tx=0).stop 2024-09-13T17:58:39.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.832+0000 7f4ae2f65640 1 -- 172.21.15.111:0/3158395775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4adc0ab680 msgr2=0x7f4adc0c4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:39.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.832+0000 7f4ae2f65640 1 --2- 172.21.15.111:0/3158395775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4adc0ab680 0x7f4adc0c4b40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800ba50 tx=0x7f4ad800bf10 comp rx=0 tx=0).stop 2024-09-13T17:58:39.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.832+0000 7f4ae2f65640 1 -- 172.21.15.111:0/3158395775 shutdown_connections 2024-09-13T17:58:39.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.832+0000 7f4ae2f65640 1 --2- 172.21.15.111:0/3158395775 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4ac806ac30 0x7f4ac806d0f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.832+0000 7f4ae2f65640 1 --2- 172.21.15.111:0/3158395775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4adc159310 0x7f4adc0c43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.832+0000 7f4ae2f65640 1 --2- 172.21.15.111:0/3158395775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4adc0b2b40 0x7f4adc150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.832+0000 7f4ae2f65640 1 --2- 172.21.15.111:0/3158395775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4adc0ab680 0x7f4adc0c4b40 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.832+0000 7f4ae2f65640 1 -- 172.21.15.111:0/3158395775 >> 172.21.15.111:0/3158395775 conn(0x7f4adc06a0b0 msgr2=0x7f4adc0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:39.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.832+0000 7f4ae2f65640 1 -- 172.21.15.111:0/3158395775 shutdown_connections 2024-09-13T17:58:39.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.832+0000 7f4ae2f65640 1 -- 172.21.15.111:0/3158395775 wait complete. 2024-09-13T17:58:39.845 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-13T17:58:39.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.942+0000 7fd527ee5640 1 Processor -- start 2024-09-13T17:58:39.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.942+0000 7fd527ee5640 1 -- start start 2024-09-13T17:58:39.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.943+0000 7fd527ee5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd520072110 0x7fd520070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.943+0000 7fd527ee5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd5200c8920 0x7fd5200c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.943+0000 7fd527ee5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5200abdf0 con 0x7fd520071d30 2024-09-13T17:58:39.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.943+0000 7fd527ee5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5200c92f0 con 0x7fd520072110 2024-09-13T17:58:39.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.943+0000 7fd527ee5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5200b2280 con 0x7fd5200c8920 2024-09-13T17:58:39.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.943+0000 7fd525480640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd520072110 0x7fd520070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:39.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.943+0000 7fd525480640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd520072110 0x7fd520070190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59114/0 (socket says 172.21.15.111:59114) 2024-09-13T17:58:39.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.943+0000 7fd525480640 1 -- 172.21.15.111:0/2442979323 learned_addr learned my addr 172.21.15.111:0/2442979323 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:39.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.943+0000 7fd526482640 1 --2- 172.21.15.111:0/2442979323 >> v2:172.21.15.188:3300/0 conn(0x7fd5200c8920 0x7fd5200c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:39.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.943+0000 7fd525480640 1 -- 172.21.15.111:0/2442979323 >> v2:172.21.15.188:3300/0 conn(0x7fd5200c8920 msgr2=0x7fd5200c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:39.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.943+0000 7fd525480640 1 --2- 172.21.15.111:0/2442979323 >> v2:172.21.15.188:3300/0 conn(0x7fd5200c8920 0x7fd5200c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.943+0000 7fd525480640 1 -- 172.21.15.111:0/2442979323 >> v1:172.21.15.188:6789/0 conn(0x7fd520071d30 legacy=0x7fd5200b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:39.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.943+0000 7fd525480640 1 -- 172.21.15.111:0/2442979323 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5200ab1e0 con 0x7fd520072110 2024-09-13T17:58:39.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.944+0000 7fd525480640 1 --2- 172.21.15.111:0/2442979323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd520072110 0x7fd520070190 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd52006ff50 tx=0x7fd51002c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3f3508766c1201 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:39.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.944+0000 7fd524c7f640 1 -- 172.21.15.111:0/2442979323 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510032640 con 0x7fd520072110 2024-09-13T17:58:39.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.944+0000 7fd524c7f640 1 -- 172.21.15.111:0/2442979323 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5100327a0 con 0x7fd520072110 2024-09-13T17:58:39.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.944+0000 7fd524c7f640 1 -- 172.21.15.111:0/2442979323 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510032b30 con 0x7fd520072110 2024-09-13T17:58:39.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.944+0000 7fd527ee5640 1 -- 172.21.15.111:0/2442979323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd520072110 msgr2=0x7fd520070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:39.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.944+0000 7fd527ee5640 1 --2- 172.21.15.111:0/2442979323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd520072110 0x7fd520070190 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd52006ff50 tx=0x7fd51002c940 comp rx=0 tx=0).stop 2024-09-13T17:58:39.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.944+0000 7fd527ee5640 1 -- 172.21.15.111:0/2442979323 shutdown_connections 2024-09-13T17:58:39.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.944+0000 7fd527ee5640 1 --2- 172.21.15.111:0/2442979323 >> v2:172.21.15.188:3300/0 conn(0x7fd5200c8920 0x7fd5200c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.944+0000 7fd527ee5640 1 --2- 172.21.15.111:0/2442979323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd520072110 0x7fd520070190 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.944+0000 7fd527ee5640 1 -- 172.21.15.111:0/2442979323 >> 172.21.15.111:0/2442979323 conn(0x7fd520069ed0 msgr2=0x7fd52006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:39.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.944+0000 7fd527ee5640 1 -- 172.21.15.111:0/2442979323 shutdown_connections 2024-09-13T17:58:39.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.945+0000 7fd527ee5640 1 -- 172.21.15.111:0/2442979323 wait complete. 2024-09-13T17:58:39.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.946+0000 7fd527ee5640 1 Processor -- start 2024-09-13T17:58:39.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.946+0000 7fd527ee5640 1 -- start start 2024-09-13T17:58:39.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.946+0000 7fd527ee5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd520071d30 0x7fd520097920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.946+0000 7fd527ee5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd520072110 0x7fd5201eddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.946+0000 7fd527ee5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5200c8920 0x7fd52006d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.946+0000 7fd527ee5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd520097e60 con 0x7fd520071d30 2024-09-13T17:58:39.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.946+0000 7fd527ee5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52006dba0 con 0x7fd5200c8920 2024-09-13T17:58:39.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.946+0000 7fd527ee5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200c6a30 con 0x7fd520072110 2024-09-13T17:58:39.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.946+0000 7fd525c81640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd520071d30 0x7fd520097920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:39.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.946+0000 7fd525c81640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd520071d30 0x7fd520097920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59124/0 (socket says 172.21.15.111:59124) 2024-09-13T17:58:39.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.947+0000 7fd525c81640 1 -- 172.21.15.111:0/4064638170 learned_addr learned my addr 172.21.15.111:0/4064638170 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:39.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.947+0000 7fd526482640 1 --2- 172.21.15.111:0/4064638170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5200c8920 0x7fd52006d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:39.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.947+0000 7fd525480640 1 --2- 172.21.15.111:0/4064638170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd520072110 0x7fd5201eddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:39.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.947+0000 7fd525c81640 1 -- 172.21.15.111:0/4064638170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd520072110 msgr2=0x7fd5201eddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:39.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.947+0000 7fd525c81640 1 --2- 172.21.15.111:0/4064638170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd520072110 0x7fd5201eddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.947+0000 7fd525c81640 1 -- 172.21.15.111:0/4064638170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5200c8920 msgr2=0x7fd52006d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:39.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.947+0000 7fd525c81640 1 --2- 172.21.15.111:0/4064638170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5200c8920 0x7fd52006d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:39.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.947+0000 7fd525c81640 1 -- 172.21.15.111:0/4064638170 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51000b770 con 0x7fd520071d30 2024-09-13T17:58:39.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.947+0000 7fd525480640 1 --2- 172.21.15.111:0/4064638170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd520072110 0x7fd5201eddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:39.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.947+0000 7fd525c81640 1 --2- 172.21.15.111:0/4064638170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd520071d30 0x7fd520097920 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c450 tx=0x7fd51c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:39.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.947+0000 7fd5167fc640 1 -- 172.21.15.111:0/4064638170 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0052d0 con 0x7fd520071d30 2024-09-13T17:58:39.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.947+0000 7fd5167fc640 1 -- 172.21.15.111:0/4064638170 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c005430 con 0x7fd520071d30 2024-09-13T17:58:39.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.947+0000 7fd527ee5640 1 -- 172.21.15.111:0/4064638170 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5200c6d10 con 0x7fd520071d30 2024-09-13T17:58:39.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.947+0000 7fd5167fc640 1 -- 172.21.15.111:0/4064638170 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c012870 con 0x7fd520071d30 2024-09-13T17:58:39.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.947+0000 7fd527ee5640 1 -- 172.21.15.111:0/4064638170 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5200c72d0 con 0x7fd520071d30 2024-09-13T17:58:39.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.948+0000 7fd527ee5640 1 -- 172.21.15.111:0/4064638170 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd520070190 con 0x7fd520071d30 2024-09-13T17:58:39.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.948+0000 7fd5167fc640 1 -- 172.21.15.111:0/4064638170 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd51c0129d0 con 0x7fd520071d30 2024-09-13T17:58:39.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.949+0000 7fd5167fc640 1 --2- 172.21.15.111:0/4064638170 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd50406ac00 0x7fd50406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:39.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.949+0000 7fd525480640 1 --2- 172.21.15.111:0/4064638170 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd50406ac00 0x7fd50406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:39.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.949+0000 7fd5167fc640 1 -- 172.21.15.111:0/4064638170 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd51c088bf0 con 0x7fd520071d30 2024-09-13T17:58:39.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.949+0000 7fd525480640 1 --2- 172.21.15.111:0/4064638170 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd50406ac00 0x7fd50406d0c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd510037000 tx=0x7fd51003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:39.955 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:39.953+0000 7fd5167fc640 1 -- 172.21.15.111:0/4064638170 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fd51c053fe0 con 0x7fd520071d30 2024-09-13T17:58:40.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.175+0000 7fd527ee5640 1 -- 172.21.15.111:0/4064638170 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fd52015cb20 con 0x7fd520071d30 2024-09-13T17:58:40.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.180+0000 7fd5167fc640 1 -- 172.21.15.111:0/4064638170 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd51c057600 con 0x7fd520071d30 2024-09-13T17:58:40.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.183+0000 7fd527ee5640 1 -- 172.21.15.111:0/4064638170 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd50406ac00 msgr2=0x7fd50406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.183+0000 7fd527ee5640 1 --2- 172.21.15.111:0/4064638170 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd50406ac00 0x7fd50406d0c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd510037000 tx=0x7fd51003d040 comp rx=0 tx=0).stop 2024-09-13T17:58:40.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.183+0000 7fd527ee5640 1 -- 172.21.15.111:0/4064638170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd520071d30 msgr2=0x7fd520097920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.183+0000 7fd527ee5640 1 --2- 172.21.15.111:0/4064638170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd520071d30 0x7fd520097920 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c450 tx=0x7fd51c00c910 comp rx=0 tx=0).stop 2024-09-13T17:58:40.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.183+0000 7fd527ee5640 1 -- 172.21.15.111:0/4064638170 shutdown_connections 2024-09-13T17:58:40.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.183+0000 7fd527ee5640 1 --2- 172.21.15.111:0/4064638170 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd50406ac00 0x7fd50406d0c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.183+0000 7fd527ee5640 1 --2- 172.21.15.111:0/4064638170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5200c8920 0x7fd52006d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.183+0000 7fd527ee5640 1 --2- 172.21.15.111:0/4064638170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd520072110 0x7fd5201eddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.183+0000 7fd527ee5640 1 --2- 172.21.15.111:0/4064638170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd520071d30 0x7fd520097920 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.183+0000 7fd527ee5640 1 -- 172.21.15.111:0/4064638170 >> 172.21.15.111:0/4064638170 conn(0x7fd520069ed0 msgr2=0x7fd5200aca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:40.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.183+0000 7fd527ee5640 1 -- 172.21.15.111:0/4064638170 shutdown_connections 2024-09-13T17:58:40.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.183+0000 7fd527ee5640 1 -- 172.21.15.111:0/4064638170 wait complete. 2024-09-13T17:58:40.196 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-13T17:58:40.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.292+0000 7fbc19d9c640 1 Processor -- start 2024-09-13T17:58:40.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.292+0000 7fbc19d9c640 1 -- start start 2024-09-13T17:58:40.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.293+0000 7fbc19d9c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbc14158b00 0x7fbc1406c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.293+0000 7fbc19d9c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbc1406c6d0 0x7fbc14070b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.293+0000 7fbc19d9c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc140b2d10 con 0x7fbc14071050 2024-09-13T17:58:40.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.293+0000 7fbc19d9c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc1406cb60 con 0x7fbc1406c6d0 2024-09-13T17:58:40.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.293+0000 7fbc19d9c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc140b2ff0 con 0x7fbc14158b00 2024-09-13T17:58:40.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.293+0000 7fbc13fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbc14071050 0x7fbc140714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48726/0 (socket says 172.21.15.111:48726) 2024-09-13T17:58:40.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.293+0000 7fbc13fff640 1 -- 172.21.15.111:0/1220556794 learned_addr learned my addr 172.21.15.111:0/1220556794 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:40.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.293+0000 7fbc12ffd640 1 --2- 172.21.15.111:0/1220556794 >> v2:172.21.15.111:3300/0 conn(0x7fbc1406c6d0 0x7fbc14070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.293+0000 7fbc127fc640 1 -- 172.21.15.111:0/1220556794 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001312613 0 0) 0x7fbc140b2d10 con 0x7fbc14071050 2024-09-13T17:58:40.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.293+0000 7fbc137fe640 1 --2- 172.21.15.111:0/1220556794 >> v2:172.21.15.188:3300/0 conn(0x7fbc14158b00 0x7fbc1406c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.293+0000 7fbc127fc640 1 -- 172.21.15.111:0/1220556794 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbfc003520 con 0x7fbc14071050 2024-09-13T17:58:40.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.293+0000 7fbc12ffd640 1 -- 172.21.15.111:0/1220556794 >> v2:172.21.15.188:3300/0 conn(0x7fbc14158b00 msgr2=0x7fbc1406c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:40.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.293+0000 7fbc12ffd640 1 --2- 172.21.15.111:0/1220556794 >> v2:172.21.15.188:3300/0 conn(0x7fbc14158b00 0x7fbc1406c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.293+0000 7fbc12ffd640 1 -- 172.21.15.111:0/1220556794 >> v1:172.21.15.111:6789/0 conn(0x7fbc14071050 legacy=0x7fbc140714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.293+0000 7fbc12ffd640 1 -- 172.21.15.111:0/1220556794 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc14070310 con 0x7fbc1406c6d0 2024-09-13T17:58:40.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.293+0000 7fbc137fe640 1 --2- 172.21.15.111:0/1220556794 >> v2:172.21.15.188:3300/0 conn(0x7fbc14158b00 0x7fbc1406c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:40.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.294+0000 7fbc12ffd640 1 --2- 172.21.15.111:0/1220556794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc1406c6d0 0x7fbc14070b10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbc1406ade0 tx=0x7fbc0802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435c9847dab1e6dc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:40.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.294+0000 7fbc127fc640 1 -- 172.21.15.111:0/1220556794 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08032440 con 0x7fbc1406c6d0 2024-09-13T17:58:40.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.294+0000 7fbc127fc640 1 -- 172.21.15.111:0/1220556794 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc080325a0 con 0x7fbc1406c6d0 2024-09-13T17:58:40.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.294+0000 7fbc127fc640 1 -- 172.21.15.111:0/1220556794 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08032930 con 0x7fbc1406c6d0 2024-09-13T17:58:40.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.295+0000 7fbc19d9c640 1 -- 172.21.15.111:0/1220556794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc1406c6d0 msgr2=0x7fbc14070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.295+0000 7fbc19d9c640 1 --2- 172.21.15.111:0/1220556794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc1406c6d0 0x7fbc14070b10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbc1406ade0 tx=0x7fbc0802a1a0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.295+0000 7fbc19d9c640 1 -- 172.21.15.111:0/1220556794 shutdown_connections 2024-09-13T17:58:40.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.295+0000 7fbc19d9c640 1 --2- 172.21.15.111:0/1220556794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc1406c6d0 0x7fbc14070b10 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.295+0000 7fbc19d9c640 1 --2- 172.21.15.111:0/1220556794 >> v2:172.21.15.188:3300/0 conn(0x7fbc14158b00 0x7fbc1406c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.295+0000 7fbc19d9c640 1 -- 172.21.15.111:0/1220556794 >> 172.21.15.111:0/1220556794 conn(0x7fbc1406a0b0 msgr2=0x7fbc1406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:40.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.295+0000 7fbc19d9c640 1 -- 172.21.15.111:0/1220556794 shutdown_connections 2024-09-13T17:58:40.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.295+0000 7fbc19d9c640 1 -- 172.21.15.111:0/1220556794 wait complete. 2024-09-13T17:58:40.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.296+0000 7fbc19d9c640 1 Processor -- start 2024-09-13T17:58:40.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.296+0000 7fbc19d9c640 1 -- start start 2024-09-13T17:58:40.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc19d9c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc1406c6d0 0x7fbc14097ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc19d9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc14071050 0x7fbc14099ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc19d9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc14158b00 0x7fbc1409a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc19d9c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc140c4810 con 0x7fbc1406c6d0 2024-09-13T17:58:40.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc19d9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc140c4ca0 con 0x7fbc14158b00 2024-09-13T17:58:40.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc19d9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc140c3c40 con 0x7fbc14071050 2024-09-13T17:58:40.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc137fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc1406c6d0 0x7fbc14097ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc137fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc1406c6d0 0x7fbc14097ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59140/0 (socket says 172.21.15.111:59140) 2024-09-13T17:58:40.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc137fe640 1 -- 172.21.15.111:0/1738774965 learned_addr learned my addr 172.21.15.111:0/1738774965 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:40.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc12ffd640 1 --2- 172.21.15.111:0/1738774965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc14071050 0x7fbc14099ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc13fff640 1 --2- 172.21.15.111:0/1738774965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc14158b00 0x7fbc1409a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc137fe640 1 -- 172.21.15.111:0/1738774965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc14071050 msgr2=0x7fbc14099ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc137fe640 1 --2- 172.21.15.111:0/1738774965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc14071050 0x7fbc14099ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc137fe640 1 -- 172.21.15.111:0/1738774965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc14158b00 msgr2=0x7fbc1409a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc137fe640 1 --2- 172.21.15.111:0/1738774965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc14158b00 0x7fbc1409a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc137fe640 1 -- 172.21.15.111:0/1738774965 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc08009400 con 0x7fbc1406c6d0 2024-09-13T17:58:40.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc13fff640 1 --2- 172.21.15.111:0/1738774965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc14158b00 0x7fbc1409a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:40.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.297+0000 7fbc137fe640 1 --2- 172.21.15.111:0/1738774965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc1406c6d0 0x7fbc14097ab0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400c7f0 tx=0x7fbc0400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:40.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.298+0000 7fbbfbfff640 1 -- 172.21.15.111:0/1738774965 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc04007820 con 0x7fbc1406c6d0 2024-09-13T17:58:40.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.298+0000 7fbbfbfff640 1 -- 172.21.15.111:0/1738774965 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc04007980 con 0x7fbc1406c6d0 2024-09-13T17:58:40.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.298+0000 7fbc19d9c640 1 -- 172.21.15.111:0/1738774965 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc140c3ee0 con 0x7fbc1406c6d0 2024-09-13T17:58:40.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.298+0000 7fbbfbfff640 1 -- 172.21.15.111:0/1738774965 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc04007cf0 con 0x7fbc1406c6d0 2024-09-13T17:58:40.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.298+0000 7fbc19d9c640 1 -- 172.21.15.111:0/1738774965 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc140c44a0 con 0x7fbc1406c6d0 2024-09-13T17:58:40.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.299+0000 7fbc19d9c640 1 -- 172.21.15.111:0/1738774965 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc14070b10 con 0x7fbc1406c6d0 2024-09-13T17:58:40.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.299+0000 7fbbfbfff640 1 -- 172.21.15.111:0/1738774965 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbc040113f0 con 0x7fbc1406c6d0 2024-09-13T17:58:40.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.299+0000 7fbbfbfff640 1 --2- 172.21.15.111:0/1738774965 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbc0006ac00 0x7fbc0006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.300+0000 7fbbfbfff640 1 -- 172.21.15.111:0/1738774965 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fbc04089270 con 0x7fbc1406c6d0 2024-09-13T17:58:40.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.300+0000 7fbc12ffd640 1 --2- 172.21.15.111:0/1738774965 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbc0006ac00 0x7fbc0006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.300+0000 7fbc12ffd640 1 --2- 172.21.15.111:0/1738774965 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbc0006ac00 0x7fbc0006d0c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbc08034000 tx=0x7fbc0802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:40.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.304+0000 7fbbfbfff640 1 -- 172.21.15.111:0/1738774965 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fbc04054660 con 0x7fbc1406c6d0 2024-09-13T17:58:40.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.528+0000 7fbc19d9c640 1 -- 172.21.15.111:0/1738774965 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fbc14071640 con 0x7fbc1406c6d0 2024-09-13T17:58:40.535 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.533+0000 7fbbfbfff640 1 -- 172.21.15.111:0/1738774965 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbc04057c80 con 0x7fbc1406c6d0 2024-09-13T17:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.535+0000 7fbc19d9c640 1 -- 172.21.15.111:0/1738774965 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbc0006ac00 msgr2=0x7fbc0006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.535+0000 7fbc19d9c640 1 --2- 172.21.15.111:0/1738774965 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbc0006ac00 0x7fbc0006d0c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbc08034000 tx=0x7fbc0802e000 comp rx=0 tx=0).stop 2024-09-13T17:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.535+0000 7fbc19d9c640 1 -- 172.21.15.111:0/1738774965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc1406c6d0 msgr2=0x7fbc14097ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.535+0000 7fbc19d9c640 1 --2- 172.21.15.111:0/1738774965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc1406c6d0 0x7fbc14097ab0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400c7f0 tx=0x7fbc0400ccb0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.535+0000 7fbc19d9c640 1 -- 172.21.15.111:0/1738774965 shutdown_connections 2024-09-13T17:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.535+0000 7fbc19d9c640 1 --2- 172.21.15.111:0/1738774965 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbc0006ac00 0x7fbc0006d0c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.535+0000 7fbc19d9c640 1 --2- 172.21.15.111:0/1738774965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc14158b00 0x7fbc1409a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.535+0000 7fbc19d9c640 1 --2- 172.21.15.111:0/1738774965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc14071050 0x7fbc14099ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.536+0000 7fbc19d9c640 1 --2- 172.21.15.111:0/1738774965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc1406c6d0 0x7fbc14097ab0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.536+0000 7fbc19d9c640 1 -- 172.21.15.111:0/1738774965 >> 172.21.15.111:0/1738774965 conn(0x7fbc1406a0b0 msgr2=0x7fbc140b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.536+0000 7fbc19d9c640 1 -- 172.21.15.111:0/1738774965 shutdown_connections 2024-09-13T17:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.536+0000 7fbc19d9c640 1 -- 172.21.15.111:0/1738774965 wait complete. 2024-09-13T17:58:40.548 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:40.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.645+0000 7f6bca9e9640 1 Processor -- start 2024-09-13T17:58:40.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.645+0000 7f6bca9e9640 1 -- start start 2024-09-13T17:58:40.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.645+0000 7f6bca9e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bc40b4660 0x7f6bc406b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.645+0000 7f6bca9e9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc40d1840 con 0x7f6bc415c6e0 2024-09-13T17:58:40.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.645+0000 7f6bca9e9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc40d0c30 con 0x7f6bc40d07b0 2024-09-13T17:58:40.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.645+0000 7f6bca9e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc40d0f10 con 0x7f6bc40b4660 2024-09-13T17:58:40.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc3fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6bc415c6e0 0x7f6bc40b3f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48730/0 (socket says 172.21.15.111:48730) 2024-09-13T17:58:40.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc3fff640 1 -- 172.21.15.111:0/720611282 learned_addr learned my addr 172.21.15.111:0/720611282 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:40.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc8f86640 1 --2- 172.21.15.111:0/720611282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bc40b4660 0x7f6bc406b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc2ffd640 1 -- 172.21.15.111:0/720611282 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4243387442 0 0) 0x7f6bc40d1840 con 0x7f6bc415c6e0 2024-09-13T17:58:40.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc2ffd640 1 -- 172.21.15.111:0/720611282 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bac003520 con 0x7f6bc415c6e0 2024-09-13T17:58:40.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc2ffd640 1 -- 172.21.15.111:0/720611282 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3991507046 0 0) 0x7f6bac003520 con 0x7f6bc415c6e0 2024-09-13T17:58:40.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc2ffd640 1 -- 172.21.15.111:0/720611282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bc40b4660 msgr2=0x7f6bc406b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:40.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc2ffd640 1 --2- 172.21.15.111:0/720611282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bc40b4660 0x7f6bc406b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc2ffd640 1 -- 172.21.15.111:0/720611282 >> v1:172.21.15.188:6789/0 conn(0x7f6bc40d07b0 legacy=0x7f6bc406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc2ffd640 1 -- 172.21.15.111:0/720611282 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc40d11f0 con 0x7f6bc415c6e0 2024-09-13T17:58:40.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc2ffd640 1 -- 172.21.15.111:0/720611282 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f6bb4001670 con 0x7f6bc415c6e0 2024-09-13T17:58:40.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc2ffd640 1 -- 172.21.15.111:0/720611282 >> v1:172.21.15.111:6789/0 conn(0x7f6bc415c6e0 legacy=0x7f6bc40b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc2ffd640 1 --2- 172.21.15.111:0/720611282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bac0039a0 0x7f6bac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc2ffd640 1 --2- 172.21.15.111:0/720611282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bc40b4660 0x7f6bac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc2ffd640 1 --2- 172.21.15.111:0/720611282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bac004d00 0x7f6bac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc2ffd640 1 -- 172.21.15.111:0/720611282 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc40d0f10 con 0x7f6bac0039a0 2024-09-13T17:58:40.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc2ffd640 1 -- 172.21.15.111:0/720611282 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac00bae0 con 0x7f6bac004d00 2024-09-13T17:58:40.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc2ffd640 1 -- 172.21.15.111:0/720611282 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac00bca0 con 0x7f6bc40b4660 2024-09-13T17:58:40.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc3fff640 1 --2- 172.21.15.111:0/720611282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bac0039a0 0x7f6bac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc37fe640 1 --2- 172.21.15.111:0/720611282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bc40b4660 0x7f6bac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc8f86640 1 --2- 172.21.15.111:0/720611282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bac004d00 0x7f6bac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc3fff640 1 -- 172.21.15.111:0/720611282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bc40b4660 msgr2=0x7f6bac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:40.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc3fff640 1 --2- 172.21.15.111:0/720611282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bc40b4660 0x7f6bac0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.646+0000 7f6bc3fff640 1 -- 172.21.15.111:0/720611282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bac004d00 msgr2=0x7f6bac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:40.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.647+0000 7f6bc3fff640 1 --2- 172.21.15.111:0/720611282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bac004d00 0x7f6bac005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.647+0000 7f6bc3fff640 1 -- 172.21.15.111:0/720611282 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac003520 con 0x7f6bac0039a0 2024-09-13T17:58:40.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.647+0000 7f6bc3fff640 1 --2- 172.21.15.111:0/720611282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bac0039a0 0x7f6bac003dc0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400c240 tx=0x7f6bac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57bc78aace64478 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:40.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.647+0000 7f6bc2ffd640 1 -- 172.21.15.111:0/720611282 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4005ad0 con 0x7f6bac0039a0 2024-09-13T17:58:40.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.647+0000 7f6bc2ffd640 1 -- 172.21.15.111:0/720611282 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb402d070 con 0x7f6bac0039a0 2024-09-13T17:58:40.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.647+0000 7f6bc2ffd640 1 -- 172.21.15.111:0/720611282 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4007070 con 0x7f6bac0039a0 2024-09-13T17:58:40.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.648+0000 7f6bca9e9640 1 -- 172.21.15.111:0/720611282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bac0039a0 msgr2=0x7f6bac003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.648+0000 7f6bca9e9640 1 --2- 172.21.15.111:0/720611282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bac0039a0 0x7f6bac003dc0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400c240 tx=0x7f6bac0057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.648+0000 7f6bc8f86640 1 -- 172.21.15.111:0/720611282 reap_dead start 2024-09-13T17:58:40.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.648+0000 7f6bca9e9640 1 -- 172.21.15.111:0/720611282 shutdown_connections 2024-09-13T17:58:40.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.648+0000 7f6bca9e9640 1 -- 172.21.15.111:0/720611282 >> 172.21.15.111:0/720611282 conn(0x7f6bc4089f60 msgr2=0x7f6bc40d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:40.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.648+0000 7f6bca9e9640 1 -- 172.21.15.111:0/720611282 shutdown_connections 2024-09-13T17:58:40.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.648+0000 7f6bca9e9640 1 -- 172.21.15.111:0/720611282 wait complete. 2024-09-13T17:58:40.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.649+0000 7f6bca9e9640 1 Processor -- start 2024-09-13T17:58:40.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.649+0000 7f6bca9e9640 1 -- start start 2024-09-13T17:58:40.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bca9e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bac0039a0 0x7f6bc40cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bca9e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bc41f2320 0x7f6bc4154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bca9e9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc40b0110 0x7f6bc40b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bca9e9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc41f3310 con 0x7f6bc40b0110 2024-09-13T17:58:40.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bca9e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc40b0ad0 con 0x7f6bc41f2320 2024-09-13T17:58:40.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bca9e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc4154da0 con 0x7f6bac0039a0 2024-09-13T17:58:40.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bc8f86640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc40b0110 0x7f6bc40b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bc8f86640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc40b0110 0x7f6bc40b0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59158/0 (socket says 172.21.15.111:59158) 2024-09-13T17:58:40.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bc8f86640 1 -- 172.21.15.111:0/2913990846 learned_addr learned my addr 172.21.15.111:0/2913990846 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:40.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bc3fff640 1 --2- 172.21.15.111:0/2913990846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bac0039a0 0x7f6bc40cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bc37fe640 1 --2- 172.21.15.111:0/2913990846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bc41f2320 0x7f6bc4154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bc8f86640 1 -- 172.21.15.111:0/2913990846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bac0039a0 msgr2=0x7f6bc40cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bc8f86640 1 --2- 172.21.15.111:0/2913990846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bac0039a0 0x7f6bc40cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bc8f86640 1 -- 172.21.15.111:0/2913990846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bc41f2320 msgr2=0x7f6bc4154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bc8f86640 1 --2- 172.21.15.111:0/2913990846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bc41f2320 0x7f6bc4154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bc8f86640 1 -- 172.21.15.111:0/2913990846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc41f27a0 con 0x7f6bc40b0110 2024-09-13T17:58:40.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bc3fff640 1 --2- 172.21.15.111:0/2913990846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bac0039a0 0x7f6bc40cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:40.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.650+0000 7f6bc8f86640 1 --2- 172.21.15.111:0/2913990846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc40b0110 0x7f6bc40b0590 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40d0540 tx=0x7f6bbc0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:40.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.651+0000 7f6bc0ff9640 1 -- 172.21.15.111:0/2913990846 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc00e070 con 0x7f6bc40b0110 2024-09-13T17:58:40.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.651+0000 7f6bc0ff9640 1 -- 172.21.15.111:0/2913990846 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bbc011e10 con 0x7f6bc40b0110 2024-09-13T17:58:40.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.651+0000 7f6bca9e9640 1 -- 172.21.15.111:0/2913990846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc41f2a70 con 0x7f6bc40b0110 2024-09-13T17:58:40.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.651+0000 7f6bc0ff9640 1 -- 172.21.15.111:0/2913990846 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc003990 con 0x7f6bc40b0110 2024-09-13T17:58:40.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.651+0000 7f6bca9e9640 1 -- 172.21.15.111:0/2913990846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc41f3030 con 0x7f6bc40b0110 2024-09-13T17:58:40.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.652+0000 7f6bca9e9640 1 -- 172.21.15.111:0/2913990846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc40ccb20 con 0x7f6bc40b0110 2024-09-13T17:58:40.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.652+0000 7f6bc0ff9640 1 -- 172.21.15.111:0/2913990846 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6bbc003af0 con 0x7f6bc40b0110 2024-09-13T17:58:40.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.652+0000 7f6bc0ff9640 1 --2- 172.21.15.111:0/2913990846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6bb006ac30 0x7f6bb006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.652+0000 7f6bc0ff9640 1 -- 172.21.15.111:0/2913990846 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f6bbc0896c0 con 0x7f6bc40b0110 2024-09-13T17:58:40.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.652+0000 7f6bc3fff640 1 --2- 172.21.15.111:0/2913990846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6bb006ac30 0x7f6bb006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.653+0000 7f6bc3fff640 1 --2- 172.21.15.111:0/2913990846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6bb006ac30 0x7f6bb006d0f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4032000 tx=0x7f6bb402d000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:40.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.656+0000 7f6bc0ff9640 1 -- 172.21.15.111:0/2913990846 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f6bbc054ab0 con 0x7f6bc40b0110 2024-09-13T17:58:40.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.818+0000 7f6bca9e9640 1 -- 172.21.15.111:0/2913990846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bc40b4c50 con 0x7f6bc40b0110 2024-09-13T17:58:40.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.819+0000 7f6bc0ff9640 1 -- 172.21.15.111:0/2913990846 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+13122 (secure 0 0 0) 0x7f6bbc0580d0 con 0x7f6bc40b0110 2024-09-13T17:58:40.821 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:40.821 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":35,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:38.061780+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:40.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.821+0000 7f6bca9e9640 1 -- 172.21.15.111:0/2913990846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6bb006ac30 msgr2=0x7f6bb006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.821+0000 7f6bca9e9640 1 --2- 172.21.15.111:0/2913990846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6bb006ac30 0x7f6bb006d0f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4032000 tx=0x7f6bb402d000 comp rx=0 tx=0).stop 2024-09-13T17:58:40.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.821+0000 7f6bca9e9640 1 -- 172.21.15.111:0/2913990846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc40b0110 msgr2=0x7f6bc40b0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.821+0000 7f6bca9e9640 1 --2- 172.21.15.111:0/2913990846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc40b0110 0x7f6bc40b0590 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40d0540 tx=0x7f6bbc0115a0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.821+0000 7f6bca9e9640 1 -- 172.21.15.111:0/2913990846 shutdown_connections 2024-09-13T17:58:40.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.821+0000 7f6bca9e9640 1 --2- 172.21.15.111:0/2913990846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc40b0110 0x7f6bc40b0590 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.821+0000 7f6bca9e9640 1 --2- 172.21.15.111:0/2913990846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6bb006ac30 0x7f6bb006d0f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.821+0000 7f6bca9e9640 1 --2- 172.21.15.111:0/2913990846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bc41f2320 0x7f6bc4154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.821+0000 7f6bca9e9640 1 --2- 172.21.15.111:0/2913990846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bac0039a0 0x7f6bc40cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.821+0000 7f6bca9e9640 1 -- 172.21.15.111:0/2913990846 >> 172.21.15.111:0/2913990846 conn(0x7f6bc4089f60 msgr2=0x7f6bc40d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:40.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.821+0000 7f6bca9e9640 1 -- 172.21.15.111:0/2913990846 shutdown_connections 2024-09-13T17:58:40.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.821+0000 7f6bca9e9640 1 -- 172.21.15.111:0/2913990846 wait complete. 2024-09-13T17:58:40.834 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:58:40.834 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:58:40.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.932+0000 7f1ad2365640 1 Processor -- start 2024-09-13T17:58:40.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.932+0000 7f1ad2365640 1 -- start start 2024-09-13T17:58:40.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.933+0000 7f1ad2365640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1acc0b20a0 0x7f1acc0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.933+0000 7f1ad2365640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1acc0cea30 0x7f1acc0ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.933+0000 7f1ad2365640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1acc0cee60 con 0x7f1acc15a3b0 2024-09-13T17:58:40.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.933+0000 7f1ad2365640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1acc15a790 con 0x7f1acc0cea30 2024-09-13T17:58:40.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.933+0000 7f1ad2365640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1acc0b2680 con 0x7f1acc0b20a0 2024-09-13T17:58:40.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.933+0000 7f1ad0902640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1acc0cea30 0x7f1acc0ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.933+0000 7f1ad0902640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1acc0cea30 0x7f1acc0ab9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59168/0 (socket says 172.21.15.111:59168) 2024-09-13T17:58:40.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.933+0000 7f1ad0902640 1 -- 172.21.15.111:0/1173759238 learned_addr learned my addr 172.21.15.111:0/1173759238 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:40.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.933+0000 7f1ac3fff640 1 --2- 172.21.15.111:0/1173759238 >> v2:172.21.15.188:3300/0 conn(0x7f1acc0b20a0 0x7f1acc0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.933+0000 7f1ad0902640 1 -- 172.21.15.111:0/1173759238 >> v2:172.21.15.188:3300/0 conn(0x7f1acc0b20a0 msgr2=0x7f1acc0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:40.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.934+0000 7f1ad0902640 1 --2- 172.21.15.111:0/1173759238 >> v2:172.21.15.188:3300/0 conn(0x7f1acc0b20a0 0x7f1acc0ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.934+0000 7f1ad0902640 1 -- 172.21.15.111:0/1173759238 >> v1:172.21.15.188:6789/0 conn(0x7f1acc15a3b0 legacy=0x7f1acc0b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:40.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.934+0000 7f1ad0902640 1 -- 172.21.15.111:0/1173759238 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1acc0b2960 con 0x7f1acc0cea30 2024-09-13T17:58:40.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.934+0000 7f1ac3fff640 1 --2- 172.21.15.111:0/1173759238 >> v2:172.21.15.188:3300/0 conn(0x7f1acc0b20a0 0x7f1acc0ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:40.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.934+0000 7f1ad0902640 1 --2- 172.21.15.111:0/1173759238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1acc0cea30 0x7f1acc0ab9e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0c5310 tx=0x7f1ac402a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dcd2979198789d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:40.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.934+0000 7f1acb7fe640 1 -- 172.21.15.111:0/1173759238 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4031600 con 0x7f1acc0cea30 2024-09-13T17:58:40.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.934+0000 7f1acb7fe640 1 -- 172.21.15.111:0/1173759238 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac402d050 con 0x7f1acc0cea30 2024-09-13T17:58:40.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.934+0000 7f1acb7fe640 1 -- 172.21.15.111:0/1173759238 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4031980 con 0x7f1acc0cea30 2024-09-13T17:58:40.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.935+0000 7f1ad2365640 1 -- 172.21.15.111:0/1173759238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1acc0cea30 msgr2=0x7f1acc0ab9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.935+0000 7f1ad2365640 1 --2- 172.21.15.111:0/1173759238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1acc0cea30 0x7f1acc0ab9e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0c5310 tx=0x7f1ac402a6b0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.935+0000 7f1ad2365640 1 -- 172.21.15.111:0/1173759238 shutdown_connections 2024-09-13T17:58:40.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.935+0000 7f1ad2365640 1 --2- 172.21.15.111:0/1173759238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1acc0cea30 0x7f1acc0ab9e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.935+0000 7f1ad2365640 1 --2- 172.21.15.111:0/1173759238 >> v2:172.21.15.188:3300/0 conn(0x7f1acc0b20a0 0x7f1acc0ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.935+0000 7f1ad2365640 1 -- 172.21.15.111:0/1173759238 >> 172.21.15.111:0/1173759238 conn(0x7f1acc06a0b0 msgr2=0x7f1acc06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:40.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.935+0000 7f1ad2365640 1 -- 172.21.15.111:0/1173759238 shutdown_connections 2024-09-13T17:58:40.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.935+0000 7f1ad2365640 1 -- 172.21.15.111:0/1173759238 wait complete. 2024-09-13T17:58:40.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.936+0000 7f1ad2365640 1 Processor -- start 2024-09-13T17:58:40.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.936+0000 7f1ad2365640 1 -- start start 2024-09-13T17:58:40.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.937+0000 7f1ad2365640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1acc0b20a0 0x7f1acc1f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.937+0000 7f1ad2365640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1acc0cea30 0x7f1acc0c3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.937+0000 7f1ad2365640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1acc15a3b0 0x7f1acc0b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.937+0000 7f1ad2365640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc0caa60 con 0x7f1acc0b20a0 2024-09-13T17:58:40.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.937+0000 7f1ad2365640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc0c3a70 con 0x7f1acc0cea30 2024-09-13T17:58:40.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.937+0000 7f1ad2365640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc0c3d50 con 0x7f1acc15a3b0 2024-09-13T17:58:40.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.937+0000 7f1acbfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1acc0b20a0 0x7f1acc1f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.937+0000 7f1acbfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1acc0b20a0 0x7f1acc1f1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59180/0 (socket says 172.21.15.111:59180) 2024-09-13T17:58:40.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.937+0000 7f1acbfff640 1 -- 172.21.15.111:0/3596508452 learned_addr learned my addr 172.21.15.111:0/3596508452 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:40.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.937+0000 7f1ac3fff640 1 --2- 172.21.15.111:0/3596508452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1acc0cea30 0x7f1acc0c3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.937+0000 7f1ad0902640 1 --2- 172.21.15.111:0/3596508452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1acc15a3b0 0x7f1acc0b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.937+0000 7f1acbfff640 1 -- 172.21.15.111:0/3596508452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1acc15a3b0 msgr2=0x7f1acc0b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.937+0000 7f1acbfff640 1 --2- 172.21.15.111:0/3596508452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1acc15a3b0 0x7f1acc0b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.937+0000 7f1acbfff640 1 -- 172.21.15.111:0/3596508452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1acc0cea30 msgr2=0x7f1acc0c3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:40.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.937+0000 7f1acbfff640 1 --2- 172.21.15.111:0/3596508452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1acc0cea30 0x7f1acc0c3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:40.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.937+0000 7f1acbfff640 1 -- 172.21.15.111:0/3596508452 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac4009b30 con 0x7f1acc0b20a0 2024-09-13T17:58:40.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.937+0000 7f1acbfff640 1 --2- 172.21.15.111:0/3596508452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1acc0b20a0 0x7f1acc1f1f60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00b3f0 tx=0x7f1abc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:40.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.938+0000 7f1ac97fa640 1 -- 172.21.15.111:0/3596508452 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc011830 con 0x7f1acc0b20a0 2024-09-13T17:58:40.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.938+0000 7f1ac97fa640 1 -- 172.21.15.111:0/3596508452 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1abc011990 con 0x7f1acc0b20a0 2024-09-13T17:58:40.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.938+0000 7f1ac97fa640 1 -- 172.21.15.111:0/3596508452 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc011d40 con 0x7f1acc0b20a0 2024-09-13T17:58:40.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.938+0000 7f1ad2365640 1 -- 172.21.15.111:0/3596508452 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1acc0c4030 con 0x7f1acc0b20a0 2024-09-13T17:58:40.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.938+0000 7f1ad2365640 1 -- 172.21.15.111:0/3596508452 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1acc0c45f0 con 0x7f1acc0b20a0 2024-09-13T17:58:40.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.939+0000 7f1ad2365640 1 -- 172.21.15.111:0/3596508452 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1acc0ce4f0 con 0x7f1acc0b20a0 2024-09-13T17:58:40.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.939+0000 7f1ac97fa640 1 -- 172.21.15.111:0/3596508452 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1abc01a400 con 0x7f1acc0b20a0 2024-09-13T17:58:40.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.939+0000 7f1ac97fa640 1 --2- 172.21.15.111:0/3596508452 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1aac06ac00 0x7f1aac06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.940+0000 7f1ac97fa640 1 -- 172.21.15.111:0/3596508452 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f1abc088f00 con 0x7f1acc0b20a0 2024-09-13T17:58:40.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.940+0000 7f1ac3fff640 1 --2- 172.21.15.111:0/3596508452 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1aac06ac00 0x7f1aac06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.940+0000 7f1ac3fff640 1 --2- 172.21.15.111:0/3596508452 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1aac06ac00 0x7f1aac06d0c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0ac960 tx=0x7f1ab4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:40.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:40.944+0000 7f1ac97fa640 1 -- 172.21.15.111:0/3596508452 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f1abc054380 con 0x7f1acc0b20a0 2024-09-13T17:58:41.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.105+0000 7f1ad2365640 1 -- 172.21.15.111:0/3596508452 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1acc0ca230 con 0x7f1acc0b20a0 2024-09-13T17:58:41.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.105+0000 7f1ac97fa640 1 -- 172.21.15.111:0/3596508452 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1abc0579a0 con 0x7f1acc0b20a0 2024-09-13T17:58:41.107 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:58:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.107+0000 7f1ad2365640 1 -- 172.21.15.111:0/3596508452 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1aac06ac00 msgr2=0x7f1aac06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.107+0000 7f1ad2365640 1 --2- 172.21.15.111:0/3596508452 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1aac06ac00 0x7f1aac06d0c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0ac960 tx=0x7f1ab4008040 comp rx=0 tx=0).stop 2024-09-13T17:58:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.107+0000 7f1ad2365640 1 -- 172.21.15.111:0/3596508452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1acc0b20a0 msgr2=0x7f1acc1f1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:41.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.107+0000 7f1ad2365640 1 --2- 172.21.15.111:0/3596508452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1acc0b20a0 0x7f1acc1f1f60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00b3f0 tx=0x7f1abc00b8b0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.107+0000 7f1ad2365640 1 -- 172.21.15.111:0/3596508452 shutdown_connections 2024-09-13T17:58:41.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.107+0000 7f1ad2365640 1 --2- 172.21.15.111:0/3596508452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1acc15a3b0 0x7f1acc0b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.107+0000 7f1ad2365640 1 --2- 172.21.15.111:0/3596508452 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f1aac06ac00 0x7f1aac06d0c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.107+0000 7f1ad2365640 1 --2- 172.21.15.111:0/3596508452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1acc0cea30 0x7f1acc0c3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.107+0000 7f1ad2365640 1 --2- 172.21.15.111:0/3596508452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1acc0b20a0 0x7f1acc1f1f60 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.107+0000 7f1ad2365640 1 -- 172.21.15.111:0/3596508452 >> 172.21.15.111:0/3596508452 conn(0x7f1acc06a0b0 msgr2=0x7f1acc071450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:41.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.107+0000 7f1ad2365640 1 -- 172.21.15.111:0/3596508452 shutdown_connections 2024-09-13T17:58:41.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.107+0000 7f1ad2365640 1 -- 172.21.15.111:0/3596508452 wait complete. 2024-09-13T17:58:41.120 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T17:58:41.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.216+0000 7fe03c726640 1 Processor -- start 2024-09-13T17:58:41.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.216+0000 7fe03c726640 1 -- start start 2024-09-13T17:58:41.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.217+0000 7fe03c726640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe0340d07b0 0x7fe03406ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:41.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.217+0000 7fe03c726640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0340d1840 con 0x7fe0340b4660 2024-09-13T17:58:41.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.217+0000 7fe03c726640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0340d0c30 con 0x7fe03415c6e0 2024-09-13T17:58:41.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.217+0000 7fe03c726640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0340d0f10 con 0x7fe0340d07b0 2024-09-13T17:58:41.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe03acc3640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe0340b4660 0x7fe03406b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48738/0 (socket says 172.21.15.111:48738) 2024-09-13T17:58:41.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe03acc3640 1 -- 172.21.15.111:0/3132453931 learned_addr learned my addr 172.21.15.111:0/3132453931 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:41.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe0394c0640 1 -- 172.21.15.111:0/3132453931 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090425024 0 0) 0x7fe0340d1840 con 0x7fe0340b4660 2024-09-13T17:58:41.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe0394c0640 1 -- 172.21.15.111:0/3132453931 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe020003520 con 0x7fe0340b4660 2024-09-13T17:58:41.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe039cc1640 1 --2- 172.21.15.111:0/3132453931 >> v2:172.21.15.188:3300/0 conn(0x7fe0340d07b0 0x7fe03406ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:41.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe0394c0640 1 -- 172.21.15.111:0/3132453931 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3339177294 0 0) 0x7fe020003520 con 0x7fe0340b4660 2024-09-13T17:58:41.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe0394c0640 1 -- 172.21.15.111:0/3132453931 >> v2:172.21.15.188:3300/0 conn(0x7fe0340d07b0 msgr2=0x7fe03406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:41.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe0394c0640 1 --2- 172.21.15.111:0/3132453931 >> v2:172.21.15.188:3300/0 conn(0x7fe0340d07b0 0x7fe03406ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe0394c0640 1 -- 172.21.15.111:0/3132453931 >> v1:172.21.15.188:6789/0 conn(0x7fe03415c6e0 legacy=0x7fe0340b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:41.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe0394c0640 1 -- 172.21.15.111:0/3132453931 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0340d11f0 con 0x7fe0340b4660 2024-09-13T17:58:41.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe0394c0640 1 -- 172.21.15.111:0/3132453931 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7fe02c0019f0 con 0x7fe0340b4660 2024-09-13T17:58:41.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe0394c0640 1 -- 172.21.15.111:0/3132453931 >> v1:172.21.15.111:6789/0 conn(0x7fe0340b4660 legacy=0x7fe03406b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:41.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe0394c0640 1 --2- 172.21.15.111:0/3132453931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe020003970 0x7fe020003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:41.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe0394c0640 1 --2- 172.21.15.111:0/3132453931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0200047c0 0x7fe020004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:41.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe0394c0640 1 --2- 172.21.15.111:0/3132453931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe020005180 0x7fe02000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:41.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe0394c0640 1 -- 172.21.15.111:0/3132453931 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0340d0f10 con 0x7fe0200047c0 2024-09-13T17:58:41.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe0394c0640 1 -- 172.21.15.111:0/3132453931 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02000c120 con 0x7fe020005180 2024-09-13T17:58:41.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe0394c0640 1 -- 172.21.15.111:0/3132453931 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02000c2e0 con 0x7fe020003970 2024-09-13T17:58:41.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe039cc1640 1 --2- 172.21.15.111:0/3132453931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0200047c0 0x7fe020004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:41.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe03a4c2640 1 --2- 172.21.15.111:0/3132453931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe020003970 0x7fe020003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:41.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.218+0000 7fe03acc3640 1 --2- 172.21.15.111:0/3132453931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe020005180 0x7fe02000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:41.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.219+0000 7fe039cc1640 1 -- 172.21.15.111:0/3132453931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe020003970 msgr2=0x7fe020003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:41.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.219+0000 7fe039cc1640 1 --2- 172.21.15.111:0/3132453931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe020003970 0x7fe020003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.219+0000 7fe039cc1640 1 -- 172.21.15.111:0/3132453931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe020005180 msgr2=0x7fe02000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:41.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.219+0000 7fe039cc1640 1 --2- 172.21.15.111:0/3132453931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe020005180 0x7fe02000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.219+0000 7fe03acc3640 1 -- 172.21.15.111:0/3132453931 reap_dead start 2024-09-13T17:58:41.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.219+0000 7fe039cc1640 1 -- 172.21.15.111:0/3132453931 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe020003520 con 0x7fe0200047c0 2024-09-13T17:58:41.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.219+0000 7fe039cc1640 1 --2- 172.21.15.111:0/3132453931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0200047c0 0x7fe020004c40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe024009ac0 tx=0x7fe0200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3935a85cfee88b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:41.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.219+0000 7fe0394c0640 1 -- 172.21.15.111:0/3132453931 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024031400 con 0x7fe0200047c0 2024-09-13T17:58:41.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.219+0000 7fe0394c0640 1 -- 172.21.15.111:0/3132453931 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02402d050 con 0x7fe0200047c0 2024-09-13T17:58:41.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.219+0000 7fe0394c0640 1 -- 172.21.15.111:0/3132453931 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024031780 con 0x7fe0200047c0 2024-09-13T17:58:41.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.220+0000 7fe03c726640 1 -- 172.21.15.111:0/3132453931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0200047c0 msgr2=0x7fe020004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:41.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.220+0000 7fe03c726640 1 --2- 172.21.15.111:0/3132453931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0200047c0 0x7fe020004c40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe024009ac0 tx=0x7fe0200057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.220+0000 7fe03c726640 1 -- 172.21.15.111:0/3132453931 shutdown_connections 2024-09-13T17:58:41.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.220+0000 7fe03c726640 1 --2- 172.21.15.111:0/3132453931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0200047c0 0x7fe020004c40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.220+0000 7fe03c726640 1 -- 172.21.15.111:0/3132453931 >> 172.21.15.111:0/3132453931 conn(0x7fe034089f60 msgr2=0x7fe0340d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:41.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.220+0000 7fe03c726640 1 -- 172.21.15.111:0/3132453931 shutdown_connections 2024-09-13T17:58:41.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.220+0000 7fe03c726640 1 -- 172.21.15.111:0/3132453931 wait complete. 2024-09-13T17:58:41.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.221+0000 7fe03c726640 1 Processor -- start 2024-09-13T17:58:41.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.221+0000 7fe03c726640 1 -- start start 2024-09-13T17:58:41.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.222+0000 7fe03c726640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0200047c0 0x7fe0340ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:41.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.222+0000 7fe03c726640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe03415a5a0 0x7fe0340b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:41.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.222+0000 7fe03c726640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0341f24d0 0x7fe0340b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:41.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.222+0000 7fe03c726640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0340ca700 con 0x7fe0200047c0 2024-09-13T17:58:41.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.222+0000 7fe03c726640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0340b6360 con 0x7fe03415a5a0 2024-09-13T17:58:41.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.222+0000 7fe03c726640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0340ca930 con 0x7fe0341f24d0 2024-09-13T17:58:41.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.222+0000 7fe03a4c2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0200047c0 0x7fe0340ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:41.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.222+0000 7fe03a4c2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0200047c0 0x7fe0340ca1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59196/0 (socket says 172.21.15.111:59196) 2024-09-13T17:58:41.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.222+0000 7fe03a4c2640 1 -- 172.21.15.111:0/3310255915 learned_addr learned my addr 172.21.15.111:0/3310255915 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:41.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.222+0000 7fe03acc3640 1 --2- 172.21.15.111:0/3310255915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0341f24d0 0x7fe0340b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:41.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.222+0000 7fe039cc1640 1 --2- 172.21.15.111:0/3310255915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe03415a5a0 0x7fe0340b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:41.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.222+0000 7fe03a4c2640 1 -- 172.21.15.111:0/3310255915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0341f24d0 msgr2=0x7fe0340b7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:41.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.222+0000 7fe03a4c2640 1 --2- 172.21.15.111:0/3310255915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0341f24d0 0x7fe0340b7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.222+0000 7fe03a4c2640 1 -- 172.21.15.111:0/3310255915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe03415a5a0 msgr2=0x7fe0340b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:41.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.222+0000 7fe03a4c2640 1 --2- 172.21.15.111:0/3310255915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe03415a5a0 0x7fe0340b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.222+0000 7fe03a4c2640 1 -- 172.21.15.111:0/3310255915 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0340cac10 con 0x7fe0200047c0 2024-09-13T17:58:41.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.222+0000 7fe03a4c2640 1 --2- 172.21.15.111:0/3310255915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0200047c0 0x7fe0340ca1c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe03000f540 tx=0x7fe03000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:41.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.223+0000 7fe02affd640 1 -- 172.21.15.111:0/3310255915 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0300052e0 con 0x7fe0200047c0 2024-09-13T17:58:41.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.223+0000 7fe02affd640 1 -- 172.21.15.111:0/3310255915 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe030005440 con 0x7fe0200047c0 2024-09-13T17:58:41.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.223+0000 7fe03c726640 1 -- 172.21.15.111:0/3310255915 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0340caef0 con 0x7fe0200047c0 2024-09-13T17:58:41.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.223+0000 7fe02affd640 1 -- 172.21.15.111:0/3310255915 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe030003830 con 0x7fe0200047c0 2024-09-13T17:58:41.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.223+0000 7fe03c726640 1 -- 172.21.15.111:0/3310255915 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0340cb4b0 con 0x7fe0200047c0 2024-09-13T17:58:41.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.224+0000 7fe03c726640 1 -- 172.21.15.111:0/3310255915 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0340cb670 con 0x7fe0200047c0 2024-09-13T17:58:41.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.224+0000 7fe02affd640 1 -- 172.21.15.111:0/3310255915 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe030003990 con 0x7fe0200047c0 2024-09-13T17:58:41.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.224+0000 7fe02affd640 1 --2- 172.21.15.111:0/3310255915 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe01806ac30 0x7fe01806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:41.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.225+0000 7fe02affd640 1 -- 172.21.15.111:0/3310255915 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe030088b70 con 0x7fe0200047c0 2024-09-13T17:58:41.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.225+0000 7fe039cc1640 1 --2- 172.21.15.111:0/3310255915 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe01806ac30 0x7fe01806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:41.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.225+0000 7fe039cc1640 1 --2- 172.21.15.111:0/3310255915 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe01806ac30 0x7fe01806d0f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe024033000 tx=0x7fe024008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:41.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.229+0000 7fe02affd640 1 -- 172.21.15.111:0/3310255915 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fe030053f60 con 0x7fe0200047c0 2024-09-13T17:58:41.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.393+0000 7fe03c726640 1 -- 172.21.15.111:0/3310255915 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe0340cc920 con 0x7fe0200047c0 2024-09-13T17:58:41.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.393+0000 7fe02affd640 1 -- 172.21.15.111:0/3310255915 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe030057580 con 0x7fe0200047c0 2024-09-13T17:58:41.395 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:41.396 INFO:teuthology.orchestra.run.smithi111.stdout:[ 2024-09-13T17:58:41.396 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-13T17:58:41.396 INFO:teuthology.orchestra.run.smithi111.stdout: "poolnum": 1, 2024-09-13T17:58:41.396 INFO:teuthology.orchestra.run.smithi111.stdout: "poolname": ".mgr" 2024-09-13T17:58:41.396 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:41.396 INFO:teuthology.orchestra.run.smithi111.stdout:] 2024-09-13T17:58:41.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.395+0000 7fe03c726640 1 -- 172.21.15.111:0/3310255915 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe01806ac30 msgr2=0x7fe01806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:41.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.395+0000 7fe03c726640 1 --2- 172.21.15.111:0/3310255915 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe01806ac30 0x7fe01806d0f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe024033000 tx=0x7fe024008030 comp rx=0 tx=0).stop 2024-09-13T17:58:41.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.395+0000 7fe03c726640 1 -- 172.21.15.111:0/3310255915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0200047c0 msgr2=0x7fe0340ca1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:41.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.395+0000 7fe03c726640 1 --2- 172.21.15.111:0/3310255915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0200047c0 0x7fe0340ca1c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe03000f540 tx=0x7fe03000fa00 comp rx=0 tx=0).stop 2024-09-13T17:58:41.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.395+0000 7fe03c726640 1 -- 172.21.15.111:0/3310255915 shutdown_connections 2024-09-13T17:58:41.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.395+0000 7fe03c726640 1 --2- 172.21.15.111:0/3310255915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0341f24d0 0x7fe0340b7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.395+0000 7fe03c726640 1 --2- 172.21.15.111:0/3310255915 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe01806ac30 0x7fe01806d0f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.395+0000 7fe03c726640 1 --2- 172.21.15.111:0/3310255915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe03415a5a0 0x7fe0340b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.395+0000 7fe03c726640 1 --2- 172.21.15.111:0/3310255915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0200047c0 0x7fe0340ca1c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.395+0000 7fe03c726640 1 -- 172.21.15.111:0/3310255915 >> 172.21.15.111:0/3310255915 conn(0x7fe034089f60 msgr2=0x7fe03406ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:41.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.395+0000 7fe03c726640 1 -- 172.21.15.111:0/3310255915 shutdown_connections 2024-09-13T17:58:41.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.395+0000 7fe03c726640 1 -- 172.21.15.111:0/3310255915 wait complete. 2024-09-13T17:58:41.408 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T17:58:41.408 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T17:58:41.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.507+0000 7f4ce5c9c640 1 Processor -- start 2024-09-13T17:58:41.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.507+0000 7f4ce5c9c640 1 -- start start 2024-09-13T17:58:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.508+0000 7f4ce5c9c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4ce0159310 0x7f4ce00ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.508+0000 7f4ce5c9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ce00b2b40 0x7f4ce00ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.508+0000 7f4ce5c9c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4ce00ab680 0x7f4ce00ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.508+0000 7f4ce5c9c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce00ad940 con 0x7f4ce0159310 2024-09-13T17:58:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.508+0000 7f4ce5c9c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce00ccbe0 con 0x7f4ce00ab680 2024-09-13T17:58:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.508+0000 7f4ce5c9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce00b1810 con 0x7f4ce00b2b40 2024-09-13T17:58:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.508+0000 7f4cdf7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4ce0159310 0x7f4ce00ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.508+0000 7f4cdf7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4ce0159310 0x7f4ce00ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59198/0 (socket says 172.21.15.111:59198) 2024-09-13T17:58:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.508+0000 7f4cdf7fe640 1 -- 172.21.15.111:0/2598101498 learned_addr learned my addr 172.21.15.111:0/2598101498 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.508+0000 7f4cdffff640 1 --2- 172.21.15.111:0/2598101498 >> v2:172.21.15.188:3300/0 conn(0x7f4ce00ab680 0x7f4ce00ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.508+0000 7f4cdeffd640 1 --2- 172.21.15.111:0/2598101498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ce00b2b40 0x7f4ce00ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.508+0000 7f4cdf7fe640 1 -- 172.21.15.111:0/2598101498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ce00b2b40 msgr2=0x7f4ce00ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.508+0000 7f4cdf7fe640 1 --2- 172.21.15.111:0/2598101498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ce00b2b40 0x7f4ce00ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.508+0000 7f4cdf7fe640 1 -- 172.21.15.111:0/2598101498 >> v2:172.21.15.188:3300/0 conn(0x7f4ce00ab680 msgr2=0x7f4ce00ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.508+0000 7f4cdf7fe640 1 --2- 172.21.15.111:0/2598101498 >> v2:172.21.15.188:3300/0 conn(0x7f4ce00ab680 0x7f4ce00ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.508+0000 7f4cdf7fe640 1 -- 172.21.15.111:0/2598101498 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce006d9e0 con 0x7f4ce0159310 2024-09-13T17:58:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.509+0000 7f4cdf7fe640 1 --2- 172.21.15.111:0/2598101498 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ce0159310 0x7f4ce00ad400 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00afdf0 tx=0x7f4cd002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffce4c3a82cdc4f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.509+0000 7f4cde7fc640 1 -- 172.21.15.111:0/2598101498 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd002ae90 con 0x7f4ce0159310 2024-09-13T17:58:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.509+0000 7f4cde7fc640 1 -- 172.21.15.111:0/2598101498 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd0032440 con 0x7f4ce0159310 2024-09-13T17:58:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.509+0000 7f4cde7fc640 1 -- 172.21.15.111:0/2598101498 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00327d0 con 0x7f4ce0159310 2024-09-13T17:58:41.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.510+0000 7f4ce5c9c640 1 -- 172.21.15.111:0/2598101498 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ce0159310 msgr2=0x7f4ce00ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:41.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.510+0000 7f4ce5c9c640 1 --2- 172.21.15.111:0/2598101498 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ce0159310 0x7f4ce00ad400 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00afdf0 tx=0x7f4cd002a140 comp rx=0 tx=0).stop 2024-09-13T17:58:41.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.510+0000 7f4ce5c9c640 1 -- 172.21.15.111:0/2598101498 shutdown_connections 2024-09-13T17:58:41.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.510+0000 7f4ce5c9c640 1 --2- 172.21.15.111:0/2598101498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ce00b2b40 0x7f4ce00ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.510+0000 7f4ce5c9c640 1 --2- 172.21.15.111:0/2598101498 >> v2:172.21.15.188:3300/0 conn(0x7f4ce00ab680 0x7f4ce00ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.510+0000 7f4ce5c9c640 1 --2- 172.21.15.111:0/2598101498 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ce0159310 0x7f4ce00ad400 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.510+0000 7f4ce5c9c640 1 -- 172.21.15.111:0/2598101498 >> 172.21.15.111:0/2598101498 conn(0x7f4ce006a0b0 msgr2=0x7f4ce006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.510+0000 7f4ce5c9c640 1 -- 172.21.15.111:0/2598101498 shutdown_connections 2024-09-13T17:58:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.510+0000 7f4ce5c9c640 1 -- 172.21.15.111:0/2598101498 wait complete. 2024-09-13T17:58:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.511+0000 7f4ce5c9c640 1 Processor -- start 2024-09-13T17:58:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.511+0000 7f4ce5c9c640 1 -- start start 2024-09-13T17:58:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4ce5c9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ce00ab680 0x7f4ce00c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4ce5c9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ce00b2b40 0x7f4ce009a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4ce5c9c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ce0159310 0x7f4ce009acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4ce5c9c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce0098a40 con 0x7f4ce0159310 2024-09-13T17:58:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4ce5c9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce01560f0 con 0x7f4ce00ab680 2024-09-13T17:58:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4ce5c9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce00c4f90 con 0x7f4ce00b2b40 2024-09-13T17:58:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4cdffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ce0159310 0x7f4ce009acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4cdffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ce0159310 0x7f4ce009acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59208/0 (socket says 172.21.15.111:59208) 2024-09-13T17:58:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4cdffff640 1 -- 172.21.15.111:0/2882354092 learned_addr learned my addr 172.21.15.111:0/2882354092 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4cdf7fe640 1 --2- 172.21.15.111:0/2882354092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ce00ab680 0x7f4ce00c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4cdeffd640 1 --2- 172.21.15.111:0/2882354092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ce00b2b40 0x7f4ce009a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4cdffff640 1 -- 172.21.15.111:0/2882354092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ce00b2b40 msgr2=0x7f4ce009a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4cdffff640 1 --2- 172.21.15.111:0/2882354092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ce00b2b40 0x7f4ce009a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4cdffff640 1 -- 172.21.15.111:0/2882354092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ce00ab680 msgr2=0x7f4ce00c4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4cdffff640 1 --2- 172.21.15.111:0/2882354092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ce00ab680 0x7f4ce00c4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4cdffff640 1 -- 172.21.15.111:0/2882354092 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd00094d0 con 0x7f4ce0159310 2024-09-13T17:58:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4cdffff640 1 --2- 172.21.15.111:0/2882354092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ce0159310 0x7f4ce009acd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800ef90 tx=0x7f4cd800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4cc7fff640 1 -- 172.21.15.111:0/2882354092 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8009ca0 con 0x7f4ce0159310 2024-09-13T17:58:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4cc7fff640 1 -- 172.21.15.111:0/2882354092 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd800f040 con 0x7f4ce0159310 2024-09-13T17:58:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4cc7fff640 1 -- 172.21.15.111:0/2882354092 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd80146c0 con 0x7f4ce0159310 2024-09-13T17:58:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.512+0000 7f4ce5c9c640 1 -- 172.21.15.111:0/2882354092 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce01559a0 con 0x7f4ce0159310 2024-09-13T17:58:41.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.513+0000 7f4ce5c9c640 1 -- 172.21.15.111:0/2882354092 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce0098380 con 0x7f4ce0159310 2024-09-13T17:58:41.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.513+0000 7f4ce5c9c640 1 -- 172.21.15.111:0/2882354092 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce0098b80 con 0x7f4ce0159310 2024-09-13T17:58:41.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.514+0000 7f4cc7fff640 1 -- 172.21.15.111:0/2882354092 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4cd8014820 con 0x7f4ce0159310 2024-09-13T17:58:41.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.514+0000 7f4cc7fff640 1 --2- 172.21.15.111:0/2882354092 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4ccc06ac00 0x7f4ccc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.514+0000 7f4cdf7fe640 1 --2- 172.21.15.111:0/2882354092 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4ccc06ac00 0x7f4ccc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.515+0000 7f4cc7fff640 1 -- 172.21.15.111:0/2882354092 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4cd808a000 con 0x7f4ce0159310 2024-09-13T17:58:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.515+0000 7f4cdf7fe640 1 --2- 172.21.15.111:0/2882354092 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4ccc06ac00 0x7f4ccc06d0c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0034000 tx=0x7f4cd002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.519+0000 7f4cc7fff640 1 -- 172.21.15.111:0/2882354092 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4cd8055480 con 0x7f4ce0159310 2024-09-13T17:58:41.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:41.680+0000 7f4ce5c9c640 1 -- 172.21.15.111:0/2882354092 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4ce00cee10 con 0x7f4ce0159310 2024-09-13T17:58:42.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.184+0000 7f4cc7fff640 1 -- 172.21.15.111:0/2882354092 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4cd8058aa0 con 0x7f4ce0159310 2024-09-13T17:58:42.186 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' created 2024-09-13T17:58:42.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.187+0000 7f4ce5c9c640 1 -- 172.21.15.111:0/2882354092 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4ccc06ac00 msgr2=0x7f4ccc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:42.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.187+0000 7f4ce5c9c640 1 --2- 172.21.15.111:0/2882354092 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4ccc06ac00 0x7f4ccc06d0c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0034000 tx=0x7f4cd002e000 comp rx=0 tx=0).stop 2024-09-13T17:58:42.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.187+0000 7f4ce5c9c640 1 -- 172.21.15.111:0/2882354092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ce0159310 msgr2=0x7f4ce009acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:42.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.187+0000 7f4ce5c9c640 1 --2- 172.21.15.111:0/2882354092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ce0159310 0x7f4ce009acd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800ef90 tx=0x7f4cd800c490 comp rx=0 tx=0).stop 2024-09-13T17:58:42.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.187+0000 7f4ce5c9c640 1 -- 172.21.15.111:0/2882354092 shutdown_connections 2024-09-13T17:58:42.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.187+0000 7f4ce5c9c640 1 --2- 172.21.15.111:0/2882354092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ce0159310 0x7f4ce009acd0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:42.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.187+0000 7f4ce5c9c640 1 --2- 172.21.15.111:0/2882354092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ce00b2b40 0x7f4ce009a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:42.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.187+0000 7f4ce5c9c640 1 --2- 172.21.15.111:0/2882354092 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4ccc06ac00 0x7f4ccc06d0c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:42.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.187+0000 7f4ce5c9c640 1 --2- 172.21.15.111:0/2882354092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ce00ab680 0x7f4ce00c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:42.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.187+0000 7f4ce5c9c640 1 -- 172.21.15.111:0/2882354092 >> 172.21.15.111:0/2882354092 conn(0x7f4ce006a0b0 msgr2=0x7f4ce006c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:42.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.188+0000 7f4ce5c9c640 1 -- 172.21.15.111:0/2882354092 shutdown_connections 2024-09-13T17:58:42.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.188+0000 7f4ce5c9c640 1 -- 172.21.15.111:0/2882354092 wait complete. 2024-09-13T17:58:42.203 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T17:58:42.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.307+0000 7f16719ec640 1 Processor -- start 2024-09-13T17:58:42.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.307+0000 7f16719ec640 1 -- start start 2024-09-13T17:58:42.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f16719ec640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f166c159310 0x7f166c0ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:42.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f16719ec640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f166c0b2b40 0x7f166c0ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:42.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f16719ec640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f166c0ab680 0x7f166c0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:42.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f16719ec640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166c0ad940 con 0x7f166c159310 2024-09-13T17:58:42.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f16719ec640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166c0ccbe0 con 0x7f166c0ab680 2024-09-13T17:58:42.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f16719ec640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c0b1810 con 0x7f166c0b2b40 2024-09-13T17:58:42.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f166affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f166c159310 0x7f166c0ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:42.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f166affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f166c159310 0x7f166c0ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40108/0 (socket says 172.21.15.111:40108) 2024-09-13T17:58:42.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f166affd640 1 -- 172.21.15.111:0/3018931125 learned_addr learned my addr 172.21.15.111:0/3018931125 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:42.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f166b7fe640 1 --2- 172.21.15.111:0/3018931125 >> v2:172.21.15.188:3300/0 conn(0x7f166c0ab680 0x7f166c0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:42.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f16627fc640 1 --2- 172.21.15.111:0/3018931125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f166c0b2b40 0x7f166c0ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:42.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f166affd640 1 -- 172.21.15.111:0/3018931125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f166c0b2b40 msgr2=0x7f166c0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:42.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f166affd640 1 --2- 172.21.15.111:0/3018931125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f166c0b2b40 0x7f166c0ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:42.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f166affd640 1 -- 172.21.15.111:0/3018931125 >> v2:172.21.15.188:3300/0 conn(0x7f166c0ab680 msgr2=0x7f166c0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:42.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f166affd640 1 --2- 172.21.15.111:0/3018931125 >> v2:172.21.15.188:3300/0 conn(0x7f166c0ab680 0x7f166c0ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:42.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f166affd640 1 -- 172.21.15.111:0/3018931125 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c06d9e0 con 0x7f166c159310 2024-09-13T17:58:42.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f166b7fe640 1 --2- 172.21.15.111:0/3018931125 >> v2:172.21.15.188:3300/0 conn(0x7f166c0ab680 0x7f166c0ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:42.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.308+0000 7f16627fc640 1 --2- 172.21.15.111:0/3018931125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f166c0b2b40 0x7f166c0ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:42.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.309+0000 7f166affd640 1 --2- 172.21.15.111:0/3018931125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f166c159310 0x7f166c0ad400 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f166c0afdf0 tx=0x7f165c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9fc9960858bb82 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:42.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.309+0000 7f166a7fc640 1 -- 172.21.15.111:0/3018931125 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c02ae90 con 0x7f166c159310 2024-09-13T17:58:42.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.309+0000 7f166a7fc640 1 -- 172.21.15.111:0/3018931125 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165c032440 con 0x7f166c159310 2024-09-13T17:58:42.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.309+0000 7f166a7fc640 1 -- 172.21.15.111:0/3018931125 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c0327d0 con 0x7f166c159310 2024-09-13T17:58:42.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.310+0000 7f16719ec640 1 -- 172.21.15.111:0/3018931125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f166c159310 msgr2=0x7f166c0ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:42.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.310+0000 7f16719ec640 1 --2- 172.21.15.111:0/3018931125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f166c159310 0x7f166c0ad400 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f166c0afdf0 tx=0x7f165c02a140 comp rx=0 tx=0).stop 2024-09-13T17:58:42.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.310+0000 7f16719ec640 1 -- 172.21.15.111:0/3018931125 shutdown_connections 2024-09-13T17:58:42.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.310+0000 7f16719ec640 1 --2- 172.21.15.111:0/3018931125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f166c0b2b40 0x7f166c0ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:42.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.310+0000 7f16719ec640 1 --2- 172.21.15.111:0/3018931125 >> v2:172.21.15.188:3300/0 conn(0x7f166c0ab680 0x7f166c0ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:42.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.310+0000 7f16719ec640 1 --2- 172.21.15.111:0/3018931125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f166c159310 0x7f166c0ad400 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:42.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.310+0000 7f16719ec640 1 -- 172.21.15.111:0/3018931125 >> 172.21.15.111:0/3018931125 conn(0x7f166c06a0b0 msgr2=0x7f166c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:42.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.310+0000 7f16719ec640 1 -- 172.21.15.111:0/3018931125 shutdown_connections 2024-09-13T17:58:42.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.310+0000 7f16719ec640 1 -- 172.21.15.111:0/3018931125 wait complete. 2024-09-13T17:58:42.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.311+0000 7f16719ec640 1 Processor -- start 2024-09-13T17:58:42.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.311+0000 7f16719ec640 1 -- start start 2024-09-13T17:58:42.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.312+0000 7f16719ec640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f166c0ab680 0x7f166c099ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:42.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.312+0000 7f16719ec640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f166c0b2b40 0x7f166c0c3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:42.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.312+0000 7f16719ec640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f166c159310 0x7f166c152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:42.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.312+0000 7f16719ec640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c09a6d0 con 0x7f166c0b2b40 2024-09-13T17:58:42.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.312+0000 7f16719ec640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c09a9b0 con 0x7f166c159310 2024-09-13T17:58:42.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.312+0000 7f16719ec640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c09ac90 con 0x7f166c0ab680 2024-09-13T17:58:42.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.312+0000 7f16627fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f166c0b2b40 0x7f166c0c3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:42.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.312+0000 7f16627fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f166c0b2b40 0x7f166c0c3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40120/0 (socket says 172.21.15.111:40120) 2024-09-13T17:58:42.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.312+0000 7f16627fc640 1 -- 172.21.15.111:0/2268084830 learned_addr learned my addr 172.21.15.111:0/2268084830 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:42.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.312+0000 7f166b7fe640 1 --2- 172.21.15.111:0/2268084830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f166c159310 0x7f166c152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:42.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.312+0000 7f166affd640 1 --2- 172.21.15.111:0/2268084830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f166c0ab680 0x7f166c099ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:42.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.312+0000 7f16627fc640 1 -- 172.21.15.111:0/2268084830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f166c0ab680 msgr2=0x7f166c099ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:42.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.312+0000 7f16627fc640 1 --2- 172.21.15.111:0/2268084830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f166c0ab680 0x7f166c099ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:42.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.312+0000 7f16627fc640 1 -- 172.21.15.111:0/2268084830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f166c159310 msgr2=0x7f166c152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:42.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.312+0000 7f16627fc640 1 --2- 172.21.15.111:0/2268084830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f166c159310 0x7f166c152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:42.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.312+0000 7f16627fc640 1 -- 172.21.15.111:0/2268084830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165c0094d0 con 0x7f166c0b2b40 2024-09-13T17:58:42.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.312+0000 7f16627fc640 1 --2- 172.21.15.111:0/2268084830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f166c0b2b40 0x7f166c0c3a40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f165400c7f0 tx=0x7f165400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:42.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.313+0000 7f1668ff9640 1 -- 172.21.15.111:0/2268084830 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1654007a30 con 0x7f166c0b2b40 2024-09-13T17:58:42.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.313+0000 7f1668ff9640 1 -- 172.21.15.111:0/2268084830 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1654007b90 con 0x7f166c0b2b40 2024-09-13T17:58:42.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.313+0000 7f16719ec640 1 -- 172.21.15.111:0/2268084830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f166c09af70 con 0x7f166c0b2b40 2024-09-13T17:58:42.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.313+0000 7f1668ff9640 1 -- 172.21.15.111:0/2268084830 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165400ce40 con 0x7f166c0b2b40 2024-09-13T17:58:42.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.313+0000 7f16719ec640 1 -- 172.21.15.111:0/2268084830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166c09a3e0 con 0x7f166c0b2b40 2024-09-13T17:58:42.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.314+0000 7f16719ec640 1 -- 172.21.15.111:0/2268084830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f166c09b490 con 0x7f166c0b2b40 2024-09-13T17:58:42.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.314+0000 7f1668ff9640 1 -- 172.21.15.111:0/2268084830 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f16540113f0 con 0x7f166c0b2b40 2024-09-13T17:58:42.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.315+0000 7f1668ff9640 1 --2- 172.21.15.111:0/2268084830 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f164c06ac00 0x7f164c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:42.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.315+0000 7f1668ff9640 1 -- 172.21.15.111:0/2268084830 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f1654089420 con 0x7f166c0b2b40 2024-09-13T17:58:42.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.315+0000 7f166affd640 1 --2- 172.21.15.111:0/2268084830 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f164c06ac00 0x7f164c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:42.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.315+0000 7f166affd640 1 --2- 172.21.15.111:0/2268084830 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f164c06ac00 0x7f164c06d0c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f165c034000 tx=0x7f165c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:42.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.319+0000 7f1668ff9640 1 -- 172.21.15.111:0/2268084830 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f166c09b490 con 0x7f166c0b2b40 2024-09-13T17:58:42.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:42.481+0000 7f16719ec640 1 -- 172.21.15.111:0/2268084830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f166c09b490 con 0x7f166c0b2b40 2024-09-13T17:58:43.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.190+0000 7f1668ff9640 1 -- 172.21.15.111:0/2268084830 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7f166c09b490 con 0x7f166c0b2b40 2024-09-13T17:58:43.192 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' created 2024-09-13T17:58:43.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.199+0000 7f16719ec640 1 -- 172.21.15.111:0/2268084830 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f164c06ac00 msgr2=0x7f164c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.199+0000 7f16719ec640 1 --2- 172.21.15.111:0/2268084830 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f164c06ac00 0x7f164c06d0c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f165c034000 tx=0x7f165c02e000 comp rx=0 tx=0).stop 2024-09-13T17:58:43.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.199+0000 7f16719ec640 1 -- 172.21.15.111:0/2268084830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f166c0b2b40 msgr2=0x7f166c0c3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.199+0000 7f16719ec640 1 --2- 172.21.15.111:0/2268084830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f166c0b2b40 0x7f166c0c3a40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f165400c7f0 tx=0x7f165400cbf0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.203+0000 7f16719ec640 1 -- 172.21.15.111:0/2268084830 shutdown_connections 2024-09-13T17:58:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.203+0000 7f16719ec640 1 --2- 172.21.15.111:0/2268084830 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f164c06ac00 0x7f164c06d0c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.203+0000 7f16719ec640 1 --2- 172.21.15.111:0/2268084830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f166c159310 0x7f166c152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.203+0000 7f16719ec640 1 --2- 172.21.15.111:0/2268084830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f166c0b2b40 0x7f166c0c3a40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.203+0000 7f16719ec640 1 --2- 172.21.15.111:0/2268084830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f166c0ab680 0x7f166c099ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.204+0000 7f16719ec640 1 -- 172.21.15.111:0/2268084830 >> 172.21.15.111:0/2268084830 conn(0x7f166c06a0b0 msgr2=0x7f166c0b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:43.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.205+0000 7f16719ec640 1 -- 172.21.15.111:0/2268084830 shutdown_connections 2024-09-13T17:58:43.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.205+0000 7f16719ec640 1 -- 172.21.15.111:0/2268084830 wait complete. 2024-09-13T17:58:43.222 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T17:58:43.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.325+0000 7f81e516c640 1 Processor -- start 2024-09-13T17:58:43.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.326+0000 7f81e516c640 1 -- start start 2024-09-13T17:58:43.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.326+0000 7f81e516c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f81e0158b00 0x7f81e006c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.326+0000 7f81e516c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e00b2d10 con 0x7f81e00b2170 2024-09-13T17:58:43.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.326+0000 7f81e516c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e00b2ff0 con 0x7f81e006c6d0 2024-09-13T17:58:43.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.326+0000 7f81e516c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e006cb60 con 0x7f81e0158b00 2024-09-13T17:58:43.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.326+0000 7f81ded9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f81e0158b00 0x7f81e006c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.326+0000 7f81df59e640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f81e00b2170 0x7f81e0071220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58118/0 (socket says 172.21.15.111:58118) 2024-09-13T17:58:43.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.326+0000 7f81df59e640 1 -- 172.21.15.111:0/4164626974 learned_addr learned my addr 172.21.15.111:0/4164626974 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:43.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.327+0000 7f81ded9d640 1 -- 172.21.15.111:0/4164626974 >> v1:172.21.15.188:6789/0 conn(0x7f81e006c6d0 legacy=0x7f81e0070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:43.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.327+0000 7f81ded9d640 1 -- 172.21.15.111:0/4164626974 >> v1:172.21.15.111:6789/0 conn(0x7f81e00b2170 legacy=0x7f81e0071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.327+0000 7f81ded9d640 1 -- 172.21.15.111:0/4164626974 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e0070310 con 0x7f81e0158b00 2024-09-13T17:58:43.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.327+0000 7f81ddd9b640 1 -- 172.21.15.111:0/4164626974 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 928237 0 0) 0x7f81e00b2d10 con 0x7f81e00b2170 2024-09-13T17:58:43.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.327+0000 7f81ded9d640 1 --2- 172.21.15.111:0/4164626974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e0158b00 0x7f81e006c190 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f81d000aea0 tx=0x7f81d002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58185b15eda7f752 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:43.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.328+0000 7f81ddd9b640 1 -- 172.21.15.111:0/4164626974 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0008640 con 0x7f81e0158b00 2024-09-13T17:58:43.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.328+0000 7f81ddd9b640 1 -- 172.21.15.111:0/4164626974 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d002e050 con 0x7f81e0158b00 2024-09-13T17:58:43.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.328+0000 7f81ddd9b640 1 -- 172.21.15.111:0/4164626974 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d00089c0 con 0x7f81e0158b00 2024-09-13T17:58:43.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.328+0000 7f81e516c640 1 -- 172.21.15.111:0/4164626974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e0158b00 msgr2=0x7f81e006c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.328+0000 7f81e516c640 1 --2- 172.21.15.111:0/4164626974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e0158b00 0x7f81e006c190 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f81d000aea0 tx=0x7f81d002b7f0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.328+0000 7f81e516c640 1 -- 172.21.15.111:0/4164626974 shutdown_connections 2024-09-13T17:58:43.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.328+0000 7f81e516c640 1 --2- 172.21.15.111:0/4164626974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e0158b00 0x7f81e006c190 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.328+0000 7f81e516c640 1 -- 172.21.15.111:0/4164626974 >> 172.21.15.111:0/4164626974 conn(0x7f81e006a0b0 msgr2=0x7f81e006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:43.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.328+0000 7f81e516c640 1 -- 172.21.15.111:0/4164626974 shutdown_connections 2024-09-13T17:58:43.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.328+0000 7f81e516c640 1 -- 172.21.15.111:0/4164626974 wait complete. 2024-09-13T17:58:43.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.330+0000 7f81e516c640 1 Processor -- start 2024-09-13T17:58:43.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.330+0000 7f81e516c640 1 -- start start 2024-09-13T17:58:43.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.330+0000 7f81e516c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81e006c6d0 0x7f81e01f1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.330+0000 7f81e516c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81e00b2170 0x7f81e00c3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.330+0000 7f81e516c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e0158b00 0x7f81e00c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.330+0000 7f81e516c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e00c4050 con 0x7f81e0158b00 2024-09-13T17:58:43.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.330+0000 7f81e516c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e00c4330 con 0x7f81e006c6d0 2024-09-13T17:58:43.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.330+0000 7f81e516c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e00c4610 con 0x7f81e00b2170 2024-09-13T17:58:43.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.330+0000 7f81df59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e0158b00 0x7f81e00c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.330+0000 7f81df59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e0158b00 0x7f81e00c39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40138/0 (socket says 172.21.15.111:40138) 2024-09-13T17:58:43.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.330+0000 7f81df59e640 1 -- 172.21.15.111:0/3197549424 learned_addr learned my addr 172.21.15.111:0/3197549424 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:43.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.331+0000 7f81ded9d640 1 --2- 172.21.15.111:0/3197549424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81e006c6d0 0x7f81e01f1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.331+0000 7f81de59c640 1 --2- 172.21.15.111:0/3197549424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81e00b2170 0x7f81e00c3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.331+0000 7f81df59e640 1 -- 172.21.15.111:0/3197549424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81e00b2170 msgr2=0x7f81e00c3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.331+0000 7f81df59e640 1 --2- 172.21.15.111:0/3197549424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81e00b2170 0x7f81e00c3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.331+0000 7f81df59e640 1 -- 172.21.15.111:0/3197549424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81e006c6d0 msgr2=0x7f81e01f1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.331+0000 7f81df59e640 1 --2- 172.21.15.111:0/3197549424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81e006c6d0 0x7f81e01f1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.331+0000 7f81df59e640 1 -- 172.21.15.111:0/3197549424 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d000ab80 con 0x7f81e0158b00 2024-09-13T17:58:43.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.331+0000 7f81de59c640 1 --2- 172.21.15.111:0/3197549424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81e00b2170 0x7f81e00c3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:43.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.331+0000 7f81df59e640 1 --2- 172.21.15.111:0/3197549424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e0158b00 0x7f81e00c39d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f81d800c450 tx=0x7f81d800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:43.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.331+0000 7f81c77fe640 1 -- 172.21.15.111:0/3197549424 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d80074e0 con 0x7f81e0158b00 2024-09-13T17:58:43.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.331+0000 7f81c77fe640 1 -- 172.21.15.111:0/3197549424 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d8007660 con 0x7f81e0158b00 2024-09-13T17:58:43.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.331+0000 7f81e516c640 1 -- 172.21.15.111:0/3197549424 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e00c48f0 con 0x7f81e0158b00 2024-09-13T17:58:43.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.331+0000 7f81c77fe640 1 -- 172.21.15.111:0/3197549424 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8004a20 con 0x7f81e0158b00 2024-09-13T17:58:43.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.331+0000 7f81e516c640 1 -- 172.21.15.111:0/3197549424 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e00c1ce0 con 0x7f81e0158b00 2024-09-13T17:58:43.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.332+0000 7f81e516c640 1 -- 172.21.15.111:0/3197549424 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e0071cc0 con 0x7f81e0158b00 2024-09-13T17:58:43.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.333+0000 7f81c77fe640 1 -- 172.21.15.111:0/3197549424 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f81d8004b80 con 0x7f81e0158b00 2024-09-13T17:58:43.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.333+0000 7f81c77fe640 1 --2- 172.21.15.111:0/3197549424 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f81cc06ac00 0x7f81cc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.333+0000 7f81c77fe640 1 -- 172.21.15.111:0/3197549424 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f81d8088ae0 con 0x7f81e0158b00 2024-09-13T17:58:43.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.333+0000 7f81ded9d640 1 --2- 172.21.15.111:0/3197549424 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f81cc06ac00 0x7f81cc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.334+0000 7f81ded9d640 1 --2- 172.21.15.111:0/3197549424 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f81cc06ac00 0x7f81cc06d0c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f81e00cf390 tx=0x7f81d000a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:43.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.337+0000 7f81c77fe640 1 -- 172.21.15.111:0/3197549424 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f81d8057950 con 0x7f81e0158b00 2024-09-13T17:58:43.534 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.533+0000 7f81e516c640 1 -- 172.21.15.111:0/3197549424 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f81e006c2e0 con 0x7f81e0158b00 2024-09-13T17:58:43.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.542+0000 7f81c77fe640 1 -- 172.21.15.111:0/3197549424 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f81d8018020 con 0x7f81e0158b00 2024-09-13T17:58:43.544 INFO:teuthology.orchestra.run.smithi111.stderr:new fs with metadata pool 5 and data pool 6 2024-09-13T17:58:43.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.549+0000 7f81e516c640 1 -- 172.21.15.111:0/3197549424 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f81cc06ac00 msgr2=0x7f81cc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.549+0000 7f81e516c640 1 --2- 172.21.15.111:0/3197549424 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f81cc06ac00 0x7f81cc06d0c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f81e00cf390 tx=0x7f81d000a000 comp rx=0 tx=0).stop 2024-09-13T17:58:43.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.549+0000 7f81e516c640 1 -- 172.21.15.111:0/3197549424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e0158b00 msgr2=0x7f81e00c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.549+0000 7f81e516c640 1 --2- 172.21.15.111:0/3197549424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e0158b00 0x7f81e00c39d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f81d800c450 tx=0x7f81d800c910 comp rx=0 tx=0).stop 2024-09-13T17:58:43.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.549+0000 7f81e516c640 1 -- 172.21.15.111:0/3197549424 shutdown_connections 2024-09-13T17:58:43.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.549+0000 7f81e516c640 1 --2- 172.21.15.111:0/3197549424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e0158b00 0x7f81e00c39d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.550+0000 7f81e516c640 1 --2- 172.21.15.111:0/3197549424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81e00b2170 0x7f81e00c3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.550+0000 7f81e516c640 1 --2- 172.21.15.111:0/3197549424 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f81cc06ac00 0x7f81cc06d0c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.550+0000 7f81e516c640 1 --2- 172.21.15.111:0/3197549424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81e006c6d0 0x7f81e01f1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.550+0000 7f81e516c640 1 -- 172.21.15.111:0/3197549424 >> 172.21.15.111:0/3197549424 conn(0x7f81e006a0b0 msgr2=0x7f81e00abbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:43.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.552+0000 7f81e516c640 1 -- 172.21.15.111:0/3197549424 shutdown_connections 2024-09-13T17:58:43.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.553+0000 7f81e516c640 1 -- 172.21.15.111:0/3197549424 wait complete. 2024-09-13T17:58:43.572 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:43.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.667+0000 7fd286533640 1 Processor -- start 2024-09-13T17:58:43.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.667+0000 7fd286533640 1 -- start start 2024-09-13T17:58:43.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.667+0000 7fd286533640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd28015a2e0 0x7fd2800c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.667+0000 7fd286533640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2800cb3f0 0x7fd28006d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.667+0000 7fd286533640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2800c8920 con 0x7fd2800c9f80 2024-09-13T17:58:43.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.667+0000 7fd286533640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd28015a6c0 con 0x7fd28015a2e0 2024-09-13T17:58:43.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.667+0000 7fd286533640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2800ca500 con 0x7fd2800cb3f0 2024-09-13T17:58:43.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.668+0000 7fd27f7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd2800c9f80 0x7fd2800cace0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58126/0 (socket says 172.21.15.111:58126) 2024-09-13T17:58:43.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.668+0000 7fd27f7fe640 1 -- 172.21.15.111:0/4159234632 learned_addr learned my addr 172.21.15.111:0/4159234632 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:43.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.668+0000 7fd27ffff640 1 --2- 172.21.15.111:0/4159234632 >> v2:172.21.15.188:3300/0 conn(0x7fd28015a2e0 0x7fd2800c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.668+0000 7fd284ad0640 1 --2- 172.21.15.111:0/4159234632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2800cb3f0 0x7fd28006d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.668+0000 7fd27effd640 1 -- 172.21.15.111:0/4159234632 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871546533 0 0) 0x7fd2800c8920 con 0x7fd2800c9f80 2024-09-13T17:58:43.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.668+0000 7fd27effd640 1 -- 172.21.15.111:0/4159234632 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd268003520 con 0x7fd2800c9f80 2024-09-13T17:58:43.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.668+0000 7fd284ad0640 1 -- 172.21.15.111:0/4159234632 >> v2:172.21.15.188:3300/0 conn(0x7fd28015a2e0 msgr2=0x7fd2800c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:43.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.668+0000 7fd284ad0640 1 --2- 172.21.15.111:0/4159234632 >> v2:172.21.15.188:3300/0 conn(0x7fd28015a2e0 0x7fd2800c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.668+0000 7fd284ad0640 1 -- 172.21.15.111:0/4159234632 >> v1:172.21.15.111:6789/0 conn(0x7fd2800c9f80 legacy=0x7fd2800cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.668+0000 7fd284ad0640 1 -- 172.21.15.111:0/4159234632 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2800c8b70 con 0x7fd2800cb3f0 2024-09-13T17:58:43.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.669+0000 7fd284ad0640 1 --2- 172.21.15.111:0/4159234632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2800cb3f0 0x7fd28006d030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd278009d90 tx=0x7fd27802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e072e6f8ad6ba5b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:43.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.669+0000 7fd27effd640 1 -- 172.21.15.111:0/4159234632 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278007740 con 0x7fd2800cb3f0 2024-09-13T17:58:43.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.669+0000 7fd27effd640 1 -- 172.21.15.111:0/4159234632 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27802d050 con 0x7fd2800cb3f0 2024-09-13T17:58:43.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.669+0000 7fd27effd640 1 -- 172.21.15.111:0/4159234632 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278007aa0 con 0x7fd2800cb3f0 2024-09-13T17:58:43.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.669+0000 7fd286533640 1 -- 172.21.15.111:0/4159234632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2800cb3f0 msgr2=0x7fd28006d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.669+0000 7fd286533640 1 --2- 172.21.15.111:0/4159234632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2800cb3f0 0x7fd28006d030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd278009d90 tx=0x7fd27802a660 comp rx=0 tx=0).stop 2024-09-13T17:58:43.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.670+0000 7fd286533640 1 -- 172.21.15.111:0/4159234632 shutdown_connections 2024-09-13T17:58:43.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.670+0000 7fd286533640 1 --2- 172.21.15.111:0/4159234632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2800cb3f0 0x7fd28006d030 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.670+0000 7fd286533640 1 --2- 172.21.15.111:0/4159234632 >> v2:172.21.15.188:3300/0 conn(0x7fd28015a2e0 0x7fd2800c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.670+0000 7fd286533640 1 -- 172.21.15.111:0/4159234632 >> 172.21.15.111:0/4159234632 conn(0x7fd280069ed0 msgr2=0x7fd28006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:43.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.670+0000 7fd286533640 1 -- 172.21.15.111:0/4159234632 shutdown_connections 2024-09-13T17:58:43.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.670+0000 7fd286533640 1 -- 172.21.15.111:0/4159234632 wait complete. 2024-09-13T17:58:43.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.671+0000 7fd286533640 1 Processor -- start 2024-09-13T17:58:43.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.671+0000 7fd286533640 1 -- start start 2024-09-13T17:58:43.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd286533640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2800c9f80 0x7fd2801f0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd286533640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2800cb3f0 0x7fd280154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd286533640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd28015a2e0 0x7fd2801f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd286533640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd280154900 con 0x7fd28015a2e0 2024-09-13T17:58:43.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd286533640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd280155760 con 0x7fd2800c9f80 2024-09-13T17:58:43.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd286533640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2801f0960 con 0x7fd2800cb3f0 2024-09-13T17:58:43.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd284ad0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd28015a2e0 0x7fd2801f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd284ad0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd28015a2e0 0x7fd2801f2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40142/0 (socket says 172.21.15.111:40142) 2024-09-13T17:58:43.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd284ad0640 1 -- 172.21.15.111:0/2522604311 learned_addr learned my addr 172.21.15.111:0/2522604311 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:43.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd27ffff640 1 --2- 172.21.15.111:0/2522604311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2800c9f80 0x7fd2801f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd27f7fe640 1 --2- 172.21.15.111:0/2522604311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2800cb3f0 0x7fd280154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd284ad0640 1 -- 172.21.15.111:0/2522604311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2800cb3f0 msgr2=0x7fd280154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd284ad0640 1 --2- 172.21.15.111:0/2522604311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2800cb3f0 0x7fd280154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd284ad0640 1 -- 172.21.15.111:0/2522604311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2800c9f80 msgr2=0x7fd2801f0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd284ad0640 1 --2- 172.21.15.111:0/2522604311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2800c9f80 0x7fd2801f0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd284ad0640 1 -- 172.21.15.111:0/2522604311 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd278009a70 con 0x7fd28015a2e0 2024-09-13T17:58:43.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd284ad0640 1 --2- 172.21.15.111:0/2522604311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd28015a2e0 0x7fd2801f2f90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd278009eb0 tx=0x7fd278004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:43.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd27cff9640 1 -- 172.21.15.111:0/2522604311 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27803c830 con 0x7fd28015a2e0 2024-09-13T17:58:43.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd27cff9640 1 -- 172.21.15.111:0/2522604311 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27803c9b0 con 0x7fd28015a2e0 2024-09-13T17:58:43.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd286533640 1 -- 172.21.15.111:0/2522604311 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd280154b50 con 0x7fd28015a2e0 2024-09-13T17:58:43.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.672+0000 7fd27cff9640 1 -- 172.21.15.111:0/2522604311 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27803cd60 con 0x7fd28015a2e0 2024-09-13T17:58:43.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.673+0000 7fd286533640 1 -- 172.21.15.111:0/2522604311 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd280155110 con 0x7fd28015a2e0 2024-09-13T17:58:43.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.673+0000 7fd286533640 1 -- 172.21.15.111:0/2522604311 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2801558d0 con 0x7fd28015a2e0 2024-09-13T17:58:43.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.674+0000 7fd27cff9640 1 -- 172.21.15.111:0/2522604311 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd278031400 con 0x7fd28015a2e0 2024-09-13T17:58:43.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.674+0000 7fd27cff9640 1 --2- 172.21.15.111:0/2522604311 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd26c06ac00 0x7fd26c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.675+0000 7fd27ffff640 1 --2- 172.21.15.111:0/2522604311 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd26c06ac00 0x7fd26c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.675+0000 7fd27cff9640 1 -- 172.21.15.111:0/2522604311 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd2780aab50 con 0x7fd28015a2e0 2024-09-13T17:58:43.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.675+0000 7fd27ffff640 1 --2- 172.21.15.111:0/2522604311 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd26c06ac00 0x7fd26c06d0c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd270009aa0 tx=0x7fd270009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:43.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.679+0000 7fd27cff9640 1 -- 172.21.15.111:0/2522604311 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fd278075c90 con 0x7fd28015a2e0 2024-09-13T17:58:43.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.842+0000 7fd286533640 1 -- 172.21.15.111:0/2522604311 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2800caf60 con 0x7fd28015a2e0 2024-09-13T17:58:43.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.843+0000 7fd27cff9640 1 -- 172.21.15.111:0/2522604311 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16847 (secure 0 0 0) 0x7fd2780792b0 con 0x7fd28015a2e0 2024-09-13T17:58:43.845 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:43.846 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":38,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:43.534241+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:58:41.698998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T17:58:42.499870+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:43.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.845+0000 7fd286533640 1 -- 172.21.15.111:0/2522604311 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd26c06ac00 msgr2=0x7fd26c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.845+0000 7fd286533640 1 --2- 172.21.15.111:0/2522604311 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd26c06ac00 0x7fd26c06d0c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd270009aa0 tx=0x7fd270009190 comp rx=0 tx=0).stop 2024-09-13T17:58:43.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.845+0000 7fd286533640 1 -- 172.21.15.111:0/2522604311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd28015a2e0 msgr2=0x7fd2801f2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.845+0000 7fd286533640 1 --2- 172.21.15.111:0/2522604311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd28015a2e0 0x7fd2801f2f90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd278009eb0 tx=0x7fd278004a40 comp rx=0 tx=0).stop 2024-09-13T17:58:43.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.845+0000 7fd286533640 1 -- 172.21.15.111:0/2522604311 shutdown_connections 2024-09-13T17:58:43.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.845+0000 7fd286533640 1 --2- 172.21.15.111:0/2522604311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd28015a2e0 0x7fd2801f2f90 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.845+0000 7fd286533640 1 --2- 172.21.15.111:0/2522604311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2800cb3f0 0x7fd280154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.845+0000 7fd286533640 1 --2- 172.21.15.111:0/2522604311 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd26c06ac00 0x7fd26c06d0c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.845+0000 7fd286533640 1 --2- 172.21.15.111:0/2522604311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2800c9f80 0x7fd2801f0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.845+0000 7fd286533640 1 -- 172.21.15.111:0/2522604311 >> 172.21.15.111:0/2522604311 conn(0x7fd280069ed0 msgr2=0x7fd28006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:43.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.845+0000 7fd286533640 1 -- 172.21.15.111:0/2522604311 shutdown_connections 2024-09-13T17:58:43.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.845+0000 7fd286533640 1 -- 172.21.15.111:0/2522604311 wait complete. 2024-09-13T17:58:43.858 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:43.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.954+0000 7fb9b9bae640 1 Processor -- start 2024-09-13T17:58:43.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.954+0000 7fb9b9bae640 1 -- start start 2024-09-13T17:58:43.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.955+0000 7fb9b9bae640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9b415a2e0 0x7fb9b40c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.955+0000 7fb9b9bae640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb9b40cb3f0 0x7fb9b406d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.955+0000 7fb9b9bae640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b40c8920 con 0x7fb9b40c9f80 2024-09-13T17:58:43.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.955+0000 7fb9b9bae640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b415a6c0 con 0x7fb9b40cb3f0 2024-09-13T17:58:43.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.955+0000 7fb9b9bae640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b40ca500 con 0x7fb9b415a2e0 2024-09-13T17:58:43.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.955+0000 7fb9b3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb9b40cb3f0 0x7fb9b406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.955+0000 7fb9b3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb9b40cb3f0 0x7fb9b406d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40154/0 (socket says 172.21.15.111:40154) 2024-09-13T17:58:43.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.955+0000 7fb9b3fff640 1 -- 172.21.15.111:0/3456684891 learned_addr learned my addr 172.21.15.111:0/3456684891 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:43.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.955+0000 7fb9b37fe640 1 --2- 172.21.15.111:0/3456684891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9b415a2e0 0x7fb9b40c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.955+0000 7fb9b3fff640 1 -- 172.21.15.111:0/3456684891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9b415a2e0 msgr2=0x7fb9b40c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:43.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.955+0000 7fb9b3fff640 1 --2- 172.21.15.111:0/3456684891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9b415a2e0 0x7fb9b40c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.955+0000 7fb9b3fff640 1 -- 172.21.15.111:0/3456684891 >> v1:172.21.15.188:6789/0 conn(0x7fb9b40c9f80 legacy=0x7fb9b40cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:43.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.955+0000 7fb9b3fff640 1 -- 172.21.15.111:0/3456684891 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b40c8b70 con 0x7fb9b40cb3f0 2024-09-13T17:58:43.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.955+0000 7fb9b37fe640 1 --2- 172.21.15.111:0/3456684891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9b415a2e0 0x7fb9b40c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:43.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.956+0000 7fb9b3fff640 1 --2- 172.21.15.111:0/3456684891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b40cb3f0 0x7fb9b406d030 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009870 tx=0x7fb9ac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caaa197ba038f092 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:43.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.956+0000 7fb9b27fc640 1 -- 172.21.15.111:0/3456684891 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac032440 con 0x7fb9b40cb3f0 2024-09-13T17:58:43.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.956+0000 7fb9b27fc640 1 -- 172.21.15.111:0/3456684891 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ac0325a0 con 0x7fb9b40cb3f0 2024-09-13T17:58:43.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.956+0000 7fb9b27fc640 1 -- 172.21.15.111:0/3456684891 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac032930 con 0x7fb9b40cb3f0 2024-09-13T17:58:43.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.957+0000 7fb9b9bae640 1 -- 172.21.15.111:0/3456684891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b40cb3f0 msgr2=0x7fb9b406d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.957+0000 7fb9b9bae640 1 --2- 172.21.15.111:0/3456684891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b40cb3f0 0x7fb9b406d030 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009870 tx=0x7fb9ac02a210 comp rx=0 tx=0).stop 2024-09-13T17:58:43.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.957+0000 7fb9b9bae640 1 -- 172.21.15.111:0/3456684891 shutdown_connections 2024-09-13T17:58:43.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.957+0000 7fb9b9bae640 1 --2- 172.21.15.111:0/3456684891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b40cb3f0 0x7fb9b406d030 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.957+0000 7fb9b9bae640 1 --2- 172.21.15.111:0/3456684891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9b415a2e0 0x7fb9b40c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.957+0000 7fb9b9bae640 1 -- 172.21.15.111:0/3456684891 >> 172.21.15.111:0/3456684891 conn(0x7fb9b4069ed0 msgr2=0x7fb9b406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:43.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.957+0000 7fb9b9bae640 1 -- 172.21.15.111:0/3456684891 shutdown_connections 2024-09-13T17:58:43.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.957+0000 7fb9b9bae640 1 -- 172.21.15.111:0/3456684891 wait complete. 2024-09-13T17:58:43.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.958+0000 7fb9b9bae640 1 Processor -- start 2024-09-13T17:58:43.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.958+0000 7fb9b9bae640 1 -- start start 2024-09-13T17:58:43.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb9b9bae640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b40c9f80 0x7fb9b4158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb9b9bae640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9b40cb3f0 0x7fb9b415a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb9b9bae640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9b415a2e0 0x7fb9b40c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb9b9bae640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b415c400 con 0x7fb9b40c9f80 2024-09-13T17:58:43.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb9b9bae640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b415b360 con 0x7fb9b415a2e0 2024-09-13T17:58:43.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb9b9bae640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b415b640 con 0x7fb9b40cb3f0 2024-09-13T17:58:43.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb9b37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b40c9f80 0x7fb9b4158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb9b37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b40c9f80 0x7fb9b4158b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40170/0 (socket says 172.21.15.111:40170) 2024-09-13T17:58:43.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb9b37fe640 1 -- 172.21.15.111:0/1125157620 learned_addr learned my addr 172.21.15.111:0/1125157620 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:43.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb9b3fff640 1 --2- 172.21.15.111:0/1125157620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9b415a2e0 0x7fb9b40c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb9b2ffd640 1 --2- 172.21.15.111:0/1125157620 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9b40cb3f0 0x7fb9b415a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb9b37fe640 1 -- 172.21.15.111:0/1125157620 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9b40cb3f0 msgr2=0x7fb9b415a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb9b37fe640 1 --2- 172.21.15.111:0/1125157620 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9b40cb3f0 0x7fb9b415a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb9b37fe640 1 -- 172.21.15.111:0/1125157620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9b415a2e0 msgr2=0x7fb9b40c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb9b37fe640 1 --2- 172.21.15.111:0/1125157620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9b415a2e0 0x7fb9b40c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb9b37fe640 1 -- 172.21.15.111:0/1125157620 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac0094d0 con 0x7fb9b40c9f80 2024-09-13T17:58:43.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb9b37fe640 1 --2- 172.21.15.111:0/1125157620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b40c9f80 0x7fb9b4158b00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400ce00 tx=0x7fb9a400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:43.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb99bfff640 1 -- 172.21.15.111:0/1125157620 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a400f890 con 0x7fb9b40c9f80 2024-09-13T17:58:43.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.959+0000 7fb99bfff640 1 -- 172.21.15.111:0/1125157620 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a400f9f0 con 0x7fb9b40c9f80 2024-09-13T17:58:43.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.960+0000 7fb99bfff640 1 -- 172.21.15.111:0/1125157620 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a400fdc0 con 0x7fb9b40c9f80 2024-09-13T17:58:43.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.960+0000 7fb9b9bae640 1 -- 172.21.15.111:0/1125157620 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b415b920 con 0x7fb9b40c9f80 2024-09-13T17:58:43.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.960+0000 7fb9b9bae640 1 -- 172.21.15.111:0/1125157620 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b415bee0 con 0x7fb9b40c9f80 2024-09-13T17:58:43.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.961+0000 7fb9b9bae640 1 -- 172.21.15.111:0/1125157620 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b40ca8b0 con 0x7fb9b40c9f80 2024-09-13T17:58:43.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.961+0000 7fb99bfff640 1 -- 172.21.15.111:0/1125157620 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb9a400b040 con 0x7fb9b40c9f80 2024-09-13T17:58:43.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.961+0000 7fb99bfff640 1 --2- 172.21.15.111:0/1125157620 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb9a006ac00 0x7fb9a006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.961+0000 7fb99bfff640 1 -- 172.21.15.111:0/1125157620 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb9a4089e70 con 0x7fb9b40c9f80 2024-09-13T17:58:43.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.961+0000 7fb9b2ffd640 1 --2- 172.21.15.111:0/1125157620 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb9a006ac00 0x7fb9a006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.962+0000 7fb9b2ffd640 1 --2- 172.21.15.111:0/1125157620 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb9a006ac00 0x7fb9a006d0c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb9b406a7f0 tx=0x7fb9a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:43.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:43.966+0000 7fb99bfff640 1 -- 172.21.15.111:0/1125157620 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fb9a4054f10 con 0x7fb9b40c9f80 2024-09-13T17:58:44.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.127+0000 7fb9b9bae640 1 -- 172.21.15.111:0/1125157620 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9b40cdc40 con 0x7fb9b40c9f80 2024-09-13T17:58:44.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.128+0000 7fb99bfff640 1 -- 172.21.15.111:0/1125157620 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16847 (secure 0 0 0) 0x7fb9a401d070 con 0x7fb9b40c9f80 2024-09-13T17:58:44.130 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:44.131 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":38,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:43.534241+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:58:41.698998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T17:58:42.499870+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:44.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.130+0000 7fb9b9bae640 1 -- 172.21.15.111:0/1125157620 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb9a006ac00 msgr2=0x7fb9a006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:44.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.130+0000 7fb9b9bae640 1 --2- 172.21.15.111:0/1125157620 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb9a006ac00 0x7fb9a006d0c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb9b406a7f0 tx=0x7fb9a8009040 comp rx=0 tx=0).stop 2024-09-13T17:58:44.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.130+0000 7fb9b9bae640 1 -- 172.21.15.111:0/1125157620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b40c9f80 msgr2=0x7fb9b4158b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:44.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.130+0000 7fb9b9bae640 1 --2- 172.21.15.111:0/1125157620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b40c9f80 0x7fb9b4158b00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400ce00 tx=0x7fb9a400a650 comp rx=0 tx=0).stop 2024-09-13T17:58:44.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.130+0000 7fb9b9bae640 1 -- 172.21.15.111:0/1125157620 shutdown_connections 2024-09-13T17:58:44.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.130+0000 7fb9b9bae640 1 --2- 172.21.15.111:0/1125157620 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fb9a006ac00 0x7fb9a006d0c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.130+0000 7fb9b9bae640 1 --2- 172.21.15.111:0/1125157620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9b415a2e0 0x7fb9b40c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.130+0000 7fb9b9bae640 1 --2- 172.21.15.111:0/1125157620 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9b40cb3f0 0x7fb9b415a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.130+0000 7fb9b9bae640 1 --2- 172.21.15.111:0/1125157620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b40c9f80 0x7fb9b4158b00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.130+0000 7fb9b9bae640 1 -- 172.21.15.111:0/1125157620 >> 172.21.15.111:0/1125157620 conn(0x7fb9b4069ed0 msgr2=0x7fb9b40cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:44.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.130+0000 7fb9b9bae640 1 -- 172.21.15.111:0/1125157620 shutdown_connections 2024-09-13T17:58:44.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.130+0000 7fb9b9bae640 1 -- 172.21.15.111:0/1125157620 wait complete. 2024-09-13T17:58:44.143 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T17:58:44.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.243+0000 7f4c61d45640 1 Processor -- start 2024-09-13T17:58:44.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.243+0000 7f4c61d45640 1 -- start start 2024-09-13T17:58:44.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.244+0000 7f4c61d45640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5c06c8a0 0x7f4c5c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:44.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.244+0000 7f4c61d45640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4c5c071b20 0x7f4c5c0b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:44.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.244+0000 7f4c61d45640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c5c0b2d10 con 0x7f4c5c158b00 2024-09-13T17:58:44.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.244+0000 7f4c61d45640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c5c0b2ff0 con 0x7f4c5c071b20 2024-09-13T17:58:44.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.244+0000 7f4c61d45640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c070310 con 0x7f4c5c06c8a0 2024-09-13T17:58:44.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.244+0000 7f4c5bfff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4c5c071b20 0x7f4c5c0b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:44.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.244+0000 7f4c5bfff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4c5c071b20 0x7f4c5c0b16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40182/0 (socket says 172.21.15.111:40182) 2024-09-13T17:58:44.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.244+0000 7f4c5bfff640 1 -- 172.21.15.111:0/2887525418 learned_addr learned my addr 172.21.15.111:0/2887525418 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:44.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.244+0000 7f4c53fff640 1 --2- 172.21.15.111:0/2887525418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5c06c8a0 0x7f4c5c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:44.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.244+0000 7f4c5bfff640 1 -- 172.21.15.111:0/2887525418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5c06c8a0 msgr2=0x7f4c5c0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:44.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.244+0000 7f4c5bfff640 1 --2- 172.21.15.111:0/2887525418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5c06c8a0 0x7f4c5c0b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.244+0000 7f4c5bfff640 1 -- 172.21.15.111:0/2887525418 >> v1:172.21.15.188:6789/0 conn(0x7f4c5c158b00 legacy=0x7f4c5c06c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:44.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.244+0000 7f4c5bfff640 1 -- 172.21.15.111:0/2887525418 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5c0705f0 con 0x7f4c5c071b20 2024-09-13T17:58:44.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.244+0000 7f4c53fff640 1 --2- 172.21.15.111:0/2887525418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5c06c8a0 0x7f4c5c0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:44.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.245+0000 7f4c5bfff640 1 --2- 172.21.15.111:0/2887525418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c5c071b20 0x7f4c5c0b16a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009e50 tx=0x7f4c5402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee661f442b889c54 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:44.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.245+0000 7f4c5affd640 1 -- 172.21.15.111:0/2887525418 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c54031600 con 0x7f4c5c071b20 2024-09-13T17:58:44.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.245+0000 7f4c5affd640 1 -- 172.21.15.111:0/2887525418 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5402d050 con 0x7f4c5c071b20 2024-09-13T17:58:44.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.245+0000 7f4c5affd640 1 -- 172.21.15.111:0/2887525418 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c54031980 con 0x7f4c5c071b20 2024-09-13T17:58:44.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.245+0000 7f4c61d45640 1 -- 172.21.15.111:0/2887525418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c5c071b20 msgr2=0x7f4c5c0b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:44.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.245+0000 7f4c61d45640 1 --2- 172.21.15.111:0/2887525418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c5c071b20 0x7f4c5c0b16a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009e50 tx=0x7f4c5402a6d0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.245+0000 7f4c61d45640 1 -- 172.21.15.111:0/2887525418 shutdown_connections 2024-09-13T17:58:44.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.245+0000 7f4c61d45640 1 --2- 172.21.15.111:0/2887525418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c5c071b20 0x7f4c5c0b16a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.245+0000 7f4c61d45640 1 --2- 172.21.15.111:0/2887525418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5c06c8a0 0x7f4c5c0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.245+0000 7f4c61d45640 1 -- 172.21.15.111:0/2887525418 >> 172.21.15.111:0/2887525418 conn(0x7f4c5c06a0b0 msgr2=0x7f4c5c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:44.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.246+0000 7f4c61d45640 1 -- 172.21.15.111:0/2887525418 shutdown_connections 2024-09-13T17:58:44.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.246+0000 7f4c61d45640 1 -- 172.21.15.111:0/2887525418 wait complete. 2024-09-13T17:58:44.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.247+0000 7f4c61d45640 1 Processor -- start 2024-09-13T17:58:44.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.247+0000 7f4c61d45640 1 -- start start 2024-09-13T17:58:44.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.247+0000 7f4c61d45640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5c06c8a0 0x7f4c5c1f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:44.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.247+0000 7f4c61d45640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c5c071b20 0x7f4c5c1f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:44.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.247+0000 7f4c61d45640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c5c158b00 0x7f4c5c1f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:44.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.247+0000 7f4c61d45640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c148f80 con 0x7f4c5c158b00 2024-09-13T17:58:44.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.247+0000 7f4c61d45640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c0caed0 con 0x7f4c5c071b20 2024-09-13T17:58:44.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.247+0000 7f4c61d45640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c0cb1b0 con 0x7f4c5c06c8a0 2024-09-13T17:58:44.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.247+0000 7f4c5bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c5c158b00 0x7f4c5c1f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:44.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.247+0000 7f4c5bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c5c158b00 0x7f4c5c1f6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40186/0 (socket says 172.21.15.111:40186) 2024-09-13T17:58:44.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.247+0000 7f4c5bfff640 1 -- 172.21.15.111:0/1700413434 learned_addr learned my addr 172.21.15.111:0/1700413434 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:44.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.248+0000 7f4c5b7fe640 1 --2- 172.21.15.111:0/1700413434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5c06c8a0 0x7f4c5c1f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:44.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.248+0000 7f4c53fff640 1 --2- 172.21.15.111:0/1700413434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c5c071b20 0x7f4c5c1f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:44.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.248+0000 7f4c5bfff640 1 -- 172.21.15.111:0/1700413434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5c06c8a0 msgr2=0x7f4c5c1f1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:44.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.248+0000 7f4c5bfff640 1 --2- 172.21.15.111:0/1700413434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5c06c8a0 0x7f4c5c1f1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.248+0000 7f4c5bfff640 1 -- 172.21.15.111:0/1700413434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c5c071b20 msgr2=0x7f4c5c1f5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:44.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.248+0000 7f4c5bfff640 1 --2- 172.21.15.111:0/1700413434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c5c071b20 0x7f4c5c1f5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.248+0000 7f4c5bfff640 1 -- 172.21.15.111:0/1700413434 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c54009b30 con 0x7f4c5c158b00 2024-09-13T17:58:44.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.248+0000 7f4c5bfff640 1 --2- 172.21.15.111:0/1700413434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c5c158b00 0x7f4c5c1f6000 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4c54034040 tx=0x7f4c54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:44.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.248+0000 7f4c58ff9640 1 -- 172.21.15.111:0/1700413434 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5403d860 con 0x7f4c5c158b00 2024-09-13T17:58:44.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.248+0000 7f4c58ff9640 1 -- 172.21.15.111:0/1700413434 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5403d9c0 con 0x7f4c5c158b00 2024-09-13T17:58:44.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.248+0000 7f4c58ff9640 1 -- 172.21.15.111:0/1700413434 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5403dd70 con 0x7f4c5c158b00 2024-09-13T17:58:44.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.248+0000 7f4c61d45640 1 -- 172.21.15.111:0/1700413434 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5c148930 con 0x7f4c5c158b00 2024-09-13T17:58:44.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.248+0000 7f4c61d45640 1 -- 172.21.15.111:0/1700413434 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5c1f2200 con 0x7f4c5c158b00 2024-09-13T17:58:44.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.249+0000 7f4c61d45640 1 -- 172.21.15.111:0/1700413434 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5c149130 con 0x7f4c5c158b00 2024-09-13T17:58:44.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.249+0000 7f4c58ff9640 1 -- 172.21.15.111:0/1700413434 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4c5403b400 con 0x7f4c5c158b00 2024-09-13T17:58:44.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.250+0000 7f4c58ff9640 1 --2- 172.21.15.111:0/1700413434 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4c3c06ac30 0x7f4c3c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:44.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.250+0000 7f4c58ff9640 1 -- 172.21.15.111:0/1700413434 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4c540aa4b0 con 0x7f4c5c158b00 2024-09-13T17:58:44.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.250+0000 7f4c5b7fe640 1 --2- 172.21.15.111:0/1700413434 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4c3c06ac30 0x7f4c3c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:44.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.250+0000 7f4c5b7fe640 1 --2- 172.21.15.111:0/1700413434 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4c3c06ac30 0x7f4c3c06d0f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c006fd0 tx=0x7f4c4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:44.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.254+0000 7f4c58ff9640 1 -- 172.21.15.111:0/1700413434 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4c5c149130 con 0x7f4c5c158b00 2024-09-13T17:58:44.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.450+0000 7f4c61d45640 1 -- 172.21.15.111:0/1700413434 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4c5c0cea20 con 0x7f4c5c158b00 2024-09-13T17:58:44.540 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.538+0000 7f4c58ff9640 1 -- 172.21.15.111:0/1700413434 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4c54075550 con 0x7f4c5c158b00 2024-09-13T17:58:44.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.540+0000 7f4c61d45640 1 -- 172.21.15.111:0/1700413434 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4c3c06ac30 msgr2=0x7f4c3c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:44.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.540+0000 7f4c61d45640 1 --2- 172.21.15.111:0/1700413434 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4c3c06ac30 0x7f4c3c06d0f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c006fd0 tx=0x7f4c4c008040 comp rx=0 tx=0).stop 2024-09-13T17:58:44.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.540+0000 7f4c61d45640 1 -- 172.21.15.111:0/1700413434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c5c158b00 msgr2=0x7f4c5c1f6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:44.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.540+0000 7f4c61d45640 1 --2- 172.21.15.111:0/1700413434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c5c158b00 0x7f4c5c1f6000 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4c54034040 tx=0x7f4c54004a40 comp rx=0 tx=0).stop 2024-09-13T17:58:44.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.540+0000 7f4c61d45640 1 -- 172.21.15.111:0/1700413434 shutdown_connections 2024-09-13T17:58:44.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.540+0000 7f4c61d45640 1 --2- 172.21.15.111:0/1700413434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c5c158b00 0x7f4c5c1f6000 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.540+0000 7f4c61d45640 1 --2- 172.21.15.111:0/1700413434 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4c3c06ac30 0x7f4c3c06d0f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.540+0000 7f4c61d45640 1 --2- 172.21.15.111:0/1700413434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c5c071b20 0x7f4c5c1f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.540+0000 7f4c61d45640 1 --2- 172.21.15.111:0/1700413434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5c06c8a0 0x7f4c5c1f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.540+0000 7f4c61d45640 1 -- 172.21.15.111:0/1700413434 >> 172.21.15.111:0/1700413434 conn(0x7f4c5c06a0b0 msgr2=0x7f4c5c070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:44.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.540+0000 7f4c61d45640 1 -- 172.21.15.111:0/1700413434 shutdown_connections 2024-09-13T17:58:44.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.540+0000 7f4c61d45640 1 -- 172.21.15.111:0/1700413434 wait complete. 2024-09-13T17:58:44.553 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T17:58:44.554 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T17:58:44.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.652+0000 7f647445a640 1 Processor -- start 2024-09-13T17:58:44.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.652+0000 7f647445a640 1 -- start start 2024-09-13T17:58:44.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.653+0000 7f647445a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f646c0b2b40 0x7f646c0ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:44.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.653+0000 7f647445a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f646c0ce4f0 0x7f646c0ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:44.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.653+0000 7f647445a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f646c0ccbe0 con 0x7f646c159310 2024-09-13T17:58:44.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.653+0000 7f647445a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f646c0ad940 con 0x7f646c0ce4f0 2024-09-13T17:58:44.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.653+0000 7f647445a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646c0ceea0 con 0x7f646c0b2b40 2024-09-13T17:58:44.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.653+0000 7f64721f6640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f646c159310 0x7f646c0ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58136/0 (socket says 172.21.15.111:58136) 2024-09-13T17:58:44.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.653+0000 7f64721f6640 1 -- 172.21.15.111:0/2381353535 learned_addr learned my addr 172.21.15.111:0/2381353535 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:44.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.653+0000 7f64729f7640 1 --2- 172.21.15.111:0/2381353535 >> v2:172.21.15.111:3300/0 conn(0x7f646c0ce4f0 0x7f646c0ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:44.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.653+0000 7f64711f4640 1 -- 172.21.15.111:0/2381353535 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612609033 0 0) 0x7f646c0ccbe0 con 0x7f646c159310 2024-09-13T17:58:44.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.653+0000 7f64711f4640 1 -- 172.21.15.111:0/2381353535 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6458003520 con 0x7f646c159310 2024-09-13T17:58:44.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.653+0000 7f64729f7640 1 -- 172.21.15.111:0/2381353535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f646c0b2b40 msgr2=0x7f646c0ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:44.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.653+0000 7f64729f7640 1 --2- 172.21.15.111:0/2381353535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f646c0b2b40 0x7f646c0ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.653+0000 7f64729f7640 1 -- 172.21.15.111:0/2381353535 >> v1:172.21.15.111:6789/0 conn(0x7f646c159310 legacy=0x7f646c0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:44.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.653+0000 7f64729f7640 1 -- 172.21.15.111:0/2381353535 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646c0b1810 con 0x7f646c0ce4f0 2024-09-13T17:58:44.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.654+0000 7f64729f7640 1 --2- 172.21.15.111:0/2381353535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f646c0ce4f0 0x7f646c0ce960 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6464009dc0 tx=0x7f646402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c507df44fb8a338a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:44.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.654+0000 7f64711f4640 1 -- 172.21.15.111:0/2381353535 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6464007740 con 0x7f646c0ce4f0 2024-09-13T17:58:44.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.654+0000 7f64711f4640 1 -- 172.21.15.111:0/2381353535 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646402d050 con 0x7f646c0ce4f0 2024-09-13T17:58:44.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.654+0000 7f64711f4640 1 -- 172.21.15.111:0/2381353535 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6464007ac0 con 0x7f646c0ce4f0 2024-09-13T17:58:44.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.655+0000 7f647445a640 1 -- 172.21.15.111:0/2381353535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f646c0ce4f0 msgr2=0x7f646c0ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:44.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.655+0000 7f647445a640 1 --2- 172.21.15.111:0/2381353535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f646c0ce4f0 0x7f646c0ce960 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6464009dc0 tx=0x7f646402a560 comp rx=0 tx=0).stop 2024-09-13T17:58:44.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.655+0000 7f647445a640 1 -- 172.21.15.111:0/2381353535 shutdown_connections 2024-09-13T17:58:44.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.655+0000 7f647445a640 1 --2- 172.21.15.111:0/2381353535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f646c0ce4f0 0x7f646c0ce960 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.655+0000 7f647445a640 1 --2- 172.21.15.111:0/2381353535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f646c0b2b40 0x7f646c0ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.655+0000 7f647445a640 1 -- 172.21.15.111:0/2381353535 >> 172.21.15.111:0/2381353535 conn(0x7f646c06a0b0 msgr2=0x7f646c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:44.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.655+0000 7f647445a640 1 -- 172.21.15.111:0/2381353535 shutdown_connections 2024-09-13T17:58:44.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.655+0000 7f647445a640 1 -- 172.21.15.111:0/2381353535 wait complete. 2024-09-13T17:58:44.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.656+0000 7f647445a640 1 Processor -- start 2024-09-13T17:58:44.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.656+0000 7f647445a640 1 -- start start 2024-09-13T17:58:44.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f647445a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f646c0b2b40 0x7f646c156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:44.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f647445a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f646c0ce4f0 0x7f646c0c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:44.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f647445a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f646c159310 0x7f646c0c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:44.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f647445a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c0c6130 con 0x7f646c159310 2024-09-13T17:58:44.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f647445a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c0c6410 con 0x7f646c0ce4f0 2024-09-13T17:58:44.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f647445a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646c0c66f0 con 0x7f646c0b2b40 2024-09-13T17:58:44.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f64729f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f646c159310 0x7f646c0c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:44.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f64729f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f646c159310 0x7f646c0c8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40204/0 (socket says 172.21.15.111:40204) 2024-09-13T17:58:44.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f64729f7640 1 -- 172.21.15.111:0/1685171393 learned_addr learned my addr 172.21.15.111:0/1685171393 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:44.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f64721f6640 1 --2- 172.21.15.111:0/1685171393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f646c0b2b40 0x7f646c156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:44.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f64729f7640 1 -- 172.21.15.111:0/1685171393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f646c0b2b40 msgr2=0x7f646c156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:44.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f64719f5640 1 --2- 172.21.15.111:0/1685171393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f646c0ce4f0 0x7f646c0c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:44.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f64729f7640 1 --2- 172.21.15.111:0/1685171393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f646c0b2b40 0x7f646c156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f64729f7640 1 -- 172.21.15.111:0/1685171393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f646c0ce4f0 msgr2=0x7f646c0c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:44.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f64729f7640 1 --2- 172.21.15.111:0/1685171393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f646c0ce4f0 0x7f646c0c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:44.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f64729f7640 1 -- 172.21.15.111:0/1685171393 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6464009a70 con 0x7f646c159310 2024-09-13T17:58:44.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f64721f6640 1 --2- 172.21.15.111:0/1685171393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f646c0b2b40 0x7f646c156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:44.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f64729f7640 1 --2- 172.21.15.111:0/1685171393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f646c159310 0x7f646c0c8c80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6464009ee0 tx=0x7f6464004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:44.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f6462ffd640 1 -- 172.21.15.111:0/1685171393 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646403c830 con 0x7f646c159310 2024-09-13T17:58:44.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f6462ffd640 1 -- 172.21.15.111:0/1685171393 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646403c990 con 0x7f646c159310 2024-09-13T17:58:44.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f647445a640 1 -- 172.21.15.111:0/1685171393 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646c0c69d0 con 0x7f646c159310 2024-09-13T17:58:44.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.657+0000 7f6462ffd640 1 -- 172.21.15.111:0/1685171393 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646403cd00 con 0x7f646c159310 2024-09-13T17:58:44.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.658+0000 7f647445a640 1 -- 172.21.15.111:0/1685171393 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646c1569b0 con 0x7f646c159310 2024-09-13T17:58:44.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.659+0000 7f647445a640 1 -- 172.21.15.111:0/1685171393 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646c0c6ef0 con 0x7f646c159310 2024-09-13T17:58:44.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.659+0000 7f6462ffd640 1 -- 172.21.15.111:0/1685171393 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6464031400 con 0x7f646c159310 2024-09-13T17:58:44.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.659+0000 7f6462ffd640 1 --2- 172.21.15.111:0/1685171393 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f645006ac00 0x7f645006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:44.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.660+0000 7f6462ffd640 1 -- 172.21.15.111:0/1685171393 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f64640ab0c0 con 0x7f646c159310 2024-09-13T17:58:44.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.660+0000 7f64721f6640 1 --2- 172.21.15.111:0/1685171393 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f645006ac00 0x7f645006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:44.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.660+0000 7f64721f6640 1 --2- 172.21.15.111:0/1685171393 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f645006ac00 0x7f645006d0c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6468007fd0 tx=0x7f6468009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:44.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.664+0000 7f6462ffd640 1 -- 172.21.15.111:0/1685171393 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f646c0c6ef0 con 0x7f646c159310 2024-09-13T17:58:44.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:44.860+0000 7f647445a640 1 -- 172.21.15.111:0/1685171393 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f646c0c6ef0 con 0x7f646c159310 2024-09-13T17:58:45.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.546+0000 7f6462ffd640 1 -- 172.21.15.111:0/1685171393 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f646c0c6ef0 con 0x7f646c159310 2024-09-13T17:58:45.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.548+0000 7f647445a640 1 -- 172.21.15.111:0/1685171393 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f645006ac00 msgr2=0x7f645006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:45.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.548+0000 7f647445a640 1 --2- 172.21.15.111:0/1685171393 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f645006ac00 0x7f645006d0c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6468007fd0 tx=0x7f6468009040 comp rx=0 tx=0).stop 2024-09-13T17:58:45.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.548+0000 7f647445a640 1 -- 172.21.15.111:0/1685171393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f646c159310 msgr2=0x7f646c0c8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:45.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.548+0000 7f647445a640 1 --2- 172.21.15.111:0/1685171393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f646c159310 0x7f646c0c8c80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6464009ee0 tx=0x7f6464004a40 comp rx=0 tx=0).stop 2024-09-13T17:58:45.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.548+0000 7f647445a640 1 -- 172.21.15.111:0/1685171393 shutdown_connections 2024-09-13T17:58:45.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.548+0000 7f647445a640 1 --2- 172.21.15.111:0/1685171393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f646c159310 0x7f646c0c8c80 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.548+0000 7f647445a640 1 --2- 172.21.15.111:0/1685171393 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f645006ac00 0x7f645006d0c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.548+0000 7f647445a640 1 --2- 172.21.15.111:0/1685171393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f646c0ce4f0 0x7f646c0c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.548+0000 7f647445a640 1 --2- 172.21.15.111:0/1685171393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f646c0b2b40 0x7f646c156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.548+0000 7f647445a640 1 -- 172.21.15.111:0/1685171393 >> 172.21.15.111:0/1685171393 conn(0x7f646c06a0b0 msgr2=0x7f646c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:45.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.548+0000 7f647445a640 1 -- 172.21.15.111:0/1685171393 shutdown_connections 2024-09-13T17:58:45.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.548+0000 7f647445a640 1 -- 172.21.15.111:0/1685171393 wait complete. 2024-09-13T17:58:45.562 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T17:58:45.563 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:58:45.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.661+0000 7fe780a01640 1 Processor -- start 2024-09-13T17:58:45.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.661+0000 7fe780a01640 1 -- start start 2024-09-13T17:58:45.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.662+0000 7fe780a01640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe77c06bf00 0x7fe77c13efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:45.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.662+0000 7fe780a01640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe77c15ca10 0x7fe77c06c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:45.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.662+0000 7fe780a01640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe77c1593f0 con 0x7fe77c152f40 2024-09-13T17:58:45.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.662+0000 7fe780a01640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe77c0cae30 con 0x7fe77c06bf00 2024-09-13T17:58:45.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.662+0000 7fe780a01640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe77c13f520 con 0x7fe77c15ca10 2024-09-13T17:58:45.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.662+0000 7fe77ad9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe77c06bf00 0x7fe77c13efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:45.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.662+0000 7fe77ad9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe77c06bf00 0x7fe77c13efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40220/0 (socket says 172.21.15.111:40220) 2024-09-13T17:58:45.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.662+0000 7fe77ad9d640 1 -- 172.21.15.111:0/3160411291 learned_addr learned my addr 172.21.15.111:0/3160411291 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:45.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.662+0000 7fe77ad9d640 1 -- 172.21.15.111:0/3160411291 >> v2:172.21.15.188:3300/0 conn(0x7fe77c15ca10 msgr2=0x7fe77c06c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:45.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.662+0000 7fe77ad9d640 1 --2- 172.21.15.111:0/3160411291 >> v2:172.21.15.188:3300/0 conn(0x7fe77c15ca10 0x7fe77c06c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.662+0000 7fe77ad9d640 1 -- 172.21.15.111:0/3160411291 >> v1:172.21.15.188:6789/0 conn(0x7fe77c152f40 legacy=0x7fe77c1533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:45.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.662+0000 7fe77ad9d640 1 -- 172.21.15.111:0/3160411291 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c06c820 con 0x7fe77c06bf00 2024-09-13T17:58:45.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.663+0000 7fe77ad9d640 1 --2- 172.21.15.111:0/3160411291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe77c06bf00 0x7fe77c13efe0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe77c13c480 tx=0x7fe76c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5b8c6689549c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:45.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.663+0000 7fe779d9b640 1 -- 172.21.15.111:0/3160411291 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c032440 con 0x7fe77c06bf00 2024-09-13T17:58:45.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.663+0000 7fe779d9b640 1 -- 172.21.15.111:0/3160411291 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76c0325a0 con 0x7fe77c06bf00 2024-09-13T17:58:45.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.663+0000 7fe779d9b640 1 -- 172.21.15.111:0/3160411291 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c032930 con 0x7fe77c06bf00 2024-09-13T17:58:45.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.664+0000 7fe780a01640 1 -- 172.21.15.111:0/3160411291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe77c06bf00 msgr2=0x7fe77c13efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:45.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.664+0000 7fe780a01640 1 --2- 172.21.15.111:0/3160411291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe77c06bf00 0x7fe77c13efe0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe77c13c480 tx=0x7fe76c02a1a0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.664+0000 7fe780a01640 1 -- 172.21.15.111:0/3160411291 shutdown_connections 2024-09-13T17:58:45.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.664+0000 7fe780a01640 1 --2- 172.21.15.111:0/3160411291 >> v2:172.21.15.188:3300/0 conn(0x7fe77c15ca10 0x7fe77c06c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.664+0000 7fe780a01640 1 --2- 172.21.15.111:0/3160411291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe77c06bf00 0x7fe77c13efe0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.664+0000 7fe780a01640 1 -- 172.21.15.111:0/3160411291 >> 172.21.15.111:0/3160411291 conn(0x7fe77c06d860 msgr2=0x7fe77c069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:45.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.664+0000 7fe780a01640 1 -- 172.21.15.111:0/3160411291 shutdown_connections 2024-09-13T17:58:45.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.664+0000 7fe780a01640 1 -- 172.21.15.111:0/3160411291 wait complete. 2024-09-13T17:58:45.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.665+0000 7fe780a01640 1 Processor -- start 2024-09-13T17:58:45.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.665+0000 7fe780a01640 1 -- start start 2024-09-13T17:58:45.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.666+0000 7fe780a01640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe77c06bf00 0x7fe77c13acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:45.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.666+0000 7fe780a01640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe77c152f40 0x7fe77c1efd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:45.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.666+0000 7fe780a01640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe77c15ca10 0x7fe77c1e3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:45.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.666+0000 7fe780a01640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c1e3c20 con 0x7fe77c06bf00 2024-09-13T17:58:45.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.666+0000 7fe780a01640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c13f890 con 0x7fe77c15ca10 2024-09-13T17:58:45.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.666+0000 7fe780a01640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c13fb70 con 0x7fe77c152f40 2024-09-13T17:58:45.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.666+0000 7fe77ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe77c06bf00 0x7fe77c13acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:45.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.666+0000 7fe77ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe77c06bf00 0x7fe77c13acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40228/0 (socket says 172.21.15.111:40228) 2024-09-13T17:58:45.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.666+0000 7fe77ad9d640 1 -- 172.21.15.111:0/3254809249 learned_addr learned my addr 172.21.15.111:0/3254809249 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:45.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.666+0000 7fe77b59e640 1 --2- 172.21.15.111:0/3254809249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe77c15ca10 0x7fe77c1e3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:45.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.666+0000 7fe77a59c640 1 --2- 172.21.15.111:0/3254809249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe77c152f40 0x7fe77c1efd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:45.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.666+0000 7fe77ad9d640 1 -- 172.21.15.111:0/3254809249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe77c152f40 msgr2=0x7fe77c1efd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:45.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.666+0000 7fe77ad9d640 1 --2- 172.21.15.111:0/3254809249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe77c152f40 0x7fe77c1efd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.667+0000 7fe77ad9d640 1 -- 172.21.15.111:0/3254809249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe77c15ca10 msgr2=0x7fe77c1e3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:45.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.667+0000 7fe77ad9d640 1 --2- 172.21.15.111:0/3254809249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe77c15ca10 0x7fe77c1e3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.667+0000 7fe77ad9d640 1 -- 172.21.15.111:0/3254809249 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76c0094d0 con 0x7fe77c06bf00 2024-09-13T17:58:45.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.667+0000 7fe77b59e640 1 --2- 172.21.15.111:0/3254809249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe77c15ca10 0x7fe77c1e3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:45.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.667+0000 7fe77ad9d640 1 --2- 172.21.15.111:0/3254809249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe77c06bf00 0x7fe77c13acd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe76c000c00 tx=0x7fe76c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:45.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.667+0000 7fe7637fe640 1 -- 172.21.15.111:0/3254809249 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c032440 con 0x7fe77c06bf00 2024-09-13T17:58:45.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.667+0000 7fe7637fe640 1 -- 172.21.15.111:0/3254809249 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76c030b80 con 0x7fe77c06bf00 2024-09-13T17:58:45.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.667+0000 7fe780a01640 1 -- 172.21.15.111:0/3254809249 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77c13fe50 con 0x7fe77c06bf00 2024-09-13T17:58:45.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.667+0000 7fe7637fe640 1 -- 172.21.15.111:0/3254809249 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c03b400 con 0x7fe77c06bf00 2024-09-13T17:58:45.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.667+0000 7fe780a01640 1 -- 172.21.15.111:0/3254809249 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77c1f0250 con 0x7fe77c06bf00 2024-09-13T17:58:45.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.668+0000 7fe780a01640 1 -- 172.21.15.111:0/3254809249 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe77c13fcb0 con 0x7fe77c06bf00 2024-09-13T17:58:45.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.668+0000 7fe7637fe640 1 -- 172.21.15.111:0/3254809249 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe76c039040 con 0x7fe77c06bf00 2024-09-13T17:58:45.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.669+0000 7fe7637fe640 1 --2- 172.21.15.111:0/3254809249 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe76806ac00 0x7fe76806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:45.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.669+0000 7fe7637fe640 1 -- 172.21.15.111:0/3254809249 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe76c0a9db0 con 0x7fe77c06bf00 2024-09-13T17:58:45.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.669+0000 7fe77a59c640 1 --2- 172.21.15.111:0/3254809249 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe76806ac00 0x7fe76806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:45.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.669+0000 7fe77a59c640 1 --2- 172.21.15.111:0/3254809249 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe76806ac00 0x7fe76806d0c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0ae100 tx=0x7fe770008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:45.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.673+0000 7fe7637fe640 1 -- 172.21.15.111:0/3254809249 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fe77c13fcb0 con 0x7fe77c06bf00 2024-09-13T17:58:45.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.867+0000 7fe780a01640 1 -- 172.21.15.111:0/3254809249 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe77c13fcb0 con 0x7fe77c06bf00 2024-09-13T17:58:45.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.867+0000 7fe7637fe640 1 -- 172.21.15.111:0/3254809249 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fe77c13fcb0 con 0x7fe77c06bf00 2024-09-13T17:58:45.870 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:45.870 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4217138481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4217138481},{"type":"v1","addr":"172.21.15.188:6834","nonce":4217138481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3099948287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3099948287},{"type":"v1","addr":"172.21.15.188:6835","nonce":3099948287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/214034224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":214034224},{"type":"v1","addr":"172.21.15.111:6835","nonce":214034224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:58:43.534567+0000","modified":"2024-09-13T17:58:45.542773+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5490":{"gid":5490,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6837/982359636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":982359636},{"type":"v1","addr":"172.21.15.111:6837","nonce":982359636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:58:45.871 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 15 2024-09-13T17:58:45.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.869+0000 7fe780a01640 1 -- 172.21.15.111:0/3254809249 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe76806ac00 msgr2=0x7fe76806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:45.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.869+0000 7fe780a01640 1 --2- 172.21.15.111:0/3254809249 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe76806ac00 0x7fe76806d0c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0ae100 tx=0x7fe770008040 comp rx=0 tx=0).stop 2024-09-13T17:58:45.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.869+0000 7fe780a01640 1 -- 172.21.15.111:0/3254809249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe77c06bf00 msgr2=0x7fe77c13acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:45.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.869+0000 7fe780a01640 1 --2- 172.21.15.111:0/3254809249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe77c06bf00 0x7fe77c13acd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe76c000c00 tx=0x7fe76c0046c0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.869+0000 7fe780a01640 1 -- 172.21.15.111:0/3254809249 shutdown_connections 2024-09-13T17:58:45.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.869+0000 7fe780a01640 1 --2- 172.21.15.111:0/3254809249 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fe76806ac00 0x7fe76806d0c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.869+0000 7fe780a01640 1 --2- 172.21.15.111:0/3254809249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe77c15ca10 0x7fe77c1e3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.869+0000 7fe780a01640 1 --2- 172.21.15.111:0/3254809249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe77c152f40 0x7fe77c1efd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.869+0000 7fe780a01640 1 --2- 172.21.15.111:0/3254809249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe77c06bf00 0x7fe77c13acd0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.869+0000 7fe780a01640 1 -- 172.21.15.111:0/3254809249 >> 172.21.15.111:0/3254809249 conn(0x7fe77c06d860 msgr2=0x7fe77c153390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:45.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.870+0000 7fe780a01640 1 -- 172.21.15.111:0/3254809249 shutdown_connections 2024-09-13T17:58:45.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.870+0000 7fe780a01640 1 -- 172.21.15.111:0/3254809249 wait complete. 2024-09-13T17:58:45.883 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:58:45.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.976+0000 7f33a3f33640 1 Processor -- start 2024-09-13T17:58:45.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.977+0000 7f33a3f33640 1 -- start start 2024-09-13T17:58:45.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.977+0000 7f33a3f33640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f339c072110 0x7f339c070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:45.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.977+0000 7f33a3f33640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f339c0abdf0 con 0x7f339c0c8920 2024-09-13T17:58:45.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.977+0000 7f33a3f33640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f339c0b2280 con 0x7f339c071d30 2024-09-13T17:58:45.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.977+0000 7f33a3f33640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339c0ab1e0 con 0x7f339c072110 2024-09-13T17:58:45.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.978+0000 7f33a24d0640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f339c0c8920 0x7f339c0ac280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58146/0 (socket says 172.21.15.111:58146) 2024-09-13T17:58:45.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.978+0000 7f33a24d0640 1 -- 172.21.15.111:0/855380740 learned_addr learned my addr 172.21.15.111:0/855380740 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:45.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.978+0000 7f33a14ce640 1 --2- 172.21.15.111:0/855380740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f339c072110 0x7f339c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:45.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.978+0000 7f33a0ccd640 1 -- 172.21.15.111:0/855380740 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149646943 0 0) 0x7f339c0abdf0 con 0x7f339c0c8920 2024-09-13T17:58:45.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.978+0000 7f33a0ccd640 1 -- 172.21.15.111:0/855380740 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3388003520 con 0x7f339c0c8920 2024-09-13T17:58:45.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.978+0000 7f33a0ccd640 1 -- 172.21.15.111:0/855380740 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995040802 0 0) 0x7f339c0b2280 con 0x7f339c071d30 2024-09-13T17:58:45.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.978+0000 7f33a0ccd640 1 -- 172.21.15.111:0/855380740 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f339c0abdf0 con 0x7f339c071d30 2024-09-13T17:58:45.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.978+0000 7f33a0ccd640 1 -- 172.21.15.111:0/855380740 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3971314701 0 0) 0x7f3388003520 con 0x7f339c0c8920 2024-09-13T17:58:45.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.978+0000 7f33a14ce640 1 -- 172.21.15.111:0/855380740 >> v1:172.21.15.188:6789/0 conn(0x7f339c071d30 legacy=0x7f339c0b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:45.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.978+0000 7f33a14ce640 1 -- 172.21.15.111:0/855380740 >> v1:172.21.15.111:6789/0 conn(0x7f339c0c8920 legacy=0x7f339c0ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:45.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.978+0000 7f33a14ce640 1 -- 172.21.15.111:0/855380740 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339c0ab4c0 con 0x7f339c072110 2024-09-13T17:58:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.979+0000 7f33a14ce640 1 --2- 172.21.15.111:0/855380740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f339c072110 0x7f339c070190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3390009800 tx=0x7f339002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9ddb2690c650f60 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.979+0000 7f33a0ccd640 1 -- 172.21.15.111:0/855380740 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33940019f0 con 0x7f339c072110 2024-09-13T17:58:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.979+0000 7f33a0ccd640 1 -- 172.21.15.111:0/855380740 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339002ac50 con 0x7f339c072110 2024-09-13T17:58:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.979+0000 7f33a0ccd640 1 -- 172.21.15.111:0/855380740 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3390032440 con 0x7f339c072110 2024-09-13T17:58:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.980+0000 7f33a3f33640 1 -- 172.21.15.111:0/855380740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f339c072110 msgr2=0x7f339c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.980+0000 7f33a3f33640 1 --2- 172.21.15.111:0/855380740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f339c072110 0x7f339c070190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3390009800 tx=0x7f339002a1c0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.980+0000 7f33a3f33640 1 -- 172.21.15.111:0/855380740 shutdown_connections 2024-09-13T17:58:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.980+0000 7f33a3f33640 1 --2- 172.21.15.111:0/855380740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f339c072110 0x7f339c070190 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.980+0000 7f33a3f33640 1 -- 172.21.15.111:0/855380740 >> 172.21.15.111:0/855380740 conn(0x7f339c069ed0 msgr2=0x7f339c06a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.980+0000 7f33a3f33640 1 -- 172.21.15.111:0/855380740 shutdown_connections 2024-09-13T17:58:45.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.980+0000 7f33a3f33640 1 -- 172.21.15.111:0/855380740 wait complete. 2024-09-13T17:58:45.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.981+0000 7f33a3f33640 1 Processor -- start 2024-09-13T17:58:45.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.981+0000 7f33a3f33640 1 -- start start 2024-09-13T17:58:45.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a3f33640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f339c071d30 0x7f339c159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:45.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a3f33640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f339c072110 0x7f339c155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:45.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a3f33640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f339c0c8920 0x7f339c156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:45.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a3f33640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339c1589e0 con 0x7f339c071d30 2024-09-13T17:58:45.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a3f33640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339c158cc0 con 0x7f339c072110 2024-09-13T17:58:45.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a3f33640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339c158fa0 con 0x7f339c0c8920 2024-09-13T17:58:45.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a1ccf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f339c071d30 0x7f339c159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:45.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a1ccf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f339c071d30 0x7f339c159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40238/0 (socket says 172.21.15.111:40238) 2024-09-13T17:58:45.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a1ccf640 1 -- 172.21.15.111:0/3993037351 learned_addr learned my addr 172.21.15.111:0/3993037351 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a1ccf640 1 -- 172.21.15.111:0/3993037351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f339c0c8920 msgr2=0x7f339c156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a1ccf640 1 --2- 172.21.15.111:0/3993037351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f339c0c8920 0x7f339c156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a1ccf640 1 -- 172.21.15.111:0/3993037351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f339c072110 msgr2=0x7f339c155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a14ce640 1 --2- 172.21.15.111:0/3993037351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f339c072110 0x7f339c155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a1ccf640 1 --2- 172.21.15.111:0/3993037351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f339c072110 0x7f339c155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a1ccf640 1 -- 172.21.15.111:0/3993037351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3390009400 con 0x7f339c071d30 2024-09-13T17:58:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a1ccf640 1 --2- 172.21.15.111:0/3993037351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f339c071d30 0x7f339c159a80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f339800c490 tx=0x7f339800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f338e7fc640 1 -- 172.21.15.111:0/3993037351 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33980052d0 con 0x7f339c071d30 2024-09-13T17:58:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f338e7fc640 1 -- 172.21.15.111:0/3993037351 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3398005430 con 0x7f339c071d30 2024-09-13T17:58:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a3f33640 1 -- 172.21.15.111:0/3993037351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339c159280 con 0x7f339c071d30 2024-09-13T17:58:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f338e7fc640 1 -- 172.21.15.111:0/3993037351 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398012870 con 0x7f339c071d30 2024-09-13T17:58:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.982+0000 7f33a3f33640 1 -- 172.21.15.111:0/3993037351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f339c155f60 con 0x7f339c071d30 2024-09-13T17:58:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.983+0000 7f33a3f33640 1 -- 172.21.15.111:0/3993037351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339c1597a0 con 0x7f339c071d30 2024-09-13T17:58:45.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.984+0000 7f338e7fc640 1 -- 172.21.15.111:0/3993037351 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f33980129d0 con 0x7f339c071d30 2024-09-13T17:58:45.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.984+0000 7f338e7fc640 1 --2- 172.21.15.111:0/3993037351 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f338006ac00 0x7f338006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:45.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.984+0000 7f338e7fc640 1 -- 172.21.15.111:0/3993037351 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f33980890f0 con 0x7f339c071d30 2024-09-13T17:58:45.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.984+0000 7f33a14ce640 1 --2- 172.21.15.111:0/3993037351 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f338006ac00 0x7f338006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:45.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.985+0000 7f33a14ce640 1 --2- 172.21.15.111:0/3993037351 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f338006ac00 0x7f338006d0c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3390034000 tx=0x7f339002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:45.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:45.988+0000 7f338e7fc640 1 -- 172.21.15.111:0/3993037351 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f339c1597a0 con 0x7f339c071d30 2024-09-13T17:58:46.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.149+0000 7f33a3f33640 1 -- 172.21.15.111:0/3993037351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f339c1597a0 con 0x7f339c071d30 2024-09-13T17:58:46.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.151+0000 7f338e7fc640 1 -- 172.21.15.111:0/3993037351 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16834 (secure 0 0 0) 0x7f339c1597a0 con 0x7f339c071d30 2024-09-13T17:58:46.154 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:46.154 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":39,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:58:44.536402+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:58:41.698998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T17:58:42.499870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:58:46.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.153+0000 7f33a3f33640 1 -- 172.21.15.111:0/3993037351 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f338006ac00 msgr2=0x7f338006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.153+0000 7f33a3f33640 1 --2- 172.21.15.111:0/3993037351 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f338006ac00 0x7f338006d0c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3390034000 tx=0x7f339002e000 comp rx=0 tx=0).stop 2024-09-13T17:58:46.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.153+0000 7f33a3f33640 1 -- 172.21.15.111:0/3993037351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f339c071d30 msgr2=0x7f339c159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.153+0000 7f33a3f33640 1 --2- 172.21.15.111:0/3993037351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f339c071d30 0x7f339c159a80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f339800c490 tx=0x7f339800c950 comp rx=0 tx=0).stop 2024-09-13T17:58:46.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.153+0000 7f33a3f33640 1 -- 172.21.15.111:0/3993037351 shutdown_connections 2024-09-13T17:58:46.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.153+0000 7f33a3f33640 1 --2- 172.21.15.111:0/3993037351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f339c0c8920 0x7f339c156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.153+0000 7f33a3f33640 1 --2- 172.21.15.111:0/3993037351 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f338006ac00 0x7f338006d0c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.153+0000 7f33a3f33640 1 --2- 172.21.15.111:0/3993037351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f339c072110 0x7f339c155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.153+0000 7f33a3f33640 1 --2- 172.21.15.111:0/3993037351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f339c071d30 0x7f339c159a80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.153+0000 7f33a3f33640 1 -- 172.21.15.111:0/3993037351 >> 172.21.15.111:0/3993037351 conn(0x7f339c069ed0 msgr2=0x7f339c15a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:46.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.153+0000 7f33a3f33640 1 -- 172.21.15.111:0/3993037351 shutdown_connections 2024-09-13T17:58:46.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.153+0000 7f33a3f33640 1 -- 172.21.15.111:0/3993037351 wait complete. 2024-09-13T17:58:46.167 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T17:58:46.167 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:58:46.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.260+0000 7ff86006c640 1 Processor -- start 2024-09-13T17:58:46.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.260+0000 7ff86006c640 1 -- start start 2024-09-13T17:58:46.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.261+0000 7ff86006c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff85806c4c0 0x7ff85806c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.261+0000 7ff86006c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8580b2170 con 0x7ff8580cbea0 2024-09-13T17:58:46.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.261+0000 7ff86006c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85806dab0 con 0x7ff8580b2590 2024-09-13T17:58:46.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.261+0000 7ff86006c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff85806deb0 con 0x7ff85806c4c0 2024-09-13T17:58:46.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.261+0000 7ff85de08640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff85806c4c0 0x7ff85806c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.261+0000 7ff85e609640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff8580cbea0 0x7ff8580cc330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58156/0 (socket says 172.21.15.111:58156) 2024-09-13T17:58:46.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.261+0000 7ff85e609640 1 -- 172.21.15.111:0/3895115747 learned_addr learned my addr 172.21.15.111:0/3895115747 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:46.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.261+0000 7ff85de08640 1 -- 172.21.15.111:0/3895115747 >> v1:172.21.15.188:6789/0 conn(0x7ff8580b2590 legacy=0x7ff85806d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:46.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.261+0000 7ff85de08640 1 -- 172.21.15.111:0/3895115747 >> v1:172.21.15.111:6789/0 conn(0x7ff8580cbea0 legacy=0x7ff8580cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.261+0000 7ff85de08640 1 -- 172.21.15.111:0/3895115747 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8580b29e0 con 0x7ff85806c4c0 2024-09-13T17:58:46.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.262+0000 7ff85de08640 1 --2- 172.21.15.111:0/3895115747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff85806c4c0 0x7ff85806c8a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff85400a940 tx=0x7ff85402b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423939c5ed1c0e1d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:46.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.262+0000 7ff85ce06640 1 -- 172.21.15.111:0/3895115747 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff854030710 con 0x7ff85806c4c0 2024-09-13T17:58:46.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.262+0000 7ff85ce06640 1 -- 172.21.15.111:0/3895115747 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85402e050 con 0x7ff85806c4c0 2024-09-13T17:58:46.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.262+0000 7ff85ce06640 1 -- 172.21.15.111:0/3895115747 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff854030a90 con 0x7ff85806c4c0 2024-09-13T17:58:46.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.263+0000 7ff86006c640 1 -- 172.21.15.111:0/3895115747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff85806c4c0 msgr2=0x7ff85806c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.263+0000 7ff86006c640 1 --2- 172.21.15.111:0/3895115747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff85806c4c0 0x7ff85806c8a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff85400a940 tx=0x7ff85402b5a0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.263+0000 7ff86006c640 1 -- 172.21.15.111:0/3895115747 shutdown_connections 2024-09-13T17:58:46.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.263+0000 7ff86006c640 1 --2- 172.21.15.111:0/3895115747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff85806c4c0 0x7ff85806c8a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.263+0000 7ff86006c640 1 -- 172.21.15.111:0/3895115747 >> 172.21.15.111:0/3895115747 conn(0x7ff858069ed0 msgr2=0x7ff85806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:46.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.263+0000 7ff86006c640 1 -- 172.21.15.111:0/3895115747 shutdown_connections 2024-09-13T17:58:46.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.263+0000 7ff86006c640 1 -- 172.21.15.111:0/3895115747 wait complete. 2024-09-13T17:58:46.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.264+0000 7ff86006c640 1 Processor -- start 2024-09-13T17:58:46.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.264+0000 7ff86006c640 1 -- start start 2024-09-13T17:58:46.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff86006c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff85806c4c0 0x7ff8580978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff86006c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8580b2590 0x7ff858071830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff86006c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8580cbea0 0x7ff8580cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff86006c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8580c71d0 con 0x7ff8580b2590 2024-09-13T17:58:46.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff86006c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8580c65c0 con 0x7ff85806c4c0 2024-09-13T17:58:46.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff86006c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8580c68a0 con 0x7ff8580cbea0 2024-09-13T17:58:46.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff85d607640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8580b2590 0x7ff858071830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff85d607640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8580b2590 0x7ff858071830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40252/0 (socket says 172.21.15.111:40252) 2024-09-13T17:58:46.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff85d607640 1 -- 172.21.15.111:0/1523951629 learned_addr learned my addr 172.21.15.111:0/1523951629 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:46.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff85d607640 1 -- 172.21.15.111:0/1523951629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8580cbea0 msgr2=0x7ff8580cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff85d607640 1 --2- 172.21.15.111:0/1523951629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8580cbea0 0x7ff8580cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff85d607640 1 -- 172.21.15.111:0/1523951629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff85806c4c0 msgr2=0x7ff8580978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff85d607640 1 --2- 172.21.15.111:0/1523951629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff85806c4c0 0x7ff8580978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff85d607640 1 -- 172.21.15.111:0/1523951629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85400a5a0 con 0x7ff8580b2590 2024-09-13T17:58:46.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff85d607640 1 --2- 172.21.15.111:0/1523951629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8580b2590 0x7ff858071830 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff84800c410 tx=0x7ff84800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:46.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff84e7fc640 1 -- 172.21.15.111:0/1523951629 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8480035f0 con 0x7ff8580b2590 2024-09-13T17:58:46.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff84e7fc640 1 -- 172.21.15.111:0/1523951629 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff848003750 con 0x7ff8580b2590 2024-09-13T17:58:46.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff84e7fc640 1 -- 172.21.15.111:0/1523951629 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff848003b40 con 0x7ff8580b2590 2024-09-13T17:58:46.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.265+0000 7ff86006c640 1 -- 172.21.15.111:0/1523951629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8580c6b80 con 0x7ff8580b2590 2024-09-13T17:58:46.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.266+0000 7ff86006c640 1 -- 172.21.15.111:0/1523951629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff858097df0 con 0x7ff8580b2590 2024-09-13T17:58:46.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.266+0000 7ff86006c640 1 -- 172.21.15.111:0/1523951629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8580ce6e0 con 0x7ff8580b2590 2024-09-13T17:58:46.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.267+0000 7ff84e7fc640 1 -- 172.21.15.111:0/1523951629 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff848003ca0 con 0x7ff8580b2590 2024-09-13T17:58:46.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.267+0000 7ff84e7fc640 1 --2- 172.21.15.111:0/1523951629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff83c06ac00 0x7ff83c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.267+0000 7ff84e7fc640 1 -- 172.21.15.111:0/1523951629 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff8480886f0 con 0x7ff8580b2590 2024-09-13T17:58:46.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.267+0000 7ff85de08640 1 --2- 172.21.15.111:0/1523951629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff83c06ac00 0x7ff83c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.268+0000 7ff85de08640 1 --2- 172.21.15.111:0/1523951629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff83c06ac00 0x7ff83c06d0c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff854035000 tx=0x7ff85402f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:46.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.272+0000 7ff84e7fc640 1 -- 172.21.15.111:0/1523951629 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff848053820 con 0x7ff8580b2590 2024-09-13T17:58:46.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.432+0000 7ff86006c640 1 -- 172.21.15.111:0/1523951629 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff85806ce80 con 0x7ff83c06ac00 2024-09-13T17:58:46.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.440+0000 7ff84e7fc640 1 -- 172.21.15.111:0/1523951629 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289554 (secure 0 0 0) 0x7ff8540308f0 con 0x7ff83c06ac00 2024-09-13T17:58:46.446 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:46.447 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-09-13T17:58:46.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.443+0000 7ff86006c640 1 -- 172.21.15.111:0/1523951629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff83c06ac00 msgr2=0x7ff83c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.443+0000 7ff86006c640 1 --2- 172.21.15.111:0/1523951629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff83c06ac00 0x7ff83c06d0c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff854035000 tx=0x7ff85402f000 comp rx=0 tx=0).stop 2024-09-13T17:58:46.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.443+0000 7ff86006c640 1 -- 172.21.15.111:0/1523951629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8580b2590 msgr2=0x7ff858071830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.443+0000 7ff86006c640 1 --2- 172.21.15.111:0/1523951629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8580b2590 0x7ff858071830 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff84800c410 tx=0x7ff84800c8d0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.443+0000 7ff86006c640 1 -- 172.21.15.111:0/1523951629 shutdown_connections 2024-09-13T17:58:46.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.443+0000 7ff86006c640 1 --2- 172.21.15.111:0/1523951629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8580cbea0 0x7ff8580cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.443+0000 7ff86006c640 1 --2- 172.21.15.111:0/1523951629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8580b2590 0x7ff858071830 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.443+0000 7ff86006c640 1 --2- 172.21.15.111:0/1523951629 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff83c06ac00 0x7ff83c06d0c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.443+0000 7ff86006c640 1 --2- 172.21.15.111:0/1523951629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff85806c4c0 0x7ff8580978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.443+0000 7ff86006c640 1 -- 172.21.15.111:0/1523951629 >> 172.21.15.111:0/1523951629 conn(0x7ff858069ed0 msgr2=0x7ff85806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:46.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.443+0000 7ff86006c640 1 -- 172.21.15.111:0/1523951629 shutdown_connections 2024-09-13T17:58:46.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.443+0000 7ff86006c640 1 -- 172.21.15.111:0/1523951629 wait complete. 2024-09-13T17:58:46.456 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-09-13T17:58:46.282895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220816,"kb_used_data":4600,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753904,"statfs":{"total":773094113280,"available":772867997696,"internally_reserved":0,"allocated":4710400,"data_stored":2668507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.099346"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562205+0000","last_change":"2024-09-13T17:58:43.556984+0000","last_active":"2024-09-13T17:58:43.562205+0000","last_peered":"2024-09-13T17:58:43.562205+0000","last_clean":"2024-09-13T17:58:43.562205+0000","last_became_active":"2024-09-13T17:58:43.556176+0000","last_became_peered":"2024-09-13T17:58:43.556176+0000","last_unstale":"2024-09-13T17:58:43.562205+0000","last_undegraded":"2024-09-13T17:58:43.562205+0000","last_fullsized":"2024-09-13T17:58:43.562205+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:10.713520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.564328+0000","last_change":"2024-09-13T17:58:43.560341+0000","last_active":"2024-09-13T17:58:43.564328+0000","last_peered":"2024-09-13T17:58:43.564328+0000","last_clean":"2024-09-13T17:58:43.564328+0000","last_became_active":"2024-09-13T17:58:43.560087+0000","last_became_peered":"2024-09-13T17:58:43.560087+0000","last_unstale":"2024-09-13T17:58:43.564328+0000","last_undegraded":"2024-09-13T17:58:43.564328+0000","last_fullsized":"2024-09-13T17:58:43.564328+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:21.360073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562070+0000","last_change":"2024-09-13T17:58:43.556764+0000","last_active":"2024-09-13T17:58:43.562070+0000","last_peered":"2024-09-13T17:58:43.562070+0000","last_clean":"2024-09-13T17:58:43.562070+0000","last_became_active":"2024-09-13T17:58:43.556351+0000","last_became_peered":"2024-09-13T17:58:43.556351+0000","last_unstale":"2024-09-13T17:58:43.562070+0000","last_undegraded":"2024-09-13T17:58:43.562070+0000","last_fullsized":"2024-09-13T17:58:43.562070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:37.690020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556882+0000","last_change":"2024-09-13T17:58:43.552094+0000","last_active":"2024-09-13T17:58:43.556882+0000","last_peered":"2024-09-13T17:58:43.556882+0000","last_clean":"2024-09-13T17:58:43.556882+0000","last_became_active":"2024-09-13T17:58:43.551589+0000","last_became_peered":"2024-09-13T17:58:43.551589+0000","last_unstale":"2024-09-13T17:58:43.556882+0000","last_undegraded":"2024-09-13T17:58:43.556882+0000","last_fullsized":"2024-09-13T17:58:43.556882+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:55:08.536744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.557015+0000","last_change":"2024-09-13T17:58:43.552157+0000","last_active":"2024-09-13T17:58:43.557015+0000","last_peered":"2024-09-13T17:58:43.557015+0000","last_clean":"2024-09-13T17:58:43.557015+0000","last_became_active":"2024-09-13T17:58:43.551818+0000","last_became_peered":"2024-09-13T17:58:43.551818+0000","last_unstale":"2024-09-13T17:58:43.557015+0000","last_undegraded":"2024-09-13T17:58:43.557015+0000","last_fullsized":"2024-09-13T17:58:43.557015+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:18.351367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556856+0000","last_change":"2024-09-13T17:58:43.551579+0000","last_active":"2024-09-13T17:58:43.556856+0000","last_peered":"2024-09-13T17:58:43.556856+0000","last_clean":"2024-09-13T17:58:43.556856+0000","last_became_active":"2024-09-13T17:58:43.551338+0000","last_became_peered":"2024-09-13T17:58:43.551338+0000","last_unstale":"2024-09-13T17:58:43.556856+0000","last_undegraded":"2024-09-13T17:58:43.556856+0000","last_fullsized":"2024-09-13T17:58:43.556856+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:14:10.174120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.564642+0000","last_change":"2024-09-13T17:58:43.561726+0000","last_active":"2024-09-13T17:58:43.564642+0000","last_peered":"2024-09-13T17:58:43.564642+0000","last_clean":"2024-09-13T17:58:43.564642+0000","last_became_active":"2024-09-13T17:58:43.560117+0000","last_became_peered":"2024-09-13T17:58:43.560117+0000","last_unstale":"2024-09-13T17:58:43.564642+0000","last_undegraded":"2024-09-13T17:58:43.564642+0000","last_fullsized":"2024-09-13T17:58:43.564642+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:02.733199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.558786+0000","last_change":"2024-09-13T17:58:43.552540+0000","last_active":"2024-09-13T17:58:43.558786+0000","last_peered":"2024-09-13T17:58:43.558786+0000","last_clean":"2024-09-13T17:58:43.558786+0000","last_became_active":"2024-09-13T17:58:43.552301+0000","last_became_peered":"2024-09-13T17:58:43.552301+0000","last_unstale":"2024-09-13T17:58:43.558786+0000","last_undegraded":"2024-09-13T17:58:43.558786+0000","last_fullsized":"2024-09-13T17:58:43.558786+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:17.159734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565307+0000","last_change":"2024-09-13T17:58:43.563956+0000","last_active":"2024-09-13T17:58:43.565307+0000","last_peered":"2024-09-13T17:58:43.565307+0000","last_clean":"2024-09-13T17:58:43.565307+0000","last_became_active":"2024-09-13T17:58:43.563691+0000","last_became_peered":"2024-09-13T17:58:43.563691+0000","last_unstale":"2024-09-13T17:58:43.565307+0000","last_undegraded":"2024-09-13T17:58:43.565307+0000","last_fullsized":"2024-09-13T17:58:43.565307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:35.154189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.568524+0000","last_change":"2024-09-13T17:58:43.564124+0000","last_active":"2024-09-13T17:58:43.568524+0000","last_peered":"2024-09-13T17:58:43.568524+0000","last_clean":"2024-09-13T17:58:43.568524+0000","last_became_active":"2024-09-13T17:58:43.562377+0000","last_became_peered":"2024-09-13T17:58:43.562377+0000","last_unstale":"2024-09-13T17:58:43.568524+0000","last_undegraded":"2024-09-13T17:58:43.568524+0000","last_fullsized":"2024-09-13T17:58:43.568524+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:00.243570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.561676+0000","last_change":"2024-09-13T17:58:43.553731+0000","last_active":"2024-09-13T17:58:43.561676+0000","last_peered":"2024-09-13T17:58:43.561676+0000","last_clean":"2024-09-13T17:58:43.561676+0000","last_became_active":"2024-09-13T17:58:43.553379+0000","last_became_peered":"2024-09-13T17:58:43.553379+0000","last_unstale":"2024-09-13T17:58:43.561676+0000","last_undegraded":"2024-09-13T17:58:43.561676+0000","last_fullsized":"2024-09-13T17:58:43.561676+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:14:10.174120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.561365+0000","last_change":"2024-09-13T17:58:43.555451+0000","last_active":"2024-09-13T17:58:43.561365+0000","last_peered":"2024-09-13T17:58:43.561365+0000","last_clean":"2024-09-13T17:58:43.561365+0000","last_became_active":"2024-09-13T17:58:43.555185+0000","last_became_peered":"2024-09-13T17:58:43.555185+0000","last_unstale":"2024-09-13T17:58:43.561365+0000","last_undegraded":"2024-09-13T17:58:43.561365+0000","last_fullsized":"2024-09-13T17:58:43.561365+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:27:21.200370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563761+0000","last_change":"2024-09-13T17:58:43.561000+0000","last_active":"2024-09-13T17:58:43.563761+0000","last_peered":"2024-09-13T17:58:43.563761+0000","last_clean":"2024-09-13T17:58:43.563761+0000","last_became_active":"2024-09-13T17:58:43.560465+0000","last_became_peered":"2024-09-13T17:58:43.560465+0000","last_unstale":"2024-09-13T17:58:43.563761+0000","last_undegraded":"2024-09-13T17:58:43.563761+0000","last_fullsized":"2024-09-13T17:58:43.563761+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:19.494551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565965+0000","last_change":"2024-09-13T17:58:43.558447+0000","last_active":"2024-09-13T17:58:43.565965+0000","last_peered":"2024-09-13T17:58:43.565965+0000","last_clean":"2024-09-13T17:58:43.565965+0000","last_became_active":"2024-09-13T17:58:43.558179+0000","last_became_peered":"2024-09-13T17:58:43.558179+0000","last_unstale":"2024-09-13T17:58:43.565965+0000","last_undegraded":"2024-09-13T17:58:43.565965+0000","last_fullsized":"2024-09-13T17:58:43.565965+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:26.374764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.564616+0000","last_change":"2024-09-13T17:58:43.561536+0000","last_active":"2024-09-13T17:58:43.564616+0000","last_peered":"2024-09-13T17:58:43.564616+0000","last_clean":"2024-09-13T17:58:43.564616+0000","last_became_active":"2024-09-13T17:58:43.555740+0000","last_became_peered":"2024-09-13T17:58:43.555740+0000","last_unstale":"2024-09-13T17:58:43.564616+0000","last_undegraded":"2024-09-13T17:58:43.564616+0000","last_fullsized":"2024-09-13T17:58:43.564616+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:26.374764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.568644+0000","last_change":"2024-09-13T17:58:43.564778+0000","last_active":"2024-09-13T17:58:43.568644+0000","last_peered":"2024-09-13T17:58:43.568644+0000","last_clean":"2024-09-13T17:58:43.568644+0000","last_became_active":"2024-09-13T17:58:43.562752+0000","last_became_peered":"2024-09-13T17:58:43.562752+0000","last_unstale":"2024-09-13T17:58:43.568644+0000","last_undegraded":"2024-09-13T17:58:43.568644+0000","last_fullsized":"2024-09-13T17:58:43.568644+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:17:50.428940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.568600+0000","last_change":"2024-09-13T17:58:43.564644+0000","last_active":"2024-09-13T17:58:43.568600+0000","last_peered":"2024-09-13T17:58:43.568600+0000","last_clean":"2024-09-13T17:58:43.568600+0000","last_became_active":"2024-09-13T17:58:43.564229+0000","last_became_peered":"2024-09-13T17:58:43.564229+0000","last_unstale":"2024-09-13T17:58:43.568600+0000","last_undegraded":"2024-09-13T17:58:43.568600+0000","last_fullsized":"2024-09-13T17:58:43.568600+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:06:10.375768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.573952+0000","last_change":"2024-09-13T17:58:43.566527+0000","last_active":"2024-09-13T17:58:43.573952+0000","last_peered":"2024-09-13T17:58:43.573952+0000","last_clean":"2024-09-13T17:58:43.573952+0000","last_became_active":"2024-09-13T17:58:43.566254+0000","last_became_peered":"2024-09-13T17:58:43.566254+0000","last_unstale":"2024-09-13T17:58:43.573952+0000","last_undegraded":"2024-09-13T17:58:43.573952+0000","last_fullsized":"2024-09-13T17:58:43.573952+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:06:10.375768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.554797+0000","last_change":"2024-09-13T17:58:43.545630+0000","last_active":"2024-09-13T17:58:43.554797+0000","last_peered":"2024-09-13T17:58:43.554797+0000","last_clean":"2024-09-13T17:58:43.554797+0000","last_became_active":"2024-09-13T17:58:43.545383+0000","last_became_peered":"2024-09-13T17:58:43.545383+0000","last_unstale":"2024-09-13T17:58:43.554797+0000","last_undegraded":"2024-09-13T17:58:43.554797+0000","last_fullsized":"2024-09-13T17:58:43.554797+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:58:38.983402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.573919+0000","last_change":"2024-09-13T17:58:43.566046+0000","last_active":"2024-09-13T17:58:43.573919+0000","last_peered":"2024-09-13T17:58:43.573919+0000","last_clean":"2024-09-13T17:58:43.573919+0000","last_became_active":"2024-09-13T17:58:43.565783+0000","last_became_peered":"2024-09-13T17:58:43.565783+0000","last_unstale":"2024-09-13T17:58:43.573919+0000","last_undegraded":"2024-09-13T17:58:43.573919+0000","last_fullsized":"2024-09-13T17:58:43.573919+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:51.256171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.561379+0000","last_change":"2024-09-13T17:58:43.550420+0000","last_active":"2024-09-13T17:58:43.561379+0000","last_peered":"2024-09-13T17:58:43.561379+0000","last_clean":"2024-09-13T17:58:43.561379+0000","last_became_active":"2024-09-13T17:58:43.549743+0000","last_became_peered":"2024-09-13T17:58:43.549743+0000","last_unstale":"2024-09-13T17:58:43.561379+0000","last_undegraded":"2024-09-13T17:58:43.561379+0000","last_fullsized":"2024-09-13T17:58:43.561379+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:16.379363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565946+0000","last_change":"2024-09-13T17:58:43.557316+0000","last_active":"2024-09-13T17:58:43.565946+0000","last_peered":"2024-09-13T17:58:43.565946+0000","last_clean":"2024-09-13T17:58:43.565946+0000","last_became_active":"2024-09-13T17:58:43.557043+0000","last_became_peered":"2024-09-13T17:58:43.557043+0000","last_unstale":"2024-09-13T17:58:43.565946+0000","last_undegraded":"2024-09-13T17:58:43.565946+0000","last_fullsized":"2024-09-13T17:58:43.565946+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:16.379363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562852+0000","last_change":"2024-09-13T17:58:43.555606+0000","last_active":"2024-09-13T17:58:43.562852+0000","last_peered":"2024-09-13T17:58:43.562852+0000","last_clean":"2024-09-13T17:58:43.562852+0000","last_became_active":"2024-09-13T17:58:43.555364+0000","last_became_peered":"2024-09-13T17:58:43.555364+0000","last_unstale":"2024-09-13T17:58:43.562852+0000","last_undegraded":"2024-09-13T17:58:43.562852+0000","last_fullsized":"2024-09-13T17:58:43.562852+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:35:24.369374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.554926+0000","last_change":"2024-09-13T17:58:43.547702+0000","last_active":"2024-09-13T17:58:43.554926+0000","last_peered":"2024-09-13T17:58:43.554926+0000","last_clean":"2024-09-13T17:58:43.554926+0000","last_became_active":"2024-09-13T17:58:43.547451+0000","last_became_peered":"2024-09-13T17:58:43.547451+0000","last_unstale":"2024-09-13T17:58:43.554926+0000","last_undegraded":"2024-09-13T17:58:43.554926+0000","last_fullsized":"2024-09-13T17:58:43.554926+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:28.862716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.568450+0000","last_change":"2024-09-13T17:58:43.564178+0000","last_active":"2024-09-13T17:58:43.568450+0000","last_peered":"2024-09-13T17:58:43.568450+0000","last_clean":"2024-09-13T17:58:43.568450+0000","last_became_active":"2024-09-13T17:58:43.563907+0000","last_became_peered":"2024-09-13T17:58:43.563907+0000","last_unstale":"2024-09-13T17:58:43.568450+0000","last_undegraded":"2024-09-13T17:58:43.568450+0000","last_fullsized":"2024-09-13T17:58:43.568450+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:46.439570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556929+0000","last_change":"2024-09-13T17:58:43.553462+0000","last_active":"2024-09-13T17:58:43.556929+0000","last_peered":"2024-09-13T17:58:43.556929+0000","last_clean":"2024-09-13T17:58:43.556929+0000","last_became_active":"2024-09-13T17:58:43.553203+0000","last_became_peered":"2024-09-13T17:58:43.553203+0000","last_unstale":"2024-09-13T17:58:43.556929+0000","last_undegraded":"2024-09-13T17:58:43.556929+0000","last_fullsized":"2024-09-13T17:58:43.556929+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:19.494551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565808+0000","last_change":"2024-09-13T17:58:43.561198+0000","last_active":"2024-09-13T17:58:43.565808+0000","last_peered":"2024-09-13T17:58:43.565808+0000","last_clean":"2024-09-13T17:58:43.565808+0000","last_became_active":"2024-09-13T17:58:43.560941+0000","last_became_peered":"2024-09-13T17:58:43.560941+0000","last_unstale":"2024-09-13T17:58:43.565808+0000","last_undegraded":"2024-09-13T17:58:43.565808+0000","last_fullsized":"2024-09-13T17:58:43.565808+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:05:37.687367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.560469+0000","last_change":"2024-09-13T17:58:43.551304+0000","last_active":"2024-09-13T17:58:43.560469+0000","last_peered":"2024-09-13T17:58:43.560469+0000","last_clean":"2024-09-13T17:58:43.560469+0000","last_became_active":"2024-09-13T17:58:43.551048+0000","last_became_peered":"2024-09-13T17:58:43.551048+0000","last_unstale":"2024-09-13T17:58:43.560469+0000","last_undegraded":"2024-09-13T17:58:43.560469+0000","last_fullsized":"2024-09-13T17:58:43.560469+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:10.713520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565459+0000","last_change":"2024-09-13T17:58:43.559805+0000","last_active":"2024-09-13T17:58:43.565459+0000","last_peered":"2024-09-13T17:58:43.565459+0000","last_clean":"2024-09-13T17:58:43.565459+0000","last_became_active":"2024-09-13T17:58:43.559537+0000","last_became_peered":"2024-09-13T17:58:43.559537+0000","last_unstale":"2024-09-13T17:58:43.565459+0000","last_undegraded":"2024-09-13T17:58:43.565459+0000","last_fullsized":"2024-09-13T17:58:43.565459+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:28.859129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565570+0000","last_change":"2024-09-13T17:58:43.561111+0000","last_active":"2024-09-13T17:58:43.565570+0000","last_peered":"2024-09-13T17:58:43.565570+0000","last_clean":"2024-09-13T17:58:43.565570+0000","last_became_active":"2024-09-13T17:58:43.560843+0000","last_became_peered":"2024-09-13T17:58:43.560843+0000","last_unstale":"2024-09-13T17:58:43.565570+0000","last_undegraded":"2024-09-13T17:58:43.565570+0000","last_fullsized":"2024-09-13T17:58:43.565570+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:17:50.428940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563656+0000","last_change":"2024-09-13T17:58:43.557564+0000","last_active":"2024-09-13T17:58:43.563656+0000","last_peered":"2024-09-13T17:58:43.563656+0000","last_clean":"2024-09-13T17:58:43.563656+0000","last_became_active":"2024-09-13T17:58:43.557326+0000","last_became_peered":"2024-09-13T17:58:43.557326+0000","last_unstale":"2024-09-13T17:58:43.563656+0000","last_undegraded":"2024-09-13T17:58:43.563656+0000","last_fullsized":"2024-09-13T17:58:43.563656+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:43:49.563423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556957+0000","last_change":"2024-09-13T17:58:43.555404+0000","last_active":"2024-09-13T17:58:43.556957+0000","last_peered":"2024-09-13T17:58:43.556957+0000","last_clean":"2024-09-13T17:58:43.556957+0000","last_became_active":"2024-09-13T17:58:43.555157+0000","last_became_peered":"2024-09-13T17:58:43.555157+0000","last_unstale":"2024-09-13T17:58:43.556957+0000","last_undegraded":"2024-09-13T17:58:43.556957+0000","last_fullsized":"2024-09-13T17:58:43.556957+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:37.690020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.568609+0000","last_change":"2024-09-13T17:58:43.564464+0000","last_active":"2024-09-13T17:58:43.568609+0000","last_peered":"2024-09-13T17:58:43.568609+0000","last_clean":"2024-09-13T17:58:43.568609+0000","last_became_active":"2024-09-13T17:58:43.564207+0000","last_became_peered":"2024-09-13T17:58:43.564207+0000","last_unstale":"2024-09-13T17:58:43.568609+0000","last_undegraded":"2024-09-13T17:58:43.568609+0000","last_fullsized":"2024-09-13T17:58:43.568609+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.615927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.567240+0000","last_change":"2024-09-13T17:58:43.207136+0000","last_active":"2024-09-13T17:58:43.567240+0000","last_peered":"2024-09-13T17:58:43.567240+0000","last_clean":"2024-09-13T17:58:43.567240+0000","last_became_active":"2024-09-13T17:58:43.206864+0000","last_became_peered":"2024-09-13T17:58:43.206864+0000","last_unstale":"2024-09-13T17:58:43.567240+0000","last_undegraded":"2024-09-13T17:58:43.567240+0000","last_fullsized":"2024-09-13T17:58:43.567240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:22.930494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556954+0000","last_change":"2024-09-13T17:58:43.554041+0000","last_active":"2024-09-13T17:58:43.556954+0000","last_peered":"2024-09-13T17:58:43.556954+0000","last_clean":"2024-09-13T17:58:43.556954+0000","last_became_active":"2024-09-13T17:58:43.553782+0000","last_became_peered":"2024-09-13T17:58:43.553782+0000","last_unstale":"2024-09-13T17:58:43.556954+0000","last_undegraded":"2024-09-13T17:58:43.556954+0000","last_fullsized":"2024-09-13T17:58:43.556954+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:26.363369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562219+0000","last_change":"2024-09-13T17:58:43.203714+0000","last_active":"2024-09-13T17:58:43.562219+0000","last_peered":"2024-09-13T17:58:43.562219+0000","last_clean":"2024-09-13T17:58:43.562219+0000","last_became_active":"2024-09-13T17:58:43.203467+0000","last_became_peered":"2024-09-13T17:58:43.203467+0000","last_unstale":"2024-09-13T17:58:43.562219+0000","last_undegraded":"2024-09-13T17:58:43.562219+0000","last_fullsized":"2024-09-13T17:58:43.562219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:45.437304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565928+0000","last_change":"2024-09-13T17:58:43.560616+0000","last_active":"2024-09-13T17:58:43.565928+0000","last_peered":"2024-09-13T17:58:43.565928+0000","last_clean":"2024-09-13T17:58:43.565928+0000","last_became_active":"2024-09-13T17:58:43.560372+0000","last_became_peered":"2024-09-13T17:58:43.560372+0000","last_unstale":"2024-09-13T17:58:43.565928+0000","last_undegraded":"2024-09-13T17:58:43.565928+0000","last_fullsized":"2024-09-13T17:58:43.565928+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:19.494551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563807+0000","last_change":"2024-09-13T17:58:43.206952+0000","last_active":"2024-09-13T17:58:43.563807+0000","last_peered":"2024-09-13T17:58:43.563807+0000","last_clean":"2024-09-13T17:58:43.563807+0000","last_became_active":"2024-09-13T17:58:43.206620+0000","last_became_peered":"2024-09-13T17:58:43.206620+0000","last_unstale":"2024-09-13T17:58:43.563807+0000","last_undegraded":"2024-09-13T17:58:43.563807+0000","last_fullsized":"2024-09-13T17:58:43.563807+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:29.730687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.564344+0000","last_change":"2024-09-13T17:58:43.560505+0000","last_active":"2024-09-13T17:58:43.564344+0000","last_peered":"2024-09-13T17:58:43.564344+0000","last_clean":"2024-09-13T17:58:43.564344+0000","last_became_active":"2024-09-13T17:58:43.560260+0000","last_became_peered":"2024-09-13T17:58:43.560260+0000","last_unstale":"2024-09-13T17:58:43.564344+0000","last_undegraded":"2024-09-13T17:58:43.564344+0000","last_fullsized":"2024-09-13T17:58:43.564344+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:25.232164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.559708+0000","last_change":"2024-09-13T17:58:43.206701+0000","last_active":"2024-09-13T17:58:43.559708+0000","last_peered":"2024-09-13T17:58:43.559708+0000","last_clean":"2024-09-13T17:58:43.559708+0000","last_became_active":"2024-09-13T17:58:43.206426+0000","last_became_peered":"2024-09-13T17:58:43.206426+0000","last_unstale":"2024-09-13T17:58:43.559708+0000","last_undegraded":"2024-09-13T17:58:43.559708+0000","last_fullsized":"2024-09-13T17:58:43.559708+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:23:25.857306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.550917+0000","last_change":"2024-09-13T17:58:43.546830+0000","last_active":"2024-09-13T17:58:43.550917+0000","last_peered":"2024-09-13T17:58:43.550917+0000","last_clean":"2024-09-13T17:58:43.550917+0000","last_became_active":"2024-09-13T17:58:43.546505+0000","last_became_peered":"2024-09-13T17:58:43.546505+0000","last_unstale":"2024-09-13T17:58:43.550917+0000","last_undegraded":"2024-09-13T17:58:43.550917+0000","last_fullsized":"2024-09-13T17:58:43.550917+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:16.379363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563796+0000","last_change":"2024-09-13T17:58:43.202035+0000","last_active":"2024-09-13T17:58:43.563796+0000","last_peered":"2024-09-13T17:58:43.563796+0000","last_clean":"2024-09-13T17:58:43.563796+0000","last_became_active":"2024-09-13T17:58:43.201759+0000","last_became_peered":"2024-09-13T17:58:43.201759+0000","last_unstale":"2024-09-13T17:58:43.563796+0000","last_undegraded":"2024-09-13T17:58:43.563796+0000","last_fullsized":"2024-09-13T17:58:43.563796+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:27.856863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562140+0000","last_change":"2024-09-13T17:58:43.550273+0000","last_active":"2024-09-13T17:58:43.562140+0000","last_peered":"2024-09-13T17:58:43.562140+0000","last_clean":"2024-09-13T17:58:43.562140+0000","last_became_active":"2024-09-13T17:58:43.549963+0000","last_became_peered":"2024-09-13T17:58:43.549963+0000","last_unstale":"2024-09-13T17:58:43.562140+0000","last_undegraded":"2024-09-13T17:58:43.562140+0000","last_fullsized":"2024-09-13T17:58:43.562140+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:46.192447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.561552+0000","last_change":"2024-09-13T17:58:43.208905+0000","last_active":"2024-09-13T17:58:43.561552+0000","last_peered":"2024-09-13T17:58:43.561552+0000","last_clean":"2024-09-13T17:58:43.561552+0000","last_became_active":"2024-09-13T17:58:43.208640+0000","last_became_peered":"2024-09-13T17:58:43.208640+0000","last_unstale":"2024-09-13T17:58:43.561552+0000","last_undegraded":"2024-09-13T17:58:43.561552+0000","last_fullsized":"2024-09-13T17:58:43.561552+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:22.930494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.566952+0000","last_change":"2024-09-13T17:58:43.562992+0000","last_active":"2024-09-13T17:58:43.566952+0000","last_peered":"2024-09-13T17:58:43.566952+0000","last_clean":"2024-09-13T17:58:43.566952+0000","last_became_active":"2024-09-13T17:58:43.562726+0000","last_became_peered":"2024-09-13T17:58:43.562726+0000","last_unstale":"2024-09-13T17:58:43.566952+0000","last_undegraded":"2024-09-13T17:58:43.566952+0000","last_fullsized":"2024-09-13T17:58:43.566952+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:46.192447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.541437+0000","last_change":"2024-09-13T17:58:43.202053+0000","last_active":"2024-09-13T17:58:43.541437+0000","last_peered":"2024-09-13T17:58:43.541437+0000","last_clean":"2024-09-13T17:58:43.541437+0000","last_became_active":"2024-09-13T17:58:43.201742+0000","last_became_peered":"2024-09-13T17:58:43.201742+0000","last_unstale":"2024-09-13T17:58:43.541437+0000","last_undegraded":"2024-09-13T17:58:43.541437+0000","last_fullsized":"2024-09-13T17:58:43.541437+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:17:49.426674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565524+0000","last_change":"2024-09-13T17:58:43.562062+0000","last_active":"2024-09-13T17:58:43.565524+0000","last_peered":"2024-09-13T17:58:43.565524+0000","last_clean":"2024-09-13T17:58:43.565524+0000","last_became_active":"2024-09-13T17:58:43.561804+0000","last_became_peered":"2024-09-13T17:58:43.561804+0000","last_unstale":"2024-09-13T17:58:43.565524+0000","last_undegraded":"2024-09-13T17:58:43.565524+0000","last_fullsized":"2024-09-13T17:58:43.565524+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:22.589848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.548234+0000","last_change":"2024-09-13T17:58:43.206179+0000","last_active":"2024-09-13T17:58:43.548234+0000","last_peered":"2024-09-13T17:58:43.548234+0000","last_clean":"2024-09-13T17:58:43.548234+0000","last_became_active":"2024-09-13T17:58:43.205885+0000","last_became_peered":"2024-09-13T17:58:43.205885+0000","last_unstale":"2024-09-13T17:58:43.548234+0000","last_undegraded":"2024-09-13T17:58:43.548234+0000","last_fullsized":"2024-09-13T17:58:43.548234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:16.937891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556864+0000","last_change":"2024-09-13T17:58:43.551628+0000","last_active":"2024-09-13T17:58:43.556864+0000","last_peered":"2024-09-13T17:58:43.556864+0000","last_clean":"2024-09-13T17:58:43.556864+0000","last_became_active":"2024-09-13T17:58:43.551244+0000","last_became_peered":"2024-09-13T17:58:43.551244+0000","last_unstale":"2024-09-13T17:58:43.556864+0000","last_undegraded":"2024-09-13T17:58:43.556864+0000","last_fullsized":"2024-09-13T17:58:43.556864+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:27.522517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.549822+0000","last_change":"2024-09-13T17:58:43.206844+0000","last_active":"2024-09-13T17:58:43.549822+0000","last_peered":"2024-09-13T17:58:43.549822+0000","last_clean":"2024-09-13T17:58:43.549822+0000","last_became_active":"2024-09-13T17:58:43.206581+0000","last_became_peered":"2024-09-13T17:58:43.206581+0000","last_unstale":"2024-09-13T17:58:43.549822+0000","last_undegraded":"2024-09-13T17:58:43.549822+0000","last_fullsized":"2024-09-13T17:58:43.549822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:43:48.561157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563646+0000","last_change":"2024-09-13T17:58:43.559853+0000","last_active":"2024-09-13T17:58:43.563646+0000","last_peered":"2024-09-13T17:58:43.563646+0000","last_clean":"2024-09-13T17:58:43.563646+0000","last_became_active":"2024-09-13T17:58:43.559406+0000","last_became_peered":"2024-09-13T17:58:43.559406+0000","last_unstale":"2024-09-13T17:58:43.563646+0000","last_undegraded":"2024-09-13T17:58:43.563646+0000","last_fullsized":"2024-09-13T17:58:43.563646+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:23:26.859572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.544992+0000","last_change":"2024-09-13T17:58:43.204768+0000","last_active":"2024-09-13T17:58:43.544992+0000","last_peered":"2024-09-13T17:58:43.544992+0000","last_clean":"2024-09-13T17:58:43.544992+0000","last_became_active":"2024-09-13T17:58:43.204490+0000","last_became_peered":"2024-09-13T17:58:43.204490+0000","last_unstale":"2024-09-13T17:58:43.544992+0000","last_undegraded":"2024-09-13T17:58:43.544992+0000","last_fullsized":"2024-09-13T17:58:43.544992+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:06:09.373502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556744+0000","last_change":"2024-09-13T17:58:43.553057+0000","last_active":"2024-09-13T17:58:43.556744+0000","last_peered":"2024-09-13T17:58:43.556744+0000","last_clean":"2024-09-13T17:58:43.556744+0000","last_became_active":"2024-09-13T17:58:43.552770+0000","last_became_peered":"2024-09-13T17:58:43.552770+0000","last_unstale":"2024-09-13T17:58:43.556744+0000","last_undegraded":"2024-09-13T17:58:43.556744+0000","last_fullsized":"2024-09-13T17:58:43.556744+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:26.363369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.561627+0000","last_change":"2024-09-13T17:58:43.202291+0000","last_active":"2024-09-13T17:58:43.561627+0000","last_peered":"2024-09-13T17:58:43.561627+0000","last_clean":"2024-09-13T17:58:43.561627+0000","last_became_active":"2024-09-13T17:58:43.202001+0000","last_became_peered":"2024-09-13T17:58:43.202001+0000","last_unstale":"2024-09-13T17:58:43.561627+0000","last_undegraded":"2024-09-13T17:58:43.561627+0000","last_fullsized":"2024-09-13T17:58:43.561627+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:25:55.280446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.566785+0000","last_change":"2024-09-13T17:58:43.562408+0000","last_active":"2024-09-13T17:58:43.566785+0000","last_peered":"2024-09-13T17:58:43.566785+0000","last_clean":"2024-09-13T17:58:43.566785+0000","last_became_active":"2024-09-13T17:58:43.562152+0000","last_became_peered":"2024-09-13T17:58:43.562152+0000","last_unstale":"2024-09-13T17:58:43.566785+0000","last_undegraded":"2024-09-13T17:58:43.566785+0000","last_fullsized":"2024-09-13T17:58:43.566785+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:00.243570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.568966+0000","last_change":"2024-09-13T17:58:43.206067+0000","last_active":"2024-09-13T17:58:43.568966+0000","last_peered":"2024-09-13T17:58:43.568966+0000","last_clean":"2024-09-13T17:58:43.568966+0000","last_became_active":"2024-09-13T17:58:43.205791+0000","last_became_peered":"2024-09-13T17:58:43.205791+0000","last_unstale":"2024-09-13T17:58:43.568966+0000","last_undegraded":"2024-09-13T17:58:43.568966+0000","last_fullsized":"2024-09-13T17:58:43.568966+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:23:25.857306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562149+0000","last_change":"2024-09-13T17:58:43.557055+0000","last_active":"2024-09-13T17:58:43.562149+0000","last_peered":"2024-09-13T17:58:43.562149+0000","last_clean":"2024-09-13T17:58:43.562149+0000","last_became_active":"2024-09-13T17:58:43.556793+0000","last_became_peered":"2024-09-13T17:58:43.556793+0000","last_unstale":"2024-09-13T17:58:43.562149+0000","last_undegraded":"2024-09-13T17:58:43.562149+0000","last_fullsized":"2024-09-13T17:58:43.562149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:19.494551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.557145+0000","last_change":"2024-09-13T17:58:43.204485+0000","last_active":"2024-09-13T17:58:43.557145+0000","last_peered":"2024-09-13T17:58:43.557145+0000","last_clean":"2024-09-13T17:58:43.557145+0000","last_became_active":"2024-09-13T17:58:43.204114+0000","last_became_peered":"2024-09-13T17:58:43.204114+0000","last_unstale":"2024-09-13T17:58:43.557145+0000","last_undegraded":"2024-09-13T17:58:43.557145+0000","last_fullsized":"2024-09-13T17:58:43.557145+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:49.999180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.558580+0000","last_change":"2024-09-13T17:58:43.551327+0000","last_active":"2024-09-13T17:58:43.558580+0000","last_peered":"2024-09-13T17:58:43.558580+0000","last_clean":"2024-09-13T17:58:43.558580+0000","last_became_active":"2024-09-13T17:58:43.550703+0000","last_became_peered":"2024-09-13T17:58:43.550703+0000","last_unstale":"2024-09-13T17:58:43.558580+0000","last_undegraded":"2024-09-13T17:58:43.558580+0000","last_fullsized":"2024-09-13T17:58:43.558580+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:02.733199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.541532+0000","last_change":"2024-09-13T17:58:43.208645+0000","last_active":"2024-09-13T17:58:43.541532+0000","last_peered":"2024-09-13T17:58:43.541532+0000","last_clean":"2024-09-13T17:58:43.541532+0000","last_became_active":"2024-09-13T17:58:43.208363+0000","last_became_peered":"2024-09-13T17:58:43.208363+0000","last_unstale":"2024-09-13T17:58:43.541532+0000","last_undegraded":"2024-09-13T17:58:43.541532+0000","last_fullsized":"2024-09-13T17:58:43.541532+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:45.437304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563579+0000","last_change":"2024-09-13T17:58:43.557005+0000","last_active":"2024-09-13T17:58:43.563579+0000","last_peered":"2024-09-13T17:58:43.563579+0000","last_clean":"2024-09-13T17:58:43.563579+0000","last_became_active":"2024-09-13T17:58:43.556759+0000","last_became_peered":"2024-09-13T17:58:43.556759+0000","last_unstale":"2024-09-13T17:58:43.563579+0000","last_undegraded":"2024-09-13T17:58:43.563579+0000","last_fullsized":"2024-09-13T17:58:43.563579+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:55:08.536744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556273+0000","last_change":"2024-09-13T17:58:43.207250+0000","last_active":"2024-09-13T17:58:43.556273+0000","last_peered":"2024-09-13T17:58:43.556273+0000","last_clean":"2024-09-13T17:58:43.556273+0000","last_became_active":"2024-09-13T17:58:43.207002+0000","last_became_peered":"2024-09-13T17:58:43.207002+0000","last_unstale":"2024-09-13T17:58:43.556273+0000","last_undegraded":"2024-09-13T17:58:43.556273+0000","last_fullsized":"2024-09-13T17:58:43.556273+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:42:49.061619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562141+0000","last_change":"2024-09-13T17:58:43.557704+0000","last_active":"2024-09-13T17:58:43.562141+0000","last_peered":"2024-09-13T17:58:43.562141+0000","last_clean":"2024-09-13T17:58:43.562141+0000","last_became_active":"2024-09-13T17:58:43.557442+0000","last_became_peered":"2024-09-13T17:58:43.557442+0000","last_unstale":"2024-09-13T17:58:43.562141+0000","last_undegraded":"2024-09-13T17:58:43.562141+0000","last_fullsized":"2024-09-13T17:58:43.562141+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:26.374764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.564941+0000","last_change":"2024-09-13T17:58:43.209352+0000","last_active":"2024-09-13T17:58:43.564941+0000","last_peered":"2024-09-13T17:58:43.564941+0000","last_clean":"2024-09-13T17:58:43.564941+0000","last_became_active":"2024-09-13T17:58:43.208719+0000","last_became_peered":"2024-09-13T17:58:43.208719+0000","last_unstale":"2024-09-13T17:58:43.564941+0000","last_undegraded":"2024-09-13T17:58:43.564941+0000","last_fullsized":"2024-09-13T17:58:43.564941+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:32.197469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.564399+0000","last_change":"2024-09-13T17:58:43.559970+0000","last_active":"2024-09-13T17:58:43.564399+0000","last_peered":"2024-09-13T17:58:43.564399+0000","last_clean":"2024-09-13T17:58:43.564399+0000","last_became_active":"2024-09-13T17:58:43.559741+0000","last_became_peered":"2024-09-13T17:58:43.559741+0000","last_unstale":"2024-09-13T17:58:43.564399+0000","last_undegraded":"2024-09-13T17:58:43.564399+0000","last_fullsized":"2024-09-13T17:58:43.564399+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:59.644737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565308+0000","last_change":"2024-09-13T17:58:43.203284+0000","last_active":"2024-09-13T17:58:43.565308+0000","last_peered":"2024-09-13T17:58:43.565308+0000","last_clean":"2024-09-13T17:58:43.565308+0000","last_became_active":"2024-09-13T17:58:43.203011+0000","last_became_peered":"2024-09-13T17:58:43.203011+0000","last_unstale":"2024-09-13T17:58:43.565308+0000","last_undegraded":"2024-09-13T17:58:43.565308+0000","last_fullsized":"2024-09-13T17:58:43.565308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:45.437304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.561572+0000","last_change":"2024-09-13T17:58:43.553143+0000","last_active":"2024-09-13T17:58:43.561572+0000","last_peered":"2024-09-13T17:58:43.561572+0000","last_clean":"2024-09-13T17:58:43.561572+0000","last_became_active":"2024-09-13T17:58:43.551929+0000","last_became_peered":"2024-09-13T17:58:43.551929+0000","last_unstale":"2024-09-13T17:58:43.561572+0000","last_undegraded":"2024-09-13T17:58:43.561572+0000","last_fullsized":"2024-09-13T17:58:43.561572+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:58:17.054244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.564889+0000","last_change":"2024-09-13T17:58:43.204148+0000","last_active":"2024-09-13T17:58:43.564889+0000","last_peered":"2024-09-13T17:58:43.564889+0000","last_clean":"2024-09-13T17:58:43.564889+0000","last_became_active":"2024-09-13T17:58:43.203749+0000","last_became_peered":"2024-09-13T17:58:43.203749+0000","last_unstale":"2024-09-13T17:58:43.564889+0000","last_undegraded":"2024-09-13T17:58:43.564889+0000","last_fullsized":"2024-09-13T17:58:43.564889+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:54:40.969092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565614+0000","last_change":"2024-09-13T17:58:43.557062+0000","last_active":"2024-09-13T17:58:43.565614+0000","last_peered":"2024-09-13T17:58:43.565614+0000","last_clean":"2024-09-13T17:58:43.565614+0000","last_became_active":"2024-09-13T17:58:43.556785+0000","last_became_peered":"2024-09-13T17:58:43.556785+0000","last_unstale":"2024-09-13T17:58:43.565614+0000","last_undegraded":"2024-09-13T17:58:43.565614+0000","last_fullsized":"2024-09-13T17:58:43.565614+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:58:17.054244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563954+0000","last_change":"2024-09-13T17:58:43.203851+0000","last_active":"2024-09-13T17:58:43.563954+0000","last_peered":"2024-09-13T17:58:43.563954+0000","last_clean":"2024-09-13T17:58:43.563954+0000","last_became_active":"2024-09-13T17:58:43.203583+0000","last_became_peered":"2024-09-13T17:58:43.203583+0000","last_unstale":"2024-09-13T17:58:43.563954+0000","last_undegraded":"2024-09-13T17:58:43.563954+0000","last_fullsized":"2024-09-13T17:58:43.563954+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:24:30.308072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.559377+0000","last_change":"2024-09-13T17:58:43.557292+0000","last_active":"2024-09-13T17:58:43.559377+0000","last_peered":"2024-09-13T17:58:43.559377+0000","last_clean":"2024-09-13T17:58:43.559377+0000","last_became_active":"2024-09-13T17:58:43.556994+0000","last_became_peered":"2024-09-13T17:58:43.556994+0000","last_unstale":"2024-09-13T17:58:43.559377+0000","last_undegraded":"2024-09-13T17:58:43.559377+0000","last_fullsized":"2024-09-13T17:58:43.559377+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:28.859129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.542156+0000","last_change":"2024-09-13T17:58:43.207290+0000","last_active":"2024-09-13T17:58:43.542156+0000","last_peered":"2024-09-13T17:58:43.542156+0000","last_clean":"2024-09-13T17:58:43.542156+0000","last_became_active":"2024-09-13T17:58:43.207041+0000","last_became_peered":"2024-09-13T17:58:43.207041+0000","last_unstale":"2024-09-13T17:58:43.542156+0000","last_undegraded":"2024-09-13T17:58:43.542156+0000","last_fullsized":"2024-09-13T17:58:43.542156+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:25:55.280446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565003+0000","last_change":"2024-09-13T17:58:43.556450+0000","last_active":"2024-09-13T17:58:43.565003+0000","last_peered":"2024-09-13T17:58:43.565003+0000","last_clean":"2024-09-13T17:58:43.565003+0000","last_became_active":"2024-09-13T17:58:43.556203+0000","last_became_peered":"2024-09-13T17:58:43.556203+0000","last_unstale":"2024-09-13T17:58:43.565003+0000","last_undegraded":"2024-09-13T17:58:43.565003+0000","last_fullsized":"2024-09-13T17:58:43.565003+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:17.940157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.559100+0000","last_change":"2024-09-13T17:58:43.201931+0000","last_active":"2024-09-13T17:58:43.559100+0000","last_peered":"2024-09-13T17:58:43.559100+0000","last_clean":"2024-09-13T17:58:43.559100+0000","last_became_active":"2024-09-13T17:58:43.201673+0000","last_became_peered":"2024-09-13T17:58:43.201673+0000","last_unstale":"2024-09-13T17:58:43.559100+0000","last_undegraded":"2024-09-13T17:58:43.559100+0000","last_fullsized":"2024-09-13T17:58:43.559100+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:52:41.765918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.561331+0000","last_change":"2024-09-13T17:58:43.553636+0000","last_active":"2024-09-13T17:58:43.561331+0000","last_peered":"2024-09-13T17:58:43.561331+0000","last_clean":"2024-09-13T17:58:43.561331+0000","last_became_active":"2024-09-13T17:58:43.553401+0000","last_became_peered":"2024-09-13T17:58:43.553401+0000","last_unstale":"2024-09-13T17:58:43.561331+0000","last_undegraded":"2024-09-13T17:58:43.561331+0000","last_fullsized":"2024-09-13T17:58:43.561331+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:14:10.174120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.538145+0000","last_change":"2024-09-13T17:58:43.207231+0000","last_active":"2024-09-13T17:58:43.538145+0000","last_peered":"2024-09-13T17:58:43.538145+0000","last_clean":"2024-09-13T17:58:43.538145+0000","last_became_active":"2024-09-13T17:58:43.206976+0000","last_became_peered":"2024-09-13T17:58:43.206976+0000","last_unstale":"2024-09-13T17:58:43.538145+0000","last_undegraded":"2024-09-13T17:58:43.538145+0000","last_fullsized":"2024-09-13T17:58:43.538145+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:45.857265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563160+0000","last_change":"2024-09-13T17:58:43.557807+0000","last_active":"2024-09-13T17:58:43.563160+0000","last_peered":"2024-09-13T17:58:43.563160+0000","last_clean":"2024-09-13T17:58:43.563160+0000","last_became_active":"2024-09-13T17:58:43.557556+0000","last_became_peered":"2024-09-13T17:58:43.557556+0000","last_unstale":"2024-09-13T17:58:43.563160+0000","last_undegraded":"2024-09-13T17:58:43.563160+0000","last_fullsized":"2024-09-13T17:58:43.563160+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:21.360073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.544596+0000","last_change":"2024-09-13T17:58:43.207362+0000","last_active":"2024-09-13T17:58:43.544596+0000","last_peered":"2024-09-13T17:58:43.544596+0000","last_clean":"2024-09-13T17:58:43.544596+0000","last_became_active":"2024-09-13T17:58:43.207084+0000","last_became_peered":"2024-09-13T17:58:43.207084+0000","last_unstale":"2024-09-13T17:58:43.544596+0000","last_undegraded":"2024-09-13T17:58:43.544596+0000","last_fullsized":"2024-09-13T17:58:43.544596+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:29:11.994579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565544+0000","last_change":"2024-09-13T17:58:43.562119+0000","last_active":"2024-09-13T17:58:43.565544+0000","last_peered":"2024-09-13T17:58:43.565544+0000","last_clean":"2024-09-13T17:58:43.565544+0000","last_became_active":"2024-09-13T17:58:43.561797+0000","last_became_peered":"2024-09-13T17:58:43.561797+0000","last_unstale":"2024-09-13T17:58:43.565544+0000","last_undegraded":"2024-09-13T17:58:43.565544+0000","last_fullsized":"2024-09-13T17:58:43.565544+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:56.388908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.538146+0000","last_change":"2024-09-13T17:58:43.202793+0000","last_active":"2024-09-13T17:58:43.538146+0000","last_peered":"2024-09-13T17:58:43.538146+0000","last_clean":"2024-09-13T17:58:43.538146+0000","last_became_active":"2024-09-13T17:58:43.202433+0000","last_became_peered":"2024-09-13T17:58:43.202433+0000","last_unstale":"2024-09-13T17:58:43.538146+0000","last_undegraded":"2024-09-13T17:58:43.538146+0000","last_fullsized":"2024-09-13T17:58:43.538146+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:52.966270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556979+0000","last_change":"2024-09-13T17:58:43.553237+0000","last_active":"2024-09-13T17:58:43.556979+0000","last_peered":"2024-09-13T17:58:43.556979+0000","last_clean":"2024-09-13T17:58:43.556979+0000","last_became_active":"2024-09-13T17:58:43.552998+0000","last_became_peered":"2024-09-13T17:58:43.552998+0000","last_unstale":"2024-09-13T17:58:43.556979+0000","last_undegraded":"2024-09-13T17:58:43.556979+0000","last_fullsized":"2024-09-13T17:58:43.556979+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:27:21.200370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.560662+0000","last_change":"2024-09-13T17:58:43.202325+0000","last_active":"2024-09-13T17:58:43.560662+0000","last_peered":"2024-09-13T17:58:43.560662+0000","last_clean":"2024-09-13T17:58:43.560662+0000","last_became_active":"2024-09-13T17:58:43.202063+0000","last_became_peered":"2024-09-13T17:58:43.202063+0000","last_unstale":"2024-09-13T17:58:43.560662+0000","last_undegraded":"2024-09-13T17:58:43.560662+0000","last_fullsized":"2024-09-13T17:58:43.560662+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:17:49.426674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563617+0000","last_change":"2024-09-13T17:58:43.559664+0000","last_active":"2024-09-13T17:58:43.563617+0000","last_peered":"2024-09-13T17:58:43.563617+0000","last_clean":"2024-09-13T17:58:43.563617+0000","last_became_active":"2024-09-13T17:58:43.559057+0000","last_became_peered":"2024-09-13T17:58:43.559057+0000","last_unstale":"2024-09-13T17:58:43.563617+0000","last_undegraded":"2024-09-13T17:58:43.563617+0000","last_fullsized":"2024-09-13T17:58:43.563617+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:51.001446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.538570+0000","last_change":"2024-09-13T17:58:43.200345+0000","last_active":"2024-09-13T17:58:43.538570+0000","last_peered":"2024-09-13T17:58:43.538570+0000","last_clean":"2024-09-13T17:58:43.538570+0000","last_became_active":"2024-09-13T17:58:43.199861+0000","last_became_peered":"2024-09-13T17:58:43.199861+0000","last_unstale":"2024-09-13T17:58:43.538570+0000","last_undegraded":"2024-09-13T17:58:43.538570+0000","last_fullsized":"2024-09-13T17:58:43.538570+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:53:30.854824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.558692+0000","last_change":"2024-09-13T17:58:43.553236+0000","last_active":"2024-09-13T17:58:43.558692+0000","last_peered":"2024-09-13T17:58:43.558692+0000","last_clean":"2024-09-13T17:58:43.558692+0000","last_became_active":"2024-09-13T17:58:43.552987+0000","last_became_peered":"2024-09-13T17:58:43.552987+0000","last_unstale":"2024-09-13T17:58:43.558692+0000","last_undegraded":"2024-09-13T17:58:43.558692+0000","last_fullsized":"2024-09-13T17:58:43.558692+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:38:28.086202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.548234+0000","last_change":"2024-09-13T17:58:43.202721+0000","last_active":"2024-09-13T17:58:43.548234+0000","last_peered":"2024-09-13T17:58:43.548234+0000","last_clean":"2024-09-13T17:58:43.548234+0000","last_became_active":"2024-09-13T17:58:43.202455+0000","last_became_peered":"2024-09-13T17:58:43.202455+0000","last_unstale":"2024-09-13T17:58:43.548234+0000","last_undegraded":"2024-09-13T17:58:43.548234+0000","last_fullsized":"2024-09-13T17:58:43.548234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:17:49.426674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562182+0000","last_change":"2024-09-13T17:58:43.557642+0000","last_active":"2024-09-13T17:58:43.562182+0000","last_peered":"2024-09-13T17:58:43.562182+0000","last_clean":"2024-09-13T17:58:43.562182+0000","last_became_active":"2024-09-13T17:58:43.557404+0000","last_became_peered":"2024-09-13T17:58:43.557404+0000","last_unstale":"2024-09-13T17:58:43.562182+0000","last_undegraded":"2024-09-13T17:58:43.562182+0000","last_fullsized":"2024-09-13T17:58:43.562182+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:00.243570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.549051+0000","last_change":"2024-09-13T17:58:43.207652+0000","last_active":"2024-09-13T17:58:43.549051+0000","last_peered":"2024-09-13T17:58:43.549051+0000","last_clean":"2024-09-13T17:58:43.549051+0000","last_became_active":"2024-09-13T17:58:43.207413+0000","last_became_peered":"2024-09-13T17:58:43.207413+0000","last_unstale":"2024-09-13T17:58:43.549051+0000","last_undegraded":"2024-09-13T17:58:43.549051+0000","last_fullsized":"2024-09-13T17:58:43.549051+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:06:09.373502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562468+0000","last_change":"2024-09-13T17:58:43.557958+0000","last_active":"2024-09-13T17:58:43.562468+0000","last_peered":"2024-09-13T17:58:43.562468+0000","last_clean":"2024-09-13T17:58:43.562468+0000","last_became_active":"2024-09-13T17:58:43.557713+0000","last_became_peered":"2024-09-13T17:58:43.557713+0000","last_unstale":"2024-09-13T17:58:43.562468+0000","last_undegraded":"2024-09-13T17:58:43.562468+0000","last_fullsized":"2024-09-13T17:58:43.562468+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:35.154189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556281+0000","last_change":"2024-09-13T17:58:43.204091+0000","last_active":"2024-09-13T17:58:43.556281+0000","last_peered":"2024-09-13T17:58:43.556281+0000","last_clean":"2024-09-13T17:58:43.556281+0000","last_became_active":"2024-09-13T17:58:43.203789+0000","last_became_peered":"2024-09-13T17:58:43.203789+0000","last_unstale":"2024-09-13T17:58:43.556281+0000","last_undegraded":"2024-09-13T17:58:43.556281+0000","last_fullsized":"2024-09-13T17:58:43.556281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:29:11.994579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565982+0000","last_change":"2024-09-13T17:58:43.560106+0000","last_active":"2024-09-13T17:58:43.565982+0000","last_peered":"2024-09-13T17:58:43.565982+0000","last_clean":"2024-09-13T17:58:43.565982+0000","last_became_active":"2024-09-13T17:58:43.559837+0000","last_became_peered":"2024-09-13T17:58:43.559837+0000","last_unstale":"2024-09-13T17:58:43.565982+0000","last_undegraded":"2024-09-13T17:58:43.565982+0000","last_fullsized":"2024-09-13T17:58:43.565982+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:02.733199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.548281+0000","last_change":"2024-09-13T17:58:43.205037+0000","last_active":"2024-09-13T17:58:43.548281+0000","last_peered":"2024-09-13T17:58:43.548281+0000","last_clean":"2024-09-13T17:58:43.548281+0000","last_became_active":"2024-09-13T17:58:43.204761+0000","last_became_peered":"2024-09-13T17:58:43.204761+0000","last_unstale":"2024-09-13T17:58:43.548281+0000","last_undegraded":"2024-09-13T17:58:43.548281+0000","last_fullsized":"2024-09-13T17:58:43.548281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:49.999180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.568031+0000","last_change":"2024-09-13T17:58:43.561359+0000","last_active":"2024-09-13T17:58:43.568031+0000","last_peered":"2024-09-13T17:58:43.568031+0000","last_clean":"2024-09-13T17:58:43.568031+0000","last_became_active":"2024-09-13T17:58:43.561084+0000","last_became_peered":"2024-09-13T17:58:43.561084+0000","last_unstale":"2024-09-13T17:58:43.568031+0000","last_undegraded":"2024-09-13T17:58:43.568031+0000","last_fullsized":"2024-09-13T17:58:43.568031+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:54:41.971358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.544428+0000","last_change":"2024-09-13T17:58:43.206844+0000","last_active":"2024-09-13T17:58:43.544428+0000","last_peered":"2024-09-13T17:58:43.544428+0000","last_clean":"2024-09-13T17:58:43.544428+0000","last_became_active":"2024-09-13T17:58:43.206566+0000","last_became_peered":"2024-09-13T17:58:43.206566+0000","last_unstale":"2024-09-13T17:58:43.544428+0000","last_undegraded":"2024-09-13T17:58:43.544428+0000","last_fullsized":"2024-09-13T17:58:43.544428+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:16.937891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563197+0000","last_change":"2024-09-13T17:58:43.558505+0000","last_active":"2024-09-13T17:58:43.563197+0000","last_peered":"2024-09-13T17:58:43.563197+0000","last_clean":"2024-09-13T17:58:43.563197+0000","last_became_active":"2024-09-13T17:58:43.558249+0000","last_became_peered":"2024-09-13T17:58:43.558249+0000","last_unstale":"2024-09-13T17:58:43.563197+0000","last_undegraded":"2024-09-13T17:58:43.563197+0000","last_fullsized":"2024-09-13T17:58:43.563197+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:38:28.086202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.545315+0000","last_change":"2024-09-13T17:58:43.204271+0000","last_active":"2024-09-13T17:58:43.545315+0000","last_peered":"2024-09-13T17:58:43.545315+0000","last_clean":"2024-09-13T17:58:43.545315+0000","last_became_active":"2024-09-13T17:58:43.204034+0000","last_became_peered":"2024-09-13T17:58:43.204034+0000","last_unstale":"2024-09-13T17:58:43.545315+0000","last_undegraded":"2024-09-13T17:58:43.545315+0000","last_fullsized":"2024-09-13T17:58:43.545315+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:42:49.061619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":182,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.198383+0000","last_change":"2024-09-13T17:57:15.560506+0000","last_active":"2024-09-13T17:58:43.198383+0000","last_peered":"2024-09-13T17:58:43.198383+0000","last_clean":"2024-09-13T17:58:43.198383+0000","last_became_active":"2024-09-13T17:57:15.559612+0000","last_became_peered":"2024-09-13T17:57:15.559612+0000","last_unstale":"2024-09-13T17:58:43.198383+0000","last_undegraded":"2024-09-13T17:58:43.198383+0000","last_fullsized":"2024-09-13T17:58:43.198383+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_clean_scrub_stamp":"2024-09-13T17:57:14.317378+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:16:27.663312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":770768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.52100000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61499999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.499}]},{"osd":4,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.629}]},{"osd":5,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.64800000000000002}]},{"osd":6,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54800000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":188077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":1.0109999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.626}]},{"osd":3,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":1.0800000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.65700000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":4.2210000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":1.621},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.436}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":187607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.376},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.39900000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.44}]},{"osd":2,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.624}]},{"osd":4,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.61099999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.55700000000000005}]},{"osd":7,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.65100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":188099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.37}]},{"osd":1,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.498}]},{"osd":3,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.56299999999999994}]},{"osd":5,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.53600000000000003}]},{"osd":6,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.60299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":189093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.441},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.623}]},{"osd":6,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.58999999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.249},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.318}]},{"osd":1,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.30399999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.34499999999999997}]},{"osd":4,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.52800000000000002}]},{"osd":6,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.51200000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.629}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":187585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.35499999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.311}]},{"osd":3,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.29999999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.76400000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.58899999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.56699999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1032192,"data_stored":776878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.48399999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.443}]},{"osd":3,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.47199999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.90200000000000002}]},{"osd":5,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.876}]},{"osd":6,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.88900000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.92100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:58:46.463 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:58:46.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.591+0000 7f28e8ea3640 1 Processor -- start 2024-09-13T17:58:46.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.591+0000 7f28e8ea3640 1 -- start start 2024-09-13T17:58:46.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.592+0000 7f28e8ea3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f28e415c6e0 0x7f28e40b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.592+0000 7f28e8ea3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28e40d07b0 0x7f28e40b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.592+0000 7f28e8ea3640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e40d1840 con 0x7f28e40ccb20 2024-09-13T17:58:46.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.592+0000 7f28e8ea3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e406bd20 con 0x7f28e415c6e0 2024-09-13T17:58:46.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.592+0000 7f28e8ea3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e40d0c30 con 0x7f28e40d07b0 2024-09-13T17:58:46.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.592+0000 7f28e2d9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f28e40ccb20 0x7f28e4160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58160/0 (socket says 172.21.15.111:58160) 2024-09-13T17:58:46.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.592+0000 7f28e2d9d640 1 -- 172.21.15.111:0/440132270 learned_addr learned my addr 172.21.15.111:0/440132270 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:46.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.593+0000 7f28e259c640 1 --2- 172.21.15.111:0/440132270 >> v2:172.21.15.188:3300/0 conn(0x7f28e415c6e0 0x7f28e40b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.593+0000 7f28e1d9b640 1 --2- 172.21.15.111:0/440132270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28e40d07b0 0x7f28e40b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.593+0000 7f28e159a640 1 -- 172.21.15.111:0/440132270 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035242977 0 0) 0x7f28e40d1840 con 0x7f28e40ccb20 2024-09-13T17:58:46.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.593+0000 7f28e159a640 1 -- 172.21.15.111:0/440132270 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28cc003520 con 0x7f28e40ccb20 2024-09-13T17:58:46.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.593+0000 7f28e259c640 1 -- 172.21.15.111:0/440132270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28e40d07b0 msgr2=0x7f28e40b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:46.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.593+0000 7f28e259c640 1 --2- 172.21.15.111:0/440132270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28e40d07b0 0x7f28e40b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.593+0000 7f28e259c640 1 -- 172.21.15.111:0/440132270 >> v1:172.21.15.111:6789/0 conn(0x7f28e40ccb20 legacy=0x7f28e4160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.593+0000 7f28e259c640 1 -- 172.21.15.111:0/440132270 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e40d0f10 con 0x7f28e415c6e0 2024-09-13T17:58:46.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.593+0000 7f28e1d9b640 1 --2- 172.21.15.111:0/440132270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28e40d07b0 0x7f28e40b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:46.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.594+0000 7f28e259c640 1 --2- 172.21.15.111:0/440132270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28e415c6e0 0x7f28e40b3f50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009d90 tx=0x7f28d402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d94a714b6781d1b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:46.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.594+0000 7f28e159a640 1 -- 172.21.15.111:0/440132270 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d4031600 con 0x7f28e415c6e0 2024-09-13T17:58:46.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.594+0000 7f28e159a640 1 -- 172.21.15.111:0/440132270 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d402d050 con 0x7f28e415c6e0 2024-09-13T17:58:46.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.594+0000 7f28e159a640 1 -- 172.21.15.111:0/440132270 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d4031960 con 0x7f28e415c6e0 2024-09-13T17:58:46.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.594+0000 7f28e8ea3640 1 -- 172.21.15.111:0/440132270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28e415c6e0 msgr2=0x7f28e40b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.594+0000 7f28e8ea3640 1 --2- 172.21.15.111:0/440132270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28e415c6e0 0x7f28e40b3f50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009d90 tx=0x7f28d402a5d0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.594+0000 7f28e8ea3640 1 -- 172.21.15.111:0/440132270 shutdown_connections 2024-09-13T17:58:46.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.595+0000 7f28e8ea3640 1 --2- 172.21.15.111:0/440132270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28e40d07b0 0x7f28e40b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.595+0000 7f28e8ea3640 1 --2- 172.21.15.111:0/440132270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28e415c6e0 0x7f28e40b3f50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.595+0000 7f28e8ea3640 1 -- 172.21.15.111:0/440132270 >> 172.21.15.111:0/440132270 conn(0x7f28e4089f60 msgr2=0x7f28e40d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:46.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.595+0000 7f28e8ea3640 1 -- 172.21.15.111:0/440132270 shutdown_connections 2024-09-13T17:58:46.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.595+0000 7f28e8ea3640 1 -- 172.21.15.111:0/440132270 wait complete. 2024-09-13T17:58:46.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.596+0000 7f28e8ea3640 1 Processor -- start 2024-09-13T17:58:46.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.596+0000 7f28e8ea3640 1 -- start start 2024-09-13T17:58:46.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e8ea3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28e40ccb20 0x7f28e41599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e8ea3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28e40d07b0 0x7f28e40b7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e8ea3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28e415c6e0 0x7f28e415dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e8ea3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e415d7c0 con 0x7f28e415c6e0 2024-09-13T17:58:46.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e8ea3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e415cbf0 con 0x7f28e40ccb20 2024-09-13T17:58:46.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e8ea3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e415ce90 con 0x7f28e40d07b0 2024-09-13T17:58:46.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e2d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28e415c6e0 0x7f28e415dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e2d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28e415c6e0 0x7f28e415dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40268/0 (socket says 172.21.15.111:40268) 2024-09-13T17:58:46.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e2d9d640 1 -- 172.21.15.111:0/1892649197 learned_addr learned my addr 172.21.15.111:0/1892649197 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:46.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e1d9b640 1 --2- 172.21.15.111:0/1892649197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28e40d07b0 0x7f28e40b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e259c640 1 --2- 172.21.15.111:0/1892649197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28e40ccb20 0x7f28e41599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e2d9d640 1 -- 172.21.15.111:0/1892649197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28e40d07b0 msgr2=0x7f28e40b7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e2d9d640 1 --2- 172.21.15.111:0/1892649197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28e40d07b0 0x7f28e40b7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e2d9d640 1 -- 172.21.15.111:0/1892649197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28e40ccb20 msgr2=0x7f28e41599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e2d9d640 1 --2- 172.21.15.111:0/1892649197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28e40ccb20 0x7f28e41599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e2d9d640 1 -- 172.21.15.111:0/1892649197 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d8009400 con 0x7f28e415c6e0 2024-09-13T17:58:46.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e2d9d640 1 --2- 172.21.15.111:0/1892649197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28e415c6e0 0x7f28e415dc50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f28dc001eb0 tx=0x7f28dc00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:46.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28d2ffd640 1 -- 172.21.15.111:0/1892649197 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc005ce0 con 0x7f28e415c6e0 2024-09-13T17:58:46.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28d2ffd640 1 -- 172.21.15.111:0/1892649197 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc005e40 con 0x7f28e415c6e0 2024-09-13T17:58:46.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e8ea3640 1 -- 172.21.15.111:0/1892649197 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d4009a70 con 0x7f28e415c6e0 2024-09-13T17:58:46.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28d2ffd640 1 -- 172.21.15.111:0/1892649197 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc0072d0 con 0x7f28e415c6e0 2024-09-13T17:58:46.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.597+0000 7f28e8ea3640 1 -- 172.21.15.111:0/1892649197 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e415d450 con 0x7f28e415c6e0 2024-09-13T17:58:46.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.598+0000 7f28e8ea3640 1 -- 172.21.15.111:0/1892649197 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e415d930 con 0x7f28e415c6e0 2024-09-13T17:58:46.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.599+0000 7f28d2ffd640 1 -- 172.21.15.111:0/1892649197 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f28dc0074f0 con 0x7f28e415c6e0 2024-09-13T17:58:46.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.599+0000 7f28d2ffd640 1 --2- 172.21.15.111:0/1892649197 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f28c406ac00 0x7f28c406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.599+0000 7f28d2ffd640 1 -- 172.21.15.111:0/1892649197 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f28dc088880 con 0x7f28e415c6e0 2024-09-13T17:58:46.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.599+0000 7f28e259c640 1 --2- 172.21.15.111:0/1892649197 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f28c406ac00 0x7f28c406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.600+0000 7f28e259c640 1 --2- 172.21.15.111:0/1892649197 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f28c406ac00 0x7f28c406d0c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f28d4034000 tx=0x7f28d402f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:46.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.603+0000 7f28d2ffd640 1 -- 172.21.15.111:0/1892649197 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f28dc053a60 con 0x7f28e415c6e0 2024-09-13T17:58:46.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.761+0000 7f28e8ea3640 1 -- 172.21.15.111:0/1892649197 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f28e40d14d0 con 0x7f28c406ac00 2024-09-13T17:58:46.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.767+0000 7f28d2ffd640 1 -- 172.21.15.111:0/1892649197 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289554 (secure 0 0 0) 0x7f28d4009450 con 0x7f28c406ac00 2024-09-13T17:58:46.773 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:46.774 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-09-13T17:58:46.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.771+0000 7f28e8ea3640 1 -- 172.21.15.111:0/1892649197 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f28c406ac00 msgr2=0x7f28c406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.771+0000 7f28e8ea3640 1 --2- 172.21.15.111:0/1892649197 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f28c406ac00 0x7f28c406d0c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f28d4034000 tx=0x7f28d402f000 comp rx=0 tx=0).stop 2024-09-13T17:58:46.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.771+0000 7f28e8ea3640 1 -- 172.21.15.111:0/1892649197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28e415c6e0 msgr2=0x7f28e415dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.771+0000 7f28e8ea3640 1 --2- 172.21.15.111:0/1892649197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28e415c6e0 0x7f28e415dc50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f28dc001eb0 tx=0x7f28dc00c820 comp rx=0 tx=0).stop 2024-09-13T17:58:46.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.771+0000 7f28e8ea3640 1 -- 172.21.15.111:0/1892649197 shutdown_connections 2024-09-13T17:58:46.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.771+0000 7f28e8ea3640 1 --2- 172.21.15.111:0/1892649197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28e415c6e0 0x7f28e415dc50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.771+0000 7f28e8ea3640 1 --2- 172.21.15.111:0/1892649197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28e40d07b0 0x7f28e40b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.771+0000 7f28e8ea3640 1 --2- 172.21.15.111:0/1892649197 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f28c406ac00 0x7f28c406d0c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.771+0000 7f28e8ea3640 1 --2- 172.21.15.111:0/1892649197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28e40ccb20 0x7f28e41599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.771+0000 7f28e8ea3640 1 -- 172.21.15.111:0/1892649197 >> 172.21.15.111:0/1892649197 conn(0x7f28e4089f60 msgr2=0x7f28e415ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:46.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.771+0000 7f28e8ea3640 1 -- 172.21.15.111:0/1892649197 shutdown_connections 2024-09-13T17:58:46.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.771+0000 7f28e8ea3640 1 -- 172.21.15.111:0/1892649197 wait complete. 2024-09-13T17:58:46.785 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-09-13T17:58:46.282895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220816,"kb_used_data":4600,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753904,"statfs":{"total":773094113280,"available":772867997696,"internally_reserved":0,"allocated":4710400,"data_stored":2668507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.099346"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562205+0000","last_change":"2024-09-13T17:58:43.556984+0000","last_active":"2024-09-13T17:58:43.562205+0000","last_peered":"2024-09-13T17:58:43.562205+0000","last_clean":"2024-09-13T17:58:43.562205+0000","last_became_active":"2024-09-13T17:58:43.556176+0000","last_became_peered":"2024-09-13T17:58:43.556176+0000","last_unstale":"2024-09-13T17:58:43.562205+0000","last_undegraded":"2024-09-13T17:58:43.562205+0000","last_fullsized":"2024-09-13T17:58:43.562205+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:10.713520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.564328+0000","last_change":"2024-09-13T17:58:43.560341+0000","last_active":"2024-09-13T17:58:43.564328+0000","last_peered":"2024-09-13T17:58:43.564328+0000","last_clean":"2024-09-13T17:58:43.564328+0000","last_became_active":"2024-09-13T17:58:43.560087+0000","last_became_peered":"2024-09-13T17:58:43.560087+0000","last_unstale":"2024-09-13T17:58:43.564328+0000","last_undegraded":"2024-09-13T17:58:43.564328+0000","last_fullsized":"2024-09-13T17:58:43.564328+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:21.360073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562070+0000","last_change":"2024-09-13T17:58:43.556764+0000","last_active":"2024-09-13T17:58:43.562070+0000","last_peered":"2024-09-13T17:58:43.562070+0000","last_clean":"2024-09-13T17:58:43.562070+0000","last_became_active":"2024-09-13T17:58:43.556351+0000","last_became_peered":"2024-09-13T17:58:43.556351+0000","last_unstale":"2024-09-13T17:58:43.562070+0000","last_undegraded":"2024-09-13T17:58:43.562070+0000","last_fullsized":"2024-09-13T17:58:43.562070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:37.690020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556882+0000","last_change":"2024-09-13T17:58:43.552094+0000","last_active":"2024-09-13T17:58:43.556882+0000","last_peered":"2024-09-13T17:58:43.556882+0000","last_clean":"2024-09-13T17:58:43.556882+0000","last_became_active":"2024-09-13T17:58:43.551589+0000","last_became_peered":"2024-09-13T17:58:43.551589+0000","last_unstale":"2024-09-13T17:58:43.556882+0000","last_undegraded":"2024-09-13T17:58:43.556882+0000","last_fullsized":"2024-09-13T17:58:43.556882+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:55:08.536744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.557015+0000","last_change":"2024-09-13T17:58:43.552157+0000","last_active":"2024-09-13T17:58:43.557015+0000","last_peered":"2024-09-13T17:58:43.557015+0000","last_clean":"2024-09-13T17:58:43.557015+0000","last_became_active":"2024-09-13T17:58:43.551818+0000","last_became_peered":"2024-09-13T17:58:43.551818+0000","last_unstale":"2024-09-13T17:58:43.557015+0000","last_undegraded":"2024-09-13T17:58:43.557015+0000","last_fullsized":"2024-09-13T17:58:43.557015+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:18.351367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556856+0000","last_change":"2024-09-13T17:58:43.551579+0000","last_active":"2024-09-13T17:58:43.556856+0000","last_peered":"2024-09-13T17:58:43.556856+0000","last_clean":"2024-09-13T17:58:43.556856+0000","last_became_active":"2024-09-13T17:58:43.551338+0000","last_became_peered":"2024-09-13T17:58:43.551338+0000","last_unstale":"2024-09-13T17:58:43.556856+0000","last_undegraded":"2024-09-13T17:58:43.556856+0000","last_fullsized":"2024-09-13T17:58:43.556856+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:14:10.174120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.564642+0000","last_change":"2024-09-13T17:58:43.561726+0000","last_active":"2024-09-13T17:58:43.564642+0000","last_peered":"2024-09-13T17:58:43.564642+0000","last_clean":"2024-09-13T17:58:43.564642+0000","last_became_active":"2024-09-13T17:58:43.560117+0000","last_became_peered":"2024-09-13T17:58:43.560117+0000","last_unstale":"2024-09-13T17:58:43.564642+0000","last_undegraded":"2024-09-13T17:58:43.564642+0000","last_fullsized":"2024-09-13T17:58:43.564642+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:02.733199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.558786+0000","last_change":"2024-09-13T17:58:43.552540+0000","last_active":"2024-09-13T17:58:43.558786+0000","last_peered":"2024-09-13T17:58:43.558786+0000","last_clean":"2024-09-13T17:58:43.558786+0000","last_became_active":"2024-09-13T17:58:43.552301+0000","last_became_peered":"2024-09-13T17:58:43.552301+0000","last_unstale":"2024-09-13T17:58:43.558786+0000","last_undegraded":"2024-09-13T17:58:43.558786+0000","last_fullsized":"2024-09-13T17:58:43.558786+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:17.159734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565307+0000","last_change":"2024-09-13T17:58:43.563956+0000","last_active":"2024-09-13T17:58:43.565307+0000","last_peered":"2024-09-13T17:58:43.565307+0000","last_clean":"2024-09-13T17:58:43.565307+0000","last_became_active":"2024-09-13T17:58:43.563691+0000","last_became_peered":"2024-09-13T17:58:43.563691+0000","last_unstale":"2024-09-13T17:58:43.565307+0000","last_undegraded":"2024-09-13T17:58:43.565307+0000","last_fullsized":"2024-09-13T17:58:43.565307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:35.154189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.568524+0000","last_change":"2024-09-13T17:58:43.564124+0000","last_active":"2024-09-13T17:58:43.568524+0000","last_peered":"2024-09-13T17:58:43.568524+0000","last_clean":"2024-09-13T17:58:43.568524+0000","last_became_active":"2024-09-13T17:58:43.562377+0000","last_became_peered":"2024-09-13T17:58:43.562377+0000","last_unstale":"2024-09-13T17:58:43.568524+0000","last_undegraded":"2024-09-13T17:58:43.568524+0000","last_fullsized":"2024-09-13T17:58:43.568524+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:00.243570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.561676+0000","last_change":"2024-09-13T17:58:43.553731+0000","last_active":"2024-09-13T17:58:43.561676+0000","last_peered":"2024-09-13T17:58:43.561676+0000","last_clean":"2024-09-13T17:58:43.561676+0000","last_became_active":"2024-09-13T17:58:43.553379+0000","last_became_peered":"2024-09-13T17:58:43.553379+0000","last_unstale":"2024-09-13T17:58:43.561676+0000","last_undegraded":"2024-09-13T17:58:43.561676+0000","last_fullsized":"2024-09-13T17:58:43.561676+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:14:10.174120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.561365+0000","last_change":"2024-09-13T17:58:43.555451+0000","last_active":"2024-09-13T17:58:43.561365+0000","last_peered":"2024-09-13T17:58:43.561365+0000","last_clean":"2024-09-13T17:58:43.561365+0000","last_became_active":"2024-09-13T17:58:43.555185+0000","last_became_peered":"2024-09-13T17:58:43.555185+0000","last_unstale":"2024-09-13T17:58:43.561365+0000","last_undegraded":"2024-09-13T17:58:43.561365+0000","last_fullsized":"2024-09-13T17:58:43.561365+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:27:21.200370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563761+0000","last_change":"2024-09-13T17:58:43.561000+0000","last_active":"2024-09-13T17:58:43.563761+0000","last_peered":"2024-09-13T17:58:43.563761+0000","last_clean":"2024-09-13T17:58:43.563761+0000","last_became_active":"2024-09-13T17:58:43.560465+0000","last_became_peered":"2024-09-13T17:58:43.560465+0000","last_unstale":"2024-09-13T17:58:43.563761+0000","last_undegraded":"2024-09-13T17:58:43.563761+0000","last_fullsized":"2024-09-13T17:58:43.563761+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:19.494551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565965+0000","last_change":"2024-09-13T17:58:43.558447+0000","last_active":"2024-09-13T17:58:43.565965+0000","last_peered":"2024-09-13T17:58:43.565965+0000","last_clean":"2024-09-13T17:58:43.565965+0000","last_became_active":"2024-09-13T17:58:43.558179+0000","last_became_peered":"2024-09-13T17:58:43.558179+0000","last_unstale":"2024-09-13T17:58:43.565965+0000","last_undegraded":"2024-09-13T17:58:43.565965+0000","last_fullsized":"2024-09-13T17:58:43.565965+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:26.374764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.564616+0000","last_change":"2024-09-13T17:58:43.561536+0000","last_active":"2024-09-13T17:58:43.564616+0000","last_peered":"2024-09-13T17:58:43.564616+0000","last_clean":"2024-09-13T17:58:43.564616+0000","last_became_active":"2024-09-13T17:58:43.555740+0000","last_became_peered":"2024-09-13T17:58:43.555740+0000","last_unstale":"2024-09-13T17:58:43.564616+0000","last_undegraded":"2024-09-13T17:58:43.564616+0000","last_fullsized":"2024-09-13T17:58:43.564616+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:26.374764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.568644+0000","last_change":"2024-09-13T17:58:43.564778+0000","last_active":"2024-09-13T17:58:43.568644+0000","last_peered":"2024-09-13T17:58:43.568644+0000","last_clean":"2024-09-13T17:58:43.568644+0000","last_became_active":"2024-09-13T17:58:43.562752+0000","last_became_peered":"2024-09-13T17:58:43.562752+0000","last_unstale":"2024-09-13T17:58:43.568644+0000","last_undegraded":"2024-09-13T17:58:43.568644+0000","last_fullsized":"2024-09-13T17:58:43.568644+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:17:50.428940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.568600+0000","last_change":"2024-09-13T17:58:43.564644+0000","last_active":"2024-09-13T17:58:43.568600+0000","last_peered":"2024-09-13T17:58:43.568600+0000","last_clean":"2024-09-13T17:58:43.568600+0000","last_became_active":"2024-09-13T17:58:43.564229+0000","last_became_peered":"2024-09-13T17:58:43.564229+0000","last_unstale":"2024-09-13T17:58:43.568600+0000","last_undegraded":"2024-09-13T17:58:43.568600+0000","last_fullsized":"2024-09-13T17:58:43.568600+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:06:10.375768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.573952+0000","last_change":"2024-09-13T17:58:43.566527+0000","last_active":"2024-09-13T17:58:43.573952+0000","last_peered":"2024-09-13T17:58:43.573952+0000","last_clean":"2024-09-13T17:58:43.573952+0000","last_became_active":"2024-09-13T17:58:43.566254+0000","last_became_peered":"2024-09-13T17:58:43.566254+0000","last_unstale":"2024-09-13T17:58:43.573952+0000","last_undegraded":"2024-09-13T17:58:43.573952+0000","last_fullsized":"2024-09-13T17:58:43.573952+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:06:10.375768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.554797+0000","last_change":"2024-09-13T17:58:43.545630+0000","last_active":"2024-09-13T17:58:43.554797+0000","last_peered":"2024-09-13T17:58:43.554797+0000","last_clean":"2024-09-13T17:58:43.554797+0000","last_became_active":"2024-09-13T17:58:43.545383+0000","last_became_peered":"2024-09-13T17:58:43.545383+0000","last_unstale":"2024-09-13T17:58:43.554797+0000","last_undegraded":"2024-09-13T17:58:43.554797+0000","last_fullsized":"2024-09-13T17:58:43.554797+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:58:38.983402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.573919+0000","last_change":"2024-09-13T17:58:43.566046+0000","last_active":"2024-09-13T17:58:43.573919+0000","last_peered":"2024-09-13T17:58:43.573919+0000","last_clean":"2024-09-13T17:58:43.573919+0000","last_became_active":"2024-09-13T17:58:43.565783+0000","last_became_peered":"2024-09-13T17:58:43.565783+0000","last_unstale":"2024-09-13T17:58:43.573919+0000","last_undegraded":"2024-09-13T17:58:43.573919+0000","last_fullsized":"2024-09-13T17:58:43.573919+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:51.256171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.561379+0000","last_change":"2024-09-13T17:58:43.550420+0000","last_active":"2024-09-13T17:58:43.561379+0000","last_peered":"2024-09-13T17:58:43.561379+0000","last_clean":"2024-09-13T17:58:43.561379+0000","last_became_active":"2024-09-13T17:58:43.549743+0000","last_became_peered":"2024-09-13T17:58:43.549743+0000","last_unstale":"2024-09-13T17:58:43.561379+0000","last_undegraded":"2024-09-13T17:58:43.561379+0000","last_fullsized":"2024-09-13T17:58:43.561379+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:16.379363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565946+0000","last_change":"2024-09-13T17:58:43.557316+0000","last_active":"2024-09-13T17:58:43.565946+0000","last_peered":"2024-09-13T17:58:43.565946+0000","last_clean":"2024-09-13T17:58:43.565946+0000","last_became_active":"2024-09-13T17:58:43.557043+0000","last_became_peered":"2024-09-13T17:58:43.557043+0000","last_unstale":"2024-09-13T17:58:43.565946+0000","last_undegraded":"2024-09-13T17:58:43.565946+0000","last_fullsized":"2024-09-13T17:58:43.565946+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:16.379363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562852+0000","last_change":"2024-09-13T17:58:43.555606+0000","last_active":"2024-09-13T17:58:43.562852+0000","last_peered":"2024-09-13T17:58:43.562852+0000","last_clean":"2024-09-13T17:58:43.562852+0000","last_became_active":"2024-09-13T17:58:43.555364+0000","last_became_peered":"2024-09-13T17:58:43.555364+0000","last_unstale":"2024-09-13T17:58:43.562852+0000","last_undegraded":"2024-09-13T17:58:43.562852+0000","last_fullsized":"2024-09-13T17:58:43.562852+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:35:24.369374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.554926+0000","last_change":"2024-09-13T17:58:43.547702+0000","last_active":"2024-09-13T17:58:43.554926+0000","last_peered":"2024-09-13T17:58:43.554926+0000","last_clean":"2024-09-13T17:58:43.554926+0000","last_became_active":"2024-09-13T17:58:43.547451+0000","last_became_peered":"2024-09-13T17:58:43.547451+0000","last_unstale":"2024-09-13T17:58:43.554926+0000","last_undegraded":"2024-09-13T17:58:43.554926+0000","last_fullsized":"2024-09-13T17:58:43.554926+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:28.862716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.568450+0000","last_change":"2024-09-13T17:58:43.564178+0000","last_active":"2024-09-13T17:58:43.568450+0000","last_peered":"2024-09-13T17:58:43.568450+0000","last_clean":"2024-09-13T17:58:43.568450+0000","last_became_active":"2024-09-13T17:58:43.563907+0000","last_became_peered":"2024-09-13T17:58:43.563907+0000","last_unstale":"2024-09-13T17:58:43.568450+0000","last_undegraded":"2024-09-13T17:58:43.568450+0000","last_fullsized":"2024-09-13T17:58:43.568450+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:46.439570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556929+0000","last_change":"2024-09-13T17:58:43.553462+0000","last_active":"2024-09-13T17:58:43.556929+0000","last_peered":"2024-09-13T17:58:43.556929+0000","last_clean":"2024-09-13T17:58:43.556929+0000","last_became_active":"2024-09-13T17:58:43.553203+0000","last_became_peered":"2024-09-13T17:58:43.553203+0000","last_unstale":"2024-09-13T17:58:43.556929+0000","last_undegraded":"2024-09-13T17:58:43.556929+0000","last_fullsized":"2024-09-13T17:58:43.556929+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:19.494551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565808+0000","last_change":"2024-09-13T17:58:43.561198+0000","last_active":"2024-09-13T17:58:43.565808+0000","last_peered":"2024-09-13T17:58:43.565808+0000","last_clean":"2024-09-13T17:58:43.565808+0000","last_became_active":"2024-09-13T17:58:43.560941+0000","last_became_peered":"2024-09-13T17:58:43.560941+0000","last_unstale":"2024-09-13T17:58:43.565808+0000","last_undegraded":"2024-09-13T17:58:43.565808+0000","last_fullsized":"2024-09-13T17:58:43.565808+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:05:37.687367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.560469+0000","last_change":"2024-09-13T17:58:43.551304+0000","last_active":"2024-09-13T17:58:43.560469+0000","last_peered":"2024-09-13T17:58:43.560469+0000","last_clean":"2024-09-13T17:58:43.560469+0000","last_became_active":"2024-09-13T17:58:43.551048+0000","last_became_peered":"2024-09-13T17:58:43.551048+0000","last_unstale":"2024-09-13T17:58:43.560469+0000","last_undegraded":"2024-09-13T17:58:43.560469+0000","last_fullsized":"2024-09-13T17:58:43.560469+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:10.713520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565459+0000","last_change":"2024-09-13T17:58:43.559805+0000","last_active":"2024-09-13T17:58:43.565459+0000","last_peered":"2024-09-13T17:58:43.565459+0000","last_clean":"2024-09-13T17:58:43.565459+0000","last_became_active":"2024-09-13T17:58:43.559537+0000","last_became_peered":"2024-09-13T17:58:43.559537+0000","last_unstale":"2024-09-13T17:58:43.565459+0000","last_undegraded":"2024-09-13T17:58:43.565459+0000","last_fullsized":"2024-09-13T17:58:43.565459+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:28.859129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565570+0000","last_change":"2024-09-13T17:58:43.561111+0000","last_active":"2024-09-13T17:58:43.565570+0000","last_peered":"2024-09-13T17:58:43.565570+0000","last_clean":"2024-09-13T17:58:43.565570+0000","last_became_active":"2024-09-13T17:58:43.560843+0000","last_became_peered":"2024-09-13T17:58:43.560843+0000","last_unstale":"2024-09-13T17:58:43.565570+0000","last_undegraded":"2024-09-13T17:58:43.565570+0000","last_fullsized":"2024-09-13T17:58:43.565570+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:17:50.428940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563656+0000","last_change":"2024-09-13T17:58:43.557564+0000","last_active":"2024-09-13T17:58:43.563656+0000","last_peered":"2024-09-13T17:58:43.563656+0000","last_clean":"2024-09-13T17:58:43.563656+0000","last_became_active":"2024-09-13T17:58:43.557326+0000","last_became_peered":"2024-09-13T17:58:43.557326+0000","last_unstale":"2024-09-13T17:58:43.563656+0000","last_undegraded":"2024-09-13T17:58:43.563656+0000","last_fullsized":"2024-09-13T17:58:43.563656+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:43:49.563423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556957+0000","last_change":"2024-09-13T17:58:43.555404+0000","last_active":"2024-09-13T17:58:43.556957+0000","last_peered":"2024-09-13T17:58:43.556957+0000","last_clean":"2024-09-13T17:58:43.556957+0000","last_became_active":"2024-09-13T17:58:43.555157+0000","last_became_peered":"2024-09-13T17:58:43.555157+0000","last_unstale":"2024-09-13T17:58:43.556957+0000","last_undegraded":"2024-09-13T17:58:43.556957+0000","last_fullsized":"2024-09-13T17:58:43.556957+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:37.690020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.568609+0000","last_change":"2024-09-13T17:58:43.564464+0000","last_active":"2024-09-13T17:58:43.568609+0000","last_peered":"2024-09-13T17:58:43.568609+0000","last_clean":"2024-09-13T17:58:43.568609+0000","last_became_active":"2024-09-13T17:58:43.564207+0000","last_became_peered":"2024-09-13T17:58:43.564207+0000","last_unstale":"2024-09-13T17:58:43.568609+0000","last_undegraded":"2024-09-13T17:58:43.568609+0000","last_fullsized":"2024-09-13T17:58:43.568609+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.615927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.567240+0000","last_change":"2024-09-13T17:58:43.207136+0000","last_active":"2024-09-13T17:58:43.567240+0000","last_peered":"2024-09-13T17:58:43.567240+0000","last_clean":"2024-09-13T17:58:43.567240+0000","last_became_active":"2024-09-13T17:58:43.206864+0000","last_became_peered":"2024-09-13T17:58:43.206864+0000","last_unstale":"2024-09-13T17:58:43.567240+0000","last_undegraded":"2024-09-13T17:58:43.567240+0000","last_fullsized":"2024-09-13T17:58:43.567240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:22.930494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556954+0000","last_change":"2024-09-13T17:58:43.554041+0000","last_active":"2024-09-13T17:58:43.556954+0000","last_peered":"2024-09-13T17:58:43.556954+0000","last_clean":"2024-09-13T17:58:43.556954+0000","last_became_active":"2024-09-13T17:58:43.553782+0000","last_became_peered":"2024-09-13T17:58:43.553782+0000","last_unstale":"2024-09-13T17:58:43.556954+0000","last_undegraded":"2024-09-13T17:58:43.556954+0000","last_fullsized":"2024-09-13T17:58:43.556954+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:26.363369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562219+0000","last_change":"2024-09-13T17:58:43.203714+0000","last_active":"2024-09-13T17:58:43.562219+0000","last_peered":"2024-09-13T17:58:43.562219+0000","last_clean":"2024-09-13T17:58:43.562219+0000","last_became_active":"2024-09-13T17:58:43.203467+0000","last_became_peered":"2024-09-13T17:58:43.203467+0000","last_unstale":"2024-09-13T17:58:43.562219+0000","last_undegraded":"2024-09-13T17:58:43.562219+0000","last_fullsized":"2024-09-13T17:58:43.562219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:45.437304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565928+0000","last_change":"2024-09-13T17:58:43.560616+0000","last_active":"2024-09-13T17:58:43.565928+0000","last_peered":"2024-09-13T17:58:43.565928+0000","last_clean":"2024-09-13T17:58:43.565928+0000","last_became_active":"2024-09-13T17:58:43.560372+0000","last_became_peered":"2024-09-13T17:58:43.560372+0000","last_unstale":"2024-09-13T17:58:43.565928+0000","last_undegraded":"2024-09-13T17:58:43.565928+0000","last_fullsized":"2024-09-13T17:58:43.565928+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:19.494551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563807+0000","last_change":"2024-09-13T17:58:43.206952+0000","last_active":"2024-09-13T17:58:43.563807+0000","last_peered":"2024-09-13T17:58:43.563807+0000","last_clean":"2024-09-13T17:58:43.563807+0000","last_became_active":"2024-09-13T17:58:43.206620+0000","last_became_peered":"2024-09-13T17:58:43.206620+0000","last_unstale":"2024-09-13T17:58:43.563807+0000","last_undegraded":"2024-09-13T17:58:43.563807+0000","last_fullsized":"2024-09-13T17:58:43.563807+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:29.730687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.564344+0000","last_change":"2024-09-13T17:58:43.560505+0000","last_active":"2024-09-13T17:58:43.564344+0000","last_peered":"2024-09-13T17:58:43.564344+0000","last_clean":"2024-09-13T17:58:43.564344+0000","last_became_active":"2024-09-13T17:58:43.560260+0000","last_became_peered":"2024-09-13T17:58:43.560260+0000","last_unstale":"2024-09-13T17:58:43.564344+0000","last_undegraded":"2024-09-13T17:58:43.564344+0000","last_fullsized":"2024-09-13T17:58:43.564344+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:25.232164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.559708+0000","last_change":"2024-09-13T17:58:43.206701+0000","last_active":"2024-09-13T17:58:43.559708+0000","last_peered":"2024-09-13T17:58:43.559708+0000","last_clean":"2024-09-13T17:58:43.559708+0000","last_became_active":"2024-09-13T17:58:43.206426+0000","last_became_peered":"2024-09-13T17:58:43.206426+0000","last_unstale":"2024-09-13T17:58:43.559708+0000","last_undegraded":"2024-09-13T17:58:43.559708+0000","last_fullsized":"2024-09-13T17:58:43.559708+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:23:25.857306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.550917+0000","last_change":"2024-09-13T17:58:43.546830+0000","last_active":"2024-09-13T17:58:43.550917+0000","last_peered":"2024-09-13T17:58:43.550917+0000","last_clean":"2024-09-13T17:58:43.550917+0000","last_became_active":"2024-09-13T17:58:43.546505+0000","last_became_peered":"2024-09-13T17:58:43.546505+0000","last_unstale":"2024-09-13T17:58:43.550917+0000","last_undegraded":"2024-09-13T17:58:43.550917+0000","last_fullsized":"2024-09-13T17:58:43.550917+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:16.379363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563796+0000","last_change":"2024-09-13T17:58:43.202035+0000","last_active":"2024-09-13T17:58:43.563796+0000","last_peered":"2024-09-13T17:58:43.563796+0000","last_clean":"2024-09-13T17:58:43.563796+0000","last_became_active":"2024-09-13T17:58:43.201759+0000","last_became_peered":"2024-09-13T17:58:43.201759+0000","last_unstale":"2024-09-13T17:58:43.563796+0000","last_undegraded":"2024-09-13T17:58:43.563796+0000","last_fullsized":"2024-09-13T17:58:43.563796+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:27.856863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562140+0000","last_change":"2024-09-13T17:58:43.550273+0000","last_active":"2024-09-13T17:58:43.562140+0000","last_peered":"2024-09-13T17:58:43.562140+0000","last_clean":"2024-09-13T17:58:43.562140+0000","last_became_active":"2024-09-13T17:58:43.549963+0000","last_became_peered":"2024-09-13T17:58:43.549963+0000","last_unstale":"2024-09-13T17:58:43.562140+0000","last_undegraded":"2024-09-13T17:58:43.562140+0000","last_fullsized":"2024-09-13T17:58:43.562140+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:46.192447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.561552+0000","last_change":"2024-09-13T17:58:43.208905+0000","last_active":"2024-09-13T17:58:43.561552+0000","last_peered":"2024-09-13T17:58:43.561552+0000","last_clean":"2024-09-13T17:58:43.561552+0000","last_became_active":"2024-09-13T17:58:43.208640+0000","last_became_peered":"2024-09-13T17:58:43.208640+0000","last_unstale":"2024-09-13T17:58:43.561552+0000","last_undegraded":"2024-09-13T17:58:43.561552+0000","last_fullsized":"2024-09-13T17:58:43.561552+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:22.930494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.566952+0000","last_change":"2024-09-13T17:58:43.562992+0000","last_active":"2024-09-13T17:58:43.566952+0000","last_peered":"2024-09-13T17:58:43.566952+0000","last_clean":"2024-09-13T17:58:43.566952+0000","last_became_active":"2024-09-13T17:58:43.562726+0000","last_became_peered":"2024-09-13T17:58:43.562726+0000","last_unstale":"2024-09-13T17:58:43.566952+0000","last_undegraded":"2024-09-13T17:58:43.566952+0000","last_fullsized":"2024-09-13T17:58:43.566952+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:46.192447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.541437+0000","last_change":"2024-09-13T17:58:43.202053+0000","last_active":"2024-09-13T17:58:43.541437+0000","last_peered":"2024-09-13T17:58:43.541437+0000","last_clean":"2024-09-13T17:58:43.541437+0000","last_became_active":"2024-09-13T17:58:43.201742+0000","last_became_peered":"2024-09-13T17:58:43.201742+0000","last_unstale":"2024-09-13T17:58:43.541437+0000","last_undegraded":"2024-09-13T17:58:43.541437+0000","last_fullsized":"2024-09-13T17:58:43.541437+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:17:49.426674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565524+0000","last_change":"2024-09-13T17:58:43.562062+0000","last_active":"2024-09-13T17:58:43.565524+0000","last_peered":"2024-09-13T17:58:43.565524+0000","last_clean":"2024-09-13T17:58:43.565524+0000","last_became_active":"2024-09-13T17:58:43.561804+0000","last_became_peered":"2024-09-13T17:58:43.561804+0000","last_unstale":"2024-09-13T17:58:43.565524+0000","last_undegraded":"2024-09-13T17:58:43.565524+0000","last_fullsized":"2024-09-13T17:58:43.565524+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:45:22.589848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.548234+0000","last_change":"2024-09-13T17:58:43.206179+0000","last_active":"2024-09-13T17:58:43.548234+0000","last_peered":"2024-09-13T17:58:43.548234+0000","last_clean":"2024-09-13T17:58:43.548234+0000","last_became_active":"2024-09-13T17:58:43.205885+0000","last_became_peered":"2024-09-13T17:58:43.205885+0000","last_unstale":"2024-09-13T17:58:43.548234+0000","last_undegraded":"2024-09-13T17:58:43.548234+0000","last_fullsized":"2024-09-13T17:58:43.548234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:16.937891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556864+0000","last_change":"2024-09-13T17:58:43.551628+0000","last_active":"2024-09-13T17:58:43.556864+0000","last_peered":"2024-09-13T17:58:43.556864+0000","last_clean":"2024-09-13T17:58:43.556864+0000","last_became_active":"2024-09-13T17:58:43.551244+0000","last_became_peered":"2024-09-13T17:58:43.551244+0000","last_unstale":"2024-09-13T17:58:43.556864+0000","last_undegraded":"2024-09-13T17:58:43.556864+0000","last_fullsized":"2024-09-13T17:58:43.556864+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:27.522517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.549822+0000","last_change":"2024-09-13T17:58:43.206844+0000","last_active":"2024-09-13T17:58:43.549822+0000","last_peered":"2024-09-13T17:58:43.549822+0000","last_clean":"2024-09-13T17:58:43.549822+0000","last_became_active":"2024-09-13T17:58:43.206581+0000","last_became_peered":"2024-09-13T17:58:43.206581+0000","last_unstale":"2024-09-13T17:58:43.549822+0000","last_undegraded":"2024-09-13T17:58:43.549822+0000","last_fullsized":"2024-09-13T17:58:43.549822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:43:48.561157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563646+0000","last_change":"2024-09-13T17:58:43.559853+0000","last_active":"2024-09-13T17:58:43.563646+0000","last_peered":"2024-09-13T17:58:43.563646+0000","last_clean":"2024-09-13T17:58:43.563646+0000","last_became_active":"2024-09-13T17:58:43.559406+0000","last_became_peered":"2024-09-13T17:58:43.559406+0000","last_unstale":"2024-09-13T17:58:43.563646+0000","last_undegraded":"2024-09-13T17:58:43.563646+0000","last_fullsized":"2024-09-13T17:58:43.563646+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:23:26.859572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.544992+0000","last_change":"2024-09-13T17:58:43.204768+0000","last_active":"2024-09-13T17:58:43.544992+0000","last_peered":"2024-09-13T17:58:43.544992+0000","last_clean":"2024-09-13T17:58:43.544992+0000","last_became_active":"2024-09-13T17:58:43.204490+0000","last_became_peered":"2024-09-13T17:58:43.204490+0000","last_unstale":"2024-09-13T17:58:43.544992+0000","last_undegraded":"2024-09-13T17:58:43.544992+0000","last_fullsized":"2024-09-13T17:58:43.544992+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:06:09.373502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556744+0000","last_change":"2024-09-13T17:58:43.553057+0000","last_active":"2024-09-13T17:58:43.556744+0000","last_peered":"2024-09-13T17:58:43.556744+0000","last_clean":"2024-09-13T17:58:43.556744+0000","last_became_active":"2024-09-13T17:58:43.552770+0000","last_became_peered":"2024-09-13T17:58:43.552770+0000","last_unstale":"2024-09-13T17:58:43.556744+0000","last_undegraded":"2024-09-13T17:58:43.556744+0000","last_fullsized":"2024-09-13T17:58:43.556744+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:26.363369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.561627+0000","last_change":"2024-09-13T17:58:43.202291+0000","last_active":"2024-09-13T17:58:43.561627+0000","last_peered":"2024-09-13T17:58:43.561627+0000","last_clean":"2024-09-13T17:58:43.561627+0000","last_became_active":"2024-09-13T17:58:43.202001+0000","last_became_peered":"2024-09-13T17:58:43.202001+0000","last_unstale":"2024-09-13T17:58:43.561627+0000","last_undegraded":"2024-09-13T17:58:43.561627+0000","last_fullsized":"2024-09-13T17:58:43.561627+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:25:55.280446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.566785+0000","last_change":"2024-09-13T17:58:43.562408+0000","last_active":"2024-09-13T17:58:43.566785+0000","last_peered":"2024-09-13T17:58:43.566785+0000","last_clean":"2024-09-13T17:58:43.566785+0000","last_became_active":"2024-09-13T17:58:43.562152+0000","last_became_peered":"2024-09-13T17:58:43.562152+0000","last_unstale":"2024-09-13T17:58:43.566785+0000","last_undegraded":"2024-09-13T17:58:43.566785+0000","last_fullsized":"2024-09-13T17:58:43.566785+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:00.243570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.568966+0000","last_change":"2024-09-13T17:58:43.206067+0000","last_active":"2024-09-13T17:58:43.568966+0000","last_peered":"2024-09-13T17:58:43.568966+0000","last_clean":"2024-09-13T17:58:43.568966+0000","last_became_active":"2024-09-13T17:58:43.205791+0000","last_became_peered":"2024-09-13T17:58:43.205791+0000","last_unstale":"2024-09-13T17:58:43.568966+0000","last_undegraded":"2024-09-13T17:58:43.568966+0000","last_fullsized":"2024-09-13T17:58:43.568966+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:23:25.857306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562149+0000","last_change":"2024-09-13T17:58:43.557055+0000","last_active":"2024-09-13T17:58:43.562149+0000","last_peered":"2024-09-13T17:58:43.562149+0000","last_clean":"2024-09-13T17:58:43.562149+0000","last_became_active":"2024-09-13T17:58:43.556793+0000","last_became_peered":"2024-09-13T17:58:43.556793+0000","last_unstale":"2024-09-13T17:58:43.562149+0000","last_undegraded":"2024-09-13T17:58:43.562149+0000","last_fullsized":"2024-09-13T17:58:43.562149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:19.494551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.557145+0000","last_change":"2024-09-13T17:58:43.204485+0000","last_active":"2024-09-13T17:58:43.557145+0000","last_peered":"2024-09-13T17:58:43.557145+0000","last_clean":"2024-09-13T17:58:43.557145+0000","last_became_active":"2024-09-13T17:58:43.204114+0000","last_became_peered":"2024-09-13T17:58:43.204114+0000","last_unstale":"2024-09-13T17:58:43.557145+0000","last_undegraded":"2024-09-13T17:58:43.557145+0000","last_fullsized":"2024-09-13T17:58:43.557145+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:49.999180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.558580+0000","last_change":"2024-09-13T17:58:43.551327+0000","last_active":"2024-09-13T17:58:43.558580+0000","last_peered":"2024-09-13T17:58:43.558580+0000","last_clean":"2024-09-13T17:58:43.558580+0000","last_became_active":"2024-09-13T17:58:43.550703+0000","last_became_peered":"2024-09-13T17:58:43.550703+0000","last_unstale":"2024-09-13T17:58:43.558580+0000","last_undegraded":"2024-09-13T17:58:43.558580+0000","last_fullsized":"2024-09-13T17:58:43.558580+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:02.733199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.541532+0000","last_change":"2024-09-13T17:58:43.208645+0000","last_active":"2024-09-13T17:58:43.541532+0000","last_peered":"2024-09-13T17:58:43.541532+0000","last_clean":"2024-09-13T17:58:43.541532+0000","last_became_active":"2024-09-13T17:58:43.208363+0000","last_became_peered":"2024-09-13T17:58:43.208363+0000","last_unstale":"2024-09-13T17:58:43.541532+0000","last_undegraded":"2024-09-13T17:58:43.541532+0000","last_fullsized":"2024-09-13T17:58:43.541532+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:45.437304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563579+0000","last_change":"2024-09-13T17:58:43.557005+0000","last_active":"2024-09-13T17:58:43.563579+0000","last_peered":"2024-09-13T17:58:43.563579+0000","last_clean":"2024-09-13T17:58:43.563579+0000","last_became_active":"2024-09-13T17:58:43.556759+0000","last_became_peered":"2024-09-13T17:58:43.556759+0000","last_unstale":"2024-09-13T17:58:43.563579+0000","last_undegraded":"2024-09-13T17:58:43.563579+0000","last_fullsized":"2024-09-13T17:58:43.563579+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:55:08.536744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556273+0000","last_change":"2024-09-13T17:58:43.207250+0000","last_active":"2024-09-13T17:58:43.556273+0000","last_peered":"2024-09-13T17:58:43.556273+0000","last_clean":"2024-09-13T17:58:43.556273+0000","last_became_active":"2024-09-13T17:58:43.207002+0000","last_became_peered":"2024-09-13T17:58:43.207002+0000","last_unstale":"2024-09-13T17:58:43.556273+0000","last_undegraded":"2024-09-13T17:58:43.556273+0000","last_fullsized":"2024-09-13T17:58:43.556273+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:42:49.061619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562141+0000","last_change":"2024-09-13T17:58:43.557704+0000","last_active":"2024-09-13T17:58:43.562141+0000","last_peered":"2024-09-13T17:58:43.562141+0000","last_clean":"2024-09-13T17:58:43.562141+0000","last_became_active":"2024-09-13T17:58:43.557442+0000","last_became_peered":"2024-09-13T17:58:43.557442+0000","last_unstale":"2024-09-13T17:58:43.562141+0000","last_undegraded":"2024-09-13T17:58:43.562141+0000","last_fullsized":"2024-09-13T17:58:43.562141+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:26.374764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.564941+0000","last_change":"2024-09-13T17:58:43.209352+0000","last_active":"2024-09-13T17:58:43.564941+0000","last_peered":"2024-09-13T17:58:43.564941+0000","last_clean":"2024-09-13T17:58:43.564941+0000","last_became_active":"2024-09-13T17:58:43.208719+0000","last_became_peered":"2024-09-13T17:58:43.208719+0000","last_unstale":"2024-09-13T17:58:43.564941+0000","last_undegraded":"2024-09-13T17:58:43.564941+0000","last_fullsized":"2024-09-13T17:58:43.564941+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:32.197469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.564399+0000","last_change":"2024-09-13T17:58:43.559970+0000","last_active":"2024-09-13T17:58:43.564399+0000","last_peered":"2024-09-13T17:58:43.564399+0000","last_clean":"2024-09-13T17:58:43.564399+0000","last_became_active":"2024-09-13T17:58:43.559741+0000","last_became_peered":"2024-09-13T17:58:43.559741+0000","last_unstale":"2024-09-13T17:58:43.564399+0000","last_undegraded":"2024-09-13T17:58:43.564399+0000","last_fullsized":"2024-09-13T17:58:43.564399+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:59.644737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565308+0000","last_change":"2024-09-13T17:58:43.203284+0000","last_active":"2024-09-13T17:58:43.565308+0000","last_peered":"2024-09-13T17:58:43.565308+0000","last_clean":"2024-09-13T17:58:43.565308+0000","last_became_active":"2024-09-13T17:58:43.203011+0000","last_became_peered":"2024-09-13T17:58:43.203011+0000","last_unstale":"2024-09-13T17:58:43.565308+0000","last_undegraded":"2024-09-13T17:58:43.565308+0000","last_fullsized":"2024-09-13T17:58:43.565308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:45.437304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.561572+0000","last_change":"2024-09-13T17:58:43.553143+0000","last_active":"2024-09-13T17:58:43.561572+0000","last_peered":"2024-09-13T17:58:43.561572+0000","last_clean":"2024-09-13T17:58:43.561572+0000","last_became_active":"2024-09-13T17:58:43.551929+0000","last_became_peered":"2024-09-13T17:58:43.551929+0000","last_unstale":"2024-09-13T17:58:43.561572+0000","last_undegraded":"2024-09-13T17:58:43.561572+0000","last_fullsized":"2024-09-13T17:58:43.561572+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:58:17.054244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.564889+0000","last_change":"2024-09-13T17:58:43.204148+0000","last_active":"2024-09-13T17:58:43.564889+0000","last_peered":"2024-09-13T17:58:43.564889+0000","last_clean":"2024-09-13T17:58:43.564889+0000","last_became_active":"2024-09-13T17:58:43.203749+0000","last_became_peered":"2024-09-13T17:58:43.203749+0000","last_unstale":"2024-09-13T17:58:43.564889+0000","last_undegraded":"2024-09-13T17:58:43.564889+0000","last_fullsized":"2024-09-13T17:58:43.564889+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:54:40.969092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565614+0000","last_change":"2024-09-13T17:58:43.557062+0000","last_active":"2024-09-13T17:58:43.565614+0000","last_peered":"2024-09-13T17:58:43.565614+0000","last_clean":"2024-09-13T17:58:43.565614+0000","last_became_active":"2024-09-13T17:58:43.556785+0000","last_became_peered":"2024-09-13T17:58:43.556785+0000","last_unstale":"2024-09-13T17:58:43.565614+0000","last_undegraded":"2024-09-13T17:58:43.565614+0000","last_fullsized":"2024-09-13T17:58:43.565614+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:58:17.054244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563954+0000","last_change":"2024-09-13T17:58:43.203851+0000","last_active":"2024-09-13T17:58:43.563954+0000","last_peered":"2024-09-13T17:58:43.563954+0000","last_clean":"2024-09-13T17:58:43.563954+0000","last_became_active":"2024-09-13T17:58:43.203583+0000","last_became_peered":"2024-09-13T17:58:43.203583+0000","last_unstale":"2024-09-13T17:58:43.563954+0000","last_undegraded":"2024-09-13T17:58:43.563954+0000","last_fullsized":"2024-09-13T17:58:43.563954+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:24:30.308072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.559377+0000","last_change":"2024-09-13T17:58:43.557292+0000","last_active":"2024-09-13T17:58:43.559377+0000","last_peered":"2024-09-13T17:58:43.559377+0000","last_clean":"2024-09-13T17:58:43.559377+0000","last_became_active":"2024-09-13T17:58:43.556994+0000","last_became_peered":"2024-09-13T17:58:43.556994+0000","last_unstale":"2024-09-13T17:58:43.559377+0000","last_undegraded":"2024-09-13T17:58:43.559377+0000","last_fullsized":"2024-09-13T17:58:43.559377+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:28.859129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.542156+0000","last_change":"2024-09-13T17:58:43.207290+0000","last_active":"2024-09-13T17:58:43.542156+0000","last_peered":"2024-09-13T17:58:43.542156+0000","last_clean":"2024-09-13T17:58:43.542156+0000","last_became_active":"2024-09-13T17:58:43.207041+0000","last_became_peered":"2024-09-13T17:58:43.207041+0000","last_unstale":"2024-09-13T17:58:43.542156+0000","last_undegraded":"2024-09-13T17:58:43.542156+0000","last_fullsized":"2024-09-13T17:58:43.542156+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:25:55.280446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565003+0000","last_change":"2024-09-13T17:58:43.556450+0000","last_active":"2024-09-13T17:58:43.565003+0000","last_peered":"2024-09-13T17:58:43.565003+0000","last_clean":"2024-09-13T17:58:43.565003+0000","last_became_active":"2024-09-13T17:58:43.556203+0000","last_became_peered":"2024-09-13T17:58:43.556203+0000","last_unstale":"2024-09-13T17:58:43.565003+0000","last_undegraded":"2024-09-13T17:58:43.565003+0000","last_fullsized":"2024-09-13T17:58:43.565003+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:17.940157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.559100+0000","last_change":"2024-09-13T17:58:43.201931+0000","last_active":"2024-09-13T17:58:43.559100+0000","last_peered":"2024-09-13T17:58:43.559100+0000","last_clean":"2024-09-13T17:58:43.559100+0000","last_became_active":"2024-09-13T17:58:43.201673+0000","last_became_peered":"2024-09-13T17:58:43.201673+0000","last_unstale":"2024-09-13T17:58:43.559100+0000","last_undegraded":"2024-09-13T17:58:43.559100+0000","last_fullsized":"2024-09-13T17:58:43.559100+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:52:41.765918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.561331+0000","last_change":"2024-09-13T17:58:43.553636+0000","last_active":"2024-09-13T17:58:43.561331+0000","last_peered":"2024-09-13T17:58:43.561331+0000","last_clean":"2024-09-13T17:58:43.561331+0000","last_became_active":"2024-09-13T17:58:43.553401+0000","last_became_peered":"2024-09-13T17:58:43.553401+0000","last_unstale":"2024-09-13T17:58:43.561331+0000","last_undegraded":"2024-09-13T17:58:43.561331+0000","last_fullsized":"2024-09-13T17:58:43.561331+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:14:10.174120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.538145+0000","last_change":"2024-09-13T17:58:43.207231+0000","last_active":"2024-09-13T17:58:43.538145+0000","last_peered":"2024-09-13T17:58:43.538145+0000","last_clean":"2024-09-13T17:58:43.538145+0000","last_became_active":"2024-09-13T17:58:43.206976+0000","last_became_peered":"2024-09-13T17:58:43.206976+0000","last_unstale":"2024-09-13T17:58:43.538145+0000","last_undegraded":"2024-09-13T17:58:43.538145+0000","last_fullsized":"2024-09-13T17:58:43.538145+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:45.857265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563160+0000","last_change":"2024-09-13T17:58:43.557807+0000","last_active":"2024-09-13T17:58:43.563160+0000","last_peered":"2024-09-13T17:58:43.563160+0000","last_clean":"2024-09-13T17:58:43.563160+0000","last_became_active":"2024-09-13T17:58:43.557556+0000","last_became_peered":"2024-09-13T17:58:43.557556+0000","last_unstale":"2024-09-13T17:58:43.563160+0000","last_undegraded":"2024-09-13T17:58:43.563160+0000","last_fullsized":"2024-09-13T17:58:43.563160+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:21.360073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.544596+0000","last_change":"2024-09-13T17:58:43.207362+0000","last_active":"2024-09-13T17:58:43.544596+0000","last_peered":"2024-09-13T17:58:43.544596+0000","last_clean":"2024-09-13T17:58:43.544596+0000","last_became_active":"2024-09-13T17:58:43.207084+0000","last_became_peered":"2024-09-13T17:58:43.207084+0000","last_unstale":"2024-09-13T17:58:43.544596+0000","last_undegraded":"2024-09-13T17:58:43.544596+0000","last_fullsized":"2024-09-13T17:58:43.544596+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:29:11.994579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565544+0000","last_change":"2024-09-13T17:58:43.562119+0000","last_active":"2024-09-13T17:58:43.565544+0000","last_peered":"2024-09-13T17:58:43.565544+0000","last_clean":"2024-09-13T17:58:43.565544+0000","last_became_active":"2024-09-13T17:58:43.561797+0000","last_became_peered":"2024-09-13T17:58:43.561797+0000","last_unstale":"2024-09-13T17:58:43.565544+0000","last_undegraded":"2024-09-13T17:58:43.565544+0000","last_fullsized":"2024-09-13T17:58:43.565544+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:56.388908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.538146+0000","last_change":"2024-09-13T17:58:43.202793+0000","last_active":"2024-09-13T17:58:43.538146+0000","last_peered":"2024-09-13T17:58:43.538146+0000","last_clean":"2024-09-13T17:58:43.538146+0000","last_became_active":"2024-09-13T17:58:43.202433+0000","last_became_peered":"2024-09-13T17:58:43.202433+0000","last_unstale":"2024-09-13T17:58:43.538146+0000","last_undegraded":"2024-09-13T17:58:43.538146+0000","last_fullsized":"2024-09-13T17:58:43.538146+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:52.966270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556979+0000","last_change":"2024-09-13T17:58:43.553237+0000","last_active":"2024-09-13T17:58:43.556979+0000","last_peered":"2024-09-13T17:58:43.556979+0000","last_clean":"2024-09-13T17:58:43.556979+0000","last_became_active":"2024-09-13T17:58:43.552998+0000","last_became_peered":"2024-09-13T17:58:43.552998+0000","last_unstale":"2024-09-13T17:58:43.556979+0000","last_undegraded":"2024-09-13T17:58:43.556979+0000","last_fullsized":"2024-09-13T17:58:43.556979+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:27:21.200370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.560662+0000","last_change":"2024-09-13T17:58:43.202325+0000","last_active":"2024-09-13T17:58:43.560662+0000","last_peered":"2024-09-13T17:58:43.560662+0000","last_clean":"2024-09-13T17:58:43.560662+0000","last_became_active":"2024-09-13T17:58:43.202063+0000","last_became_peered":"2024-09-13T17:58:43.202063+0000","last_unstale":"2024-09-13T17:58:43.560662+0000","last_undegraded":"2024-09-13T17:58:43.560662+0000","last_fullsized":"2024-09-13T17:58:43.560662+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:17:49.426674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563617+0000","last_change":"2024-09-13T17:58:43.559664+0000","last_active":"2024-09-13T17:58:43.563617+0000","last_peered":"2024-09-13T17:58:43.563617+0000","last_clean":"2024-09-13T17:58:43.563617+0000","last_became_active":"2024-09-13T17:58:43.559057+0000","last_became_peered":"2024-09-13T17:58:43.559057+0000","last_unstale":"2024-09-13T17:58:43.563617+0000","last_undegraded":"2024-09-13T17:58:43.563617+0000","last_fullsized":"2024-09-13T17:58:43.563617+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:51.001446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.538570+0000","last_change":"2024-09-13T17:58:43.200345+0000","last_active":"2024-09-13T17:58:43.538570+0000","last_peered":"2024-09-13T17:58:43.538570+0000","last_clean":"2024-09-13T17:58:43.538570+0000","last_became_active":"2024-09-13T17:58:43.199861+0000","last_became_peered":"2024-09-13T17:58:43.199861+0000","last_unstale":"2024-09-13T17:58:43.538570+0000","last_undegraded":"2024-09-13T17:58:43.538570+0000","last_fullsized":"2024-09-13T17:58:43.538570+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:53:30.854824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.558692+0000","last_change":"2024-09-13T17:58:43.553236+0000","last_active":"2024-09-13T17:58:43.558692+0000","last_peered":"2024-09-13T17:58:43.558692+0000","last_clean":"2024-09-13T17:58:43.558692+0000","last_became_active":"2024-09-13T17:58:43.552987+0000","last_became_peered":"2024-09-13T17:58:43.552987+0000","last_unstale":"2024-09-13T17:58:43.558692+0000","last_undegraded":"2024-09-13T17:58:43.558692+0000","last_fullsized":"2024-09-13T17:58:43.558692+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:38:28.086202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.548234+0000","last_change":"2024-09-13T17:58:43.202721+0000","last_active":"2024-09-13T17:58:43.548234+0000","last_peered":"2024-09-13T17:58:43.548234+0000","last_clean":"2024-09-13T17:58:43.548234+0000","last_became_active":"2024-09-13T17:58:43.202455+0000","last_became_peered":"2024-09-13T17:58:43.202455+0000","last_unstale":"2024-09-13T17:58:43.548234+0000","last_undegraded":"2024-09-13T17:58:43.548234+0000","last_fullsized":"2024-09-13T17:58:43.548234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:17:49.426674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562182+0000","last_change":"2024-09-13T17:58:43.557642+0000","last_active":"2024-09-13T17:58:43.562182+0000","last_peered":"2024-09-13T17:58:43.562182+0000","last_clean":"2024-09-13T17:58:43.562182+0000","last_became_active":"2024-09-13T17:58:43.557404+0000","last_became_peered":"2024-09-13T17:58:43.557404+0000","last_unstale":"2024-09-13T17:58:43.562182+0000","last_undegraded":"2024-09-13T17:58:43.562182+0000","last_fullsized":"2024-09-13T17:58:43.562182+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:00.243570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.549051+0000","last_change":"2024-09-13T17:58:43.207652+0000","last_active":"2024-09-13T17:58:43.549051+0000","last_peered":"2024-09-13T17:58:43.549051+0000","last_clean":"2024-09-13T17:58:43.549051+0000","last_became_active":"2024-09-13T17:58:43.207413+0000","last_became_peered":"2024-09-13T17:58:43.207413+0000","last_unstale":"2024-09-13T17:58:43.549051+0000","last_undegraded":"2024-09-13T17:58:43.549051+0000","last_fullsized":"2024-09-13T17:58:43.549051+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:06:09.373502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.562468+0000","last_change":"2024-09-13T17:58:43.557958+0000","last_active":"2024-09-13T17:58:43.562468+0000","last_peered":"2024-09-13T17:58:43.562468+0000","last_clean":"2024-09-13T17:58:43.562468+0000","last_became_active":"2024-09-13T17:58:43.557713+0000","last_became_peered":"2024-09-13T17:58:43.557713+0000","last_unstale":"2024-09-13T17:58:43.562468+0000","last_undegraded":"2024-09-13T17:58:43.562468+0000","last_fullsized":"2024-09-13T17:58:43.562468+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:35.154189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.556281+0000","last_change":"2024-09-13T17:58:43.204091+0000","last_active":"2024-09-13T17:58:43.556281+0000","last_peered":"2024-09-13T17:58:43.556281+0000","last_clean":"2024-09-13T17:58:43.556281+0000","last_became_active":"2024-09-13T17:58:43.203789+0000","last_became_peered":"2024-09-13T17:58:43.203789+0000","last_unstale":"2024-09-13T17:58:43.556281+0000","last_undegraded":"2024-09-13T17:58:43.556281+0000","last_fullsized":"2024-09-13T17:58:43.556281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:29:11.994579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.565982+0000","last_change":"2024-09-13T17:58:43.560106+0000","last_active":"2024-09-13T17:58:43.565982+0000","last_peered":"2024-09-13T17:58:43.565982+0000","last_clean":"2024-09-13T17:58:43.565982+0000","last_became_active":"2024-09-13T17:58:43.559837+0000","last_became_peered":"2024-09-13T17:58:43.559837+0000","last_unstale":"2024-09-13T17:58:43.565982+0000","last_undegraded":"2024-09-13T17:58:43.565982+0000","last_fullsized":"2024-09-13T17:58:43.565982+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:02.733199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.548281+0000","last_change":"2024-09-13T17:58:43.205037+0000","last_active":"2024-09-13T17:58:43.548281+0000","last_peered":"2024-09-13T17:58:43.548281+0000","last_clean":"2024-09-13T17:58:43.548281+0000","last_became_active":"2024-09-13T17:58:43.204761+0000","last_became_peered":"2024-09-13T17:58:43.204761+0000","last_unstale":"2024-09-13T17:58:43.548281+0000","last_undegraded":"2024-09-13T17:58:43.548281+0000","last_fullsized":"2024-09-13T17:58:43.548281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:49.999180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.568031+0000","last_change":"2024-09-13T17:58:43.561359+0000","last_active":"2024-09-13T17:58:43.568031+0000","last_peered":"2024-09-13T17:58:43.568031+0000","last_clean":"2024-09-13T17:58:43.568031+0000","last_became_active":"2024-09-13T17:58:43.561084+0000","last_became_peered":"2024-09-13T17:58:43.561084+0000","last_unstale":"2024-09-13T17:58:43.568031+0000","last_undegraded":"2024-09-13T17:58:43.568031+0000","last_fullsized":"2024-09-13T17:58:43.568031+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:54:41.971358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.544428+0000","last_change":"2024-09-13T17:58:43.206844+0000","last_active":"2024-09-13T17:58:43.544428+0000","last_peered":"2024-09-13T17:58:43.544428+0000","last_clean":"2024-09-13T17:58:43.544428+0000","last_became_active":"2024-09-13T17:58:43.206566+0000","last_became_peered":"2024-09-13T17:58:43.206566+0000","last_unstale":"2024-09-13T17:58:43.544428+0000","last_undegraded":"2024-09-13T17:58:43.544428+0000","last_fullsized":"2024-09-13T17:58:43.544428+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:16.937891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.563197+0000","last_change":"2024-09-13T17:58:43.558505+0000","last_active":"2024-09-13T17:58:43.563197+0000","last_peered":"2024-09-13T17:58:43.563197+0000","last_clean":"2024-09-13T17:58:43.563197+0000","last_became_active":"2024-09-13T17:58:43.558249+0000","last_became_peered":"2024-09-13T17:58:43.558249+0000","last_unstale":"2024-09-13T17:58:43.563197+0000","last_undegraded":"2024-09-13T17:58:43.563197+0000","last_fullsized":"2024-09-13T17:58:43.563197+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:43.183741+0000","last_clean_scrub_stamp":"2024-09-13T17:58:43.183741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:38:28.086202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.545315+0000","last_change":"2024-09-13T17:58:43.204271+0000","last_active":"2024-09-13T17:58:43.545315+0000","last_peered":"2024-09-13T17:58:43.545315+0000","last_clean":"2024-09-13T17:58:43.545315+0000","last_became_active":"2024-09-13T17:58:43.204034+0000","last_became_peered":"2024-09-13T17:58:43.204034+0000","last_unstale":"2024-09-13T17:58:43.545315+0000","last_undegraded":"2024-09-13T17:58:43.545315+0000","last_fullsized":"2024-09-13T17:58:43.545315+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:58:42.181475+0000","last_clean_scrub_stamp":"2024-09-13T17:58:42.181475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:42:49.061619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":182,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:58:43.198383+0000","last_change":"2024-09-13T17:57:15.560506+0000","last_active":"2024-09-13T17:58:43.198383+0000","last_peered":"2024-09-13T17:58:43.198383+0000","last_clean":"2024-09-13T17:58:43.198383+0000","last_became_active":"2024-09-13T17:57:15.559612+0000","last_became_peered":"2024-09-13T17:57:15.559612+0000","last_unstale":"2024-09-13T17:58:43.198383+0000","last_undegraded":"2024-09-13T17:58:43.198383+0000","last_fullsized":"2024-09-13T17:58:43.198383+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_clean_scrub_stamp":"2024-09-13T17:57:14.317378+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:16:27.663312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":770768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.52100000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61499999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.499}]},{"osd":4,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.629}]},{"osd":5,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.64800000000000002}]},{"osd":6,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54800000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":188077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":1.0109999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.626}]},{"osd":3,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":1.0800000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.65700000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":4.2210000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":1.621},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.436}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":187607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.376},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.39900000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.44}]},{"osd":2,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.624}]},{"osd":4,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.61099999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.55700000000000005}]},{"osd":7,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.65100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":188099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.37}]},{"osd":1,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.498}]},{"osd":3,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.56299999999999994}]},{"osd":5,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.53600000000000003}]},{"osd":6,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.60299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":189093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.441},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.623}]},{"osd":6,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.58999999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.249},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.318}]},{"osd":1,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.30399999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.34499999999999997}]},{"osd":4,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.52800000000000002}]},{"osd":6,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.51200000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.629}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":187585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.35499999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.311}]},{"osd":3,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.29999999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.76400000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.58899999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.56699999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1032192,"data_stored":776878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.48399999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.443}]},{"osd":3,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.47199999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.90200000000000002}]},{"osd":5,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.876}]},{"osd":6,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.88900000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.92100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:58:46.794 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T17:58:46.795 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T17:58:46.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.928+0000 7f7327dd4640 1 Processor -- start 2024-09-13T17:58:46.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.928+0000 7f7327dd4640 1 -- start start 2024-09-13T17:58:46.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.929+0000 7f7327dd4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73200b2b40 0x7f73200ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.929+0000 7f7327dd4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f73200ce4f0 0x7f73200ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.929+0000 7f7327dd4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73200ccbe0 con 0x7f7320159310 2024-09-13T17:58:46.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.929+0000 7f7327dd4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73200ad940 con 0x7f73200ce4f0 2024-09-13T17:58:46.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.929+0000 7f7327dd4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73200ceea0 con 0x7f73200b2b40 2024-09-13T17:58:46.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.929+0000 7f732536f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73200b2b40 0x7f73200ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.929+0000 7f7326371640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f73200ce4f0 0x7f73200ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.930+0000 7f7325b70640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f7320159310 0x7f73200ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:46678/0 (socket says 172.21.15.111:46678) 2024-09-13T17:58:46.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.930+0000 7f732536f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73200b2b40 0x7f73200ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:50110/0 (socket says 172.21.15.111:50110) 2024-09-13T17:58:46.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.930+0000 7f7325b70640 1 -- 172.21.15.111:0/3096583242 learned_addr learned my addr 172.21.15.111:0/3096583242 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:46.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.930+0000 7f7324b6e640 1 -- 172.21.15.111:0/3096583242 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889071740 0 0) 0x7f73200ccbe0 con 0x7f7320159310 2024-09-13T17:58:46.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.930+0000 7f7324b6e640 1 -- 172.21.15.111:0/3096583242 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7308003520 con 0x7f7320159310 2024-09-13T17:58:46.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.930+0000 7f732536f640 1 -- 172.21.15.111:0/3096583242 >> v2:172.21.15.188:3300/0 conn(0x7f73200ce4f0 msgr2=0x7f73200ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:46.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.930+0000 7f732536f640 1 --2- 172.21.15.111:0/3096583242 >> v2:172.21.15.188:3300/0 conn(0x7f73200ce4f0 0x7f73200ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.930+0000 7f732536f640 1 -- 172.21.15.111:0/3096583242 >> v1:172.21.15.188:6789/0 conn(0x7f7320159310 legacy=0x7f73200ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.930+0000 7f732536f640 1 -- 172.21.15.111:0/3096583242 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73200b1810 con 0x7f73200b2b40 2024-09-13T17:58:46.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.930+0000 7f7326371640 1 --2- 172.21.15.111:0/3096583242 >> v2:172.21.15.188:3300/0 conn(0x7f73200ce4f0 0x7f73200ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:46.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.931+0000 7f732536f640 1 --2- 172.21.15.111:0/3096583242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73200b2b40 0x7f73200ad400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f73140097a0 tx=0x7f731402a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6187ffe8c96f2a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:46.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.931+0000 7f7324b6e640 1 -- 172.21.15.111:0/3096583242 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731402adb0 con 0x7f73200b2b40 2024-09-13T17:58:46.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.931+0000 7f7324b6e640 1 -- 172.21.15.111:0/3096583242 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7314032440 con 0x7f73200b2b40 2024-09-13T17:58:46.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.931+0000 7f7324b6e640 1 -- 172.21.15.111:0/3096583242 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73140327d0 con 0x7f73200b2b40 2024-09-13T17:58:46.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.931+0000 7f7327dd4640 1 -- 172.21.15.111:0/3096583242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73200b2b40 msgr2=0x7f73200ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.931+0000 7f7327dd4640 1 --2- 172.21.15.111:0/3096583242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73200b2b40 0x7f73200ad400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f73140097a0 tx=0x7f731402a060 comp rx=0 tx=0).stop 2024-09-13T17:58:46.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.931+0000 7f7327dd4640 1 -- 172.21.15.111:0/3096583242 shutdown_connections 2024-09-13T17:58:46.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.931+0000 7f7327dd4640 1 --2- 172.21.15.111:0/3096583242 >> v2:172.21.15.188:3300/0 conn(0x7f73200ce4f0 0x7f73200ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.931+0000 7f7327dd4640 1 --2- 172.21.15.111:0/3096583242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73200b2b40 0x7f73200ad400 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.931+0000 7f7327dd4640 1 -- 172.21.15.111:0/3096583242 >> 172.21.15.111:0/3096583242 conn(0x7f732006a0b0 msgr2=0x7f732006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:46.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.932+0000 7f7327dd4640 1 -- 172.21.15.111:0/3096583242 shutdown_connections 2024-09-13T17:58:46.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.932+0000 7f7327dd4640 1 -- 172.21.15.111:0/3096583242 wait complete. 2024-09-13T17:58:46.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.933+0000 7f7327dd4640 1 Processor -- start 2024-09-13T17:58:46.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.933+0000 7f7327dd4640 1 -- start start 2024-09-13T17:58:46.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.933+0000 7f7327dd4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73200b2b40 0x7f73201f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.933+0000 7f7327dd4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73200ce4f0 0x7f732009b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.933+0000 7f7327dd4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7320159310 0x7f73201efa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.933+0000 7f7327dd4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7320098940 con 0x7f73200b2b40 2024-09-13T17:58:46.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.933+0000 7f7327dd4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73200985f0 con 0x7f73200ce4f0 2024-09-13T17:58:46.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.933+0000 7f7327dd4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732009bab0 con 0x7f7320159310 2024-09-13T17:58:46.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.933+0000 7f7325b70640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73200b2b40 0x7f73201f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.934+0000 7f7325b70640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73200b2b40 0x7f73201f4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40280/0 (socket says 172.21.15.111:40280) 2024-09-13T17:58:46.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.934+0000 7f7325b70640 1 -- 172.21.15.111:0/1496100409 learned_addr learned my addr 172.21.15.111:0/1496100409 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:46.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.934+0000 7f732536f640 1 --2- 172.21.15.111:0/1496100409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73200ce4f0 0x7f732009b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.934+0000 7f7326371640 1 --2- 172.21.15.111:0/1496100409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7320159310 0x7f73201efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.934+0000 7f7325b70640 1 -- 172.21.15.111:0/1496100409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7320159310 msgr2=0x7f73201efa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.934+0000 7f7325b70640 1 --2- 172.21.15.111:0/1496100409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7320159310 0x7f73201efa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.934+0000 7f7325b70640 1 -- 172.21.15.111:0/1496100409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73200ce4f0 msgr2=0x7f732009b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:46.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.934+0000 7f7325b70640 1 --2- 172.21.15.111:0/1496100409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73200ce4f0 0x7f732009b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.934+0000 7f7325b70640 1 -- 172.21.15.111:0/1496100409 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7318009400 con 0x7f73200b2b40 2024-09-13T17:58:46.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.934+0000 7f7325b70640 1 --2- 172.21.15.111:0/1496100409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73200b2b40 0x7f73201f4360 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f731000cbc0 tx=0x7f73100021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:46.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.934+0000 7f731e7fc640 1 -- 172.21.15.111:0/1496100409 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731000ac00 con 0x7f73200b2b40 2024-09-13T17:58:46.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.934+0000 7f731e7fc640 1 -- 172.21.15.111:0/1496100409 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731000ad60 con 0x7f73200b2b40 2024-09-13T17:58:46.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.934+0000 7f7327dd4640 1 -- 172.21.15.111:0/1496100409 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7314009400 con 0x7f73200b2b40 2024-09-13T17:58:46.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.934+0000 7f731e7fc640 1 -- 172.21.15.111:0/1496100409 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7310012620 con 0x7f73200b2b40 2024-09-13T17:58:46.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.934+0000 7f7327dd4640 1 -- 172.21.15.111:0/1496100409 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732009c070 con 0x7f73200b2b40 2024-09-13T17:58:46.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.935+0000 7f7327dd4640 1 -- 172.21.15.111:0/1496100409 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732006dfa0 con 0x7f73200b2b40 2024-09-13T17:58:46.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.936+0000 7f731e7fc640 1 -- 172.21.15.111:0/1496100409 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7310007070 con 0x7f73200b2b40 2024-09-13T17:58:46.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.936+0000 7f731e7fc640 1 --2- 172.21.15.111:0/1496100409 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f730c06ac00 0x7f730c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.936+0000 7f731e7fc640 1 -- 172.21.15.111:0/1496100409 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7310089310 con 0x7f73200b2b40 2024-09-13T17:58:46.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.936+0000 7f732536f640 1 --2- 172.21.15.111:0/1496100409 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f730c06ac00 0x7f730c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.937+0000 7f732536f640 1 --2- 172.21.15.111:0/1496100409 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f730c06ac00 0x7f730c06d0c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f73140097a0 tx=0x7f731402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:46.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:46.940+0000 7f731e7fc640 1 -- 172.21.15.111:0/1496100409 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f73100544f0 con 0x7f73200b2b40 2024-09-13T17:58:47.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.163+0000 7f7327dd4640 1 -- 172.21.15.111:0/1496100409 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f732006e4b0 con 0x7f73200b2b40 2024-09-13T17:58:47.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.166+0000 7f731e7fc640 1 -- 172.21.15.111:0/1496100409 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7f7310057b10 con 0x7f73200b2b40 2024-09-13T17:58:47.169 INFO:teuthology.orchestra.run.smithi111.stderr:updated caps for client.0 2024-09-13T17:58:47.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.168+0000 7f7327dd4640 1 -- 172.21.15.111:0/1496100409 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f730c06ac00 msgr2=0x7f730c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:47.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.168+0000 7f7327dd4640 1 --2- 172.21.15.111:0/1496100409 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f730c06ac00 0x7f730c06d0c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f73140097a0 tx=0x7f731402e000 comp rx=0 tx=0).stop 2024-09-13T17:58:47.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.168+0000 7f7327dd4640 1 -- 172.21.15.111:0/1496100409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73200b2b40 msgr2=0x7f73201f4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:47.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.168+0000 7f7327dd4640 1 --2- 172.21.15.111:0/1496100409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73200b2b40 0x7f73201f4360 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f731000cbc0 tx=0x7f73100021c0 comp rx=0 tx=0).stop 2024-09-13T17:58:47.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.168+0000 7f7327dd4640 1 -- 172.21.15.111:0/1496100409 shutdown_connections 2024-09-13T17:58:47.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.168+0000 7f7327dd4640 1 --2- 172.21.15.111:0/1496100409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7320159310 0x7f73201efa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:47.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.168+0000 7f7327dd4640 1 --2- 172.21.15.111:0/1496100409 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f730c06ac00 0x7f730c06d0c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:47.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.168+0000 7f7327dd4640 1 --2- 172.21.15.111:0/1496100409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73200ce4f0 0x7f732009b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:47.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.168+0000 7f7327dd4640 1 --2- 172.21.15.111:0/1496100409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73200b2b40 0x7f73201f4360 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:47.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.168+0000 7f7327dd4640 1 -- 172.21.15.111:0/1496100409 >> 172.21.15.111:0/1496100409 conn(0x7f732006a0b0 msgr2=0x7f73200ae980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:47.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.168+0000 7f7327dd4640 1 -- 172.21.15.111:0/1496100409 shutdown_connections 2024-09-13T17:58:47.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.168+0000 7f7327dd4640 1 -- 172.21.15.111:0/1496100409 wait complete. 2024-09-13T17:58:47.182 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:58:47.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.277+0000 7f59d860d640 1 Processor -- start 2024-09-13T17:58:47.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.277+0000 7f59d860d640 1 -- start start 2024-09-13T17:58:47.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.278+0000 7f59d860d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f59d006c4c0 0x7f59d006c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:47.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.278+0000 7f59d860d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f59d00cbea0 0x7f59d00cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:47.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.278+0000 7f59d860d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d006dab0 con 0x7f59d00b2590 2024-09-13T17:58:47.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.278+0000 7f59d860d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d006deb0 con 0x7f59d00cbea0 2024-09-13T17:58:47.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.278+0000 7f59d860d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d00b2170 con 0x7f59d006c4c0 2024-09-13T17:58:47.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.278+0000 7f59d6baa640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f59d00cbea0 0x7f59d00cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:47.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.278+0000 7f59d6baa640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f59d00cbea0 0x7f59d00cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40296/0 (socket says 172.21.15.111:40296) 2024-09-13T17:58:47.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.278+0000 7f59d6baa640 1 -- 172.21.15.111:0/794001310 learned_addr learned my addr 172.21.15.111:0/794001310 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:47.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.278+0000 7f59d63a9640 1 --2- 172.21.15.111:0/794001310 >> v2:172.21.15.188:3300/0 conn(0x7f59d006c4c0 0x7f59d006c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:47.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.278+0000 7f59d6baa640 1 -- 172.21.15.111:0/794001310 >> v2:172.21.15.188:3300/0 conn(0x7f59d006c4c0 msgr2=0x7f59d006c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:47.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.278+0000 7f59d6baa640 1 --2- 172.21.15.111:0/794001310 >> v2:172.21.15.188:3300/0 conn(0x7f59d006c4c0 0x7f59d006c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:47.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.278+0000 7f59d6baa640 1 -- 172.21.15.111:0/794001310 >> v1:172.21.15.188:6789/0 conn(0x7f59d00b2590 legacy=0x7f59d006d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:47.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.278+0000 7f59d6baa640 1 -- 172.21.15.111:0/794001310 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d00cc870 con 0x7f59d00cbea0 2024-09-13T17:58:47.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.279+0000 7f59d6baa640 1 --2- 172.21.15.111:0/794001310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59d00cbea0 0x7f59d00cc330 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f59d00cb910 tx=0x7f59c802c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ba204f66591f24 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:47.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.279+0000 7f59d53a7640 1 -- 172.21.15.111:0/794001310 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c800b2e0 con 0x7f59d00cbea0 2024-09-13T17:58:47.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.279+0000 7f59d53a7640 1 -- 172.21.15.111:0/794001310 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c800b440 con 0x7f59d00cbea0 2024-09-13T17:58:47.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.279+0000 7f59d53a7640 1 -- 172.21.15.111:0/794001310 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c8009820 con 0x7f59d00cbea0 2024-09-13T17:58:47.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.279+0000 7f59d860d640 1 -- 172.21.15.111:0/794001310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59d00cbea0 msgr2=0x7f59d00cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:47.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.279+0000 7f59d860d640 1 --2- 172.21.15.111:0/794001310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59d00cbea0 0x7f59d00cc330 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f59d00cb910 tx=0x7f59c802c940 comp rx=0 tx=0).stop 2024-09-13T17:58:47.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.280+0000 7f59d860d640 1 -- 172.21.15.111:0/794001310 shutdown_connections 2024-09-13T17:58:47.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.280+0000 7f59d860d640 1 --2- 172.21.15.111:0/794001310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59d00cbea0 0x7f59d00cc330 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:47.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.280+0000 7f59d860d640 1 --2- 172.21.15.111:0/794001310 >> v2:172.21.15.188:3300/0 conn(0x7f59d006c4c0 0x7f59d006c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:47.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.280+0000 7f59d860d640 1 -- 172.21.15.111:0/794001310 >> 172.21.15.111:0/794001310 conn(0x7f59d0069ed0 msgr2=0x7f59d006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:47.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.280+0000 7f59d860d640 1 -- 172.21.15.111:0/794001310 shutdown_connections 2024-09-13T17:58:47.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.280+0000 7f59d860d640 1 -- 172.21.15.111:0/794001310 wait complete. 2024-09-13T17:58:47.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.281+0000 7f59d860d640 1 Processor -- start 2024-09-13T17:58:47.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.281+0000 7f59d860d640 1 -- start start 2024-09-13T17:58:47.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.281+0000 7f59d860d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59d006c4c0 0x7f59d0154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:47.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.281+0000 7f59d860d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59d00b2590 0x7f59d0156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:47.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.281+0000 7f59d860d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59d00cbea0 0x7f59d0156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:47.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.281+0000 7f59d860d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d00c6370 con 0x7f59d00b2590 2024-09-13T17:58:47.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.281+0000 7f59d860d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d00c56f0 con 0x7f59d00cbea0 2024-09-13T17:58:47.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.281+0000 7f59d860d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d00c59d0 con 0x7f59d006c4c0 2024-09-13T17:58:47.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.282+0000 7f59d5ba8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59d00b2590 0x7f59d0156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:47.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.282+0000 7f59d5ba8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59d00b2590 0x7f59d0156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40302/0 (socket says 172.21.15.111:40302) 2024-09-13T17:58:47.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.282+0000 7f59d5ba8640 1 -- 172.21.15.111:0/3652811081 learned_addr learned my addr 172.21.15.111:0/3652811081 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:47.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.282+0000 7f59d63a9640 1 --2- 172.21.15.111:0/3652811081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59d006c4c0 0x7f59d0154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:47.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.282+0000 7f59d6baa640 1 --2- 172.21.15.111:0/3652811081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59d00cbea0 0x7f59d0156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:47.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.282+0000 7f59d5ba8640 1 -- 172.21.15.111:0/3652811081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59d006c4c0 msgr2=0x7f59d0154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:47.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.282+0000 7f59d5ba8640 1 --2- 172.21.15.111:0/3652811081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59d006c4c0 0x7f59d0154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:47.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.282+0000 7f59d5ba8640 1 -- 172.21.15.111:0/3652811081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59d00cbea0 msgr2=0x7f59d0156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:47.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.282+0000 7f59d5ba8640 1 --2- 172.21.15.111:0/3652811081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59d00cbea0 0x7f59d0156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:47.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.282+0000 7f59d5ba8640 1 -- 172.21.15.111:0/3652811081 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c800bd40 con 0x7f59d00b2590 2024-09-13T17:58:47.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.282+0000 7f59d5ba8640 1 --2- 172.21.15.111:0/3652811081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59d00b2590 0x7f59d0156300 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f59c000c450 tx=0x7f59c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:47.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.282+0000 7f59c6ffd640 1 -- 172.21.15.111:0/3652811081 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c00052d0 con 0x7f59d00b2590 2024-09-13T17:58:47.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.282+0000 7f59c6ffd640 1 -- 172.21.15.111:0/3652811081 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c0005430 con 0x7f59d00b2590 2024-09-13T17:58:47.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.282+0000 7f59d860d640 1 -- 172.21.15.111:0/3652811081 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d00c5cb0 con 0x7f59d00b2590 2024-09-13T17:58:47.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.282+0000 7f59c6ffd640 1 -- 172.21.15.111:0/3652811081 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c0012870 con 0x7f59d00b2590 2024-09-13T17:58:47.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.282+0000 7f59d860d640 1 -- 172.21.15.111:0/3652811081 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d00c53a0 con 0x7f59d00b2590 2024-09-13T17:58:47.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.283+0000 7f59d860d640 1 -- 172.21.15.111:0/3652811081 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d00c64b0 con 0x7f59d00b2590 2024-09-13T17:58:47.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.284+0000 7f59c6ffd640 1 -- 172.21.15.111:0/3652811081 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f59c00129d0 con 0x7f59d00b2590 2024-09-13T17:58:47.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.284+0000 7f59c6ffd640 1 --2- 172.21.15.111:0/3652811081 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f59b406ac00 0x7f59b406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:47.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.284+0000 7f59c6ffd640 1 -- 172.21.15.111:0/3652811081 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f59c0088e20 con 0x7f59d00b2590 2024-09-13T17:58:47.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.284+0000 7f59d63a9640 1 --2- 172.21.15.111:0/3652811081 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f59b406ac00 0x7f59b406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:47.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.284+0000 7f59d63a9640 1 --2- 172.21.15.111:0/3652811081 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f59b406ac00 0x7f59b406d0c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009aa0 tx=0x7f59cc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:47.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.288+0000 7f59c6ffd640 1 -- 172.21.15.111:0/3652811081 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f59d00c64b0 con 0x7f59d00b2590 2024-09-13T17:58:47.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.483+0000 7f59d860d640 1 -- 172.21.15.111:0/3652811081 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59d00c64b0 con 0x7f59d00b2590 2024-09-13T17:58:47.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.484+0000 7f59c6ffd640 1 -- 172.21.15.111:0/3652811081 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f59d00c64b0 con 0x7f59d00b2590 2024-09-13T17:58:47.487 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:47.487 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4217138481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4217138481},{"type":"v1","addr":"172.21.15.188:6834","nonce":4217138481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3099948287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3099948287},{"type":"v1","addr":"172.21.15.188:6835","nonce":3099948287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/214034224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":214034224},{"type":"v1","addr":"172.21.15.111:6835","nonce":214034224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:58:43.534567+0000","modified":"2024-09-13T17:58:45.542773+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5490":{"gid":5490,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6837/982359636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":982359636},{"type":"v1","addr":"172.21.15.111:6837","nonce":982359636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:58:47.488 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 15 2024-09-13T17:58:47.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.486+0000 7f59d860d640 1 -- 172.21.15.111:0/3652811081 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f59b406ac00 msgr2=0x7f59b406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:47.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.486+0000 7f59d860d640 1 --2- 172.21.15.111:0/3652811081 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f59b406ac00 0x7f59b406d0c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009aa0 tx=0x7f59cc009190 comp rx=0 tx=0).stop 2024-09-13T17:58:47.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.486+0000 7f59d860d640 1 -- 172.21.15.111:0/3652811081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59d00b2590 msgr2=0x7f59d0156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:47.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.486+0000 7f59d860d640 1 --2- 172.21.15.111:0/3652811081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59d00b2590 0x7f59d0156300 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f59c000c450 tx=0x7f59c000c910 comp rx=0 tx=0).stop 2024-09-13T17:58:47.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.486+0000 7f59d860d640 1 -- 172.21.15.111:0/3652811081 shutdown_connections 2024-09-13T17:58:47.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.486+0000 7f59d860d640 1 --2- 172.21.15.111:0/3652811081 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f59b406ac00 0x7f59b406d0c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:47.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.486+0000 7f59d860d640 1 --2- 172.21.15.111:0/3652811081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59d00cbea0 0x7f59d0156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:47.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.486+0000 7f59d860d640 1 --2- 172.21.15.111:0/3652811081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59d00b2590 0x7f59d0156300 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:47.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.486+0000 7f59d860d640 1 --2- 172.21.15.111:0/3652811081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59d006c4c0 0x7f59d0154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:47.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.486+0000 7f59d860d640 1 -- 172.21.15.111:0/3652811081 >> 172.21.15.111:0/3652811081 conn(0x7f59d0069ed0 msgr2=0x7f59d00af4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:47.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.486+0000 7f59d860d640 1 -- 172.21.15.111:0/3652811081 shutdown_connections 2024-09-13T17:58:47.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.486+0000 7f59d860d640 1 -- 172.21.15.111:0/3652811081 wait complete. 2024-09-13T17:58:47.874 INFO:tasks.ceph.mds.a.smithi111.stderr:2024-09-13T17:58:47.871+0000 7f33429de640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:58:47.902 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:58:43.534567+0000', 'modified': '2024-09-13T17:58:45.542773+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5490}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5490': {'gid': 5490, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6837/982359636', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 982359636}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 982359636}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T17:58:47.902 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T17:58:47.902 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-13T17:58:48.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.999+0000 7f0d01412640 1 Processor -- start 2024-09-13T17:58:48.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:47.999+0000 7f0d01412640 1 -- start start 2024-09-13T17:58:48.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.000+0000 7f0d01412640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0cfc072110 0x7f0cfc070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.000+0000 7f0d01412640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cfc0abdf0 con 0x7f0cfc071d30 2024-09-13T17:58:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.000+0000 7f0d01412640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cfc0b2280 con 0x7f0cfc0c8920 2024-09-13T17:58:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.000+0000 7f0d01412640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cfc0ab1e0 con 0x7f0cfc072110 2024-09-13T17:58:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.000+0000 7f0cfaffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f0cfc071d30 0x7f0cfc0b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58168/0 (socket says 172.21.15.111:58168) 2024-09-13T17:58:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.000+0000 7f0cfaffd640 1 -- 172.21.15.111:0/179205882 learned_addr learned my addr 172.21.15.111:0/179205882 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:58:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.000+0000 7f0cfa7fc640 1 --2- 172.21.15.111:0/179205882 >> v2:172.21.15.188:3300/0 conn(0x7f0cfc072110 0x7f0cfc070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.000+0000 7f0cf9ffb640 1 -- 172.21.15.111:0/179205882 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4182958344 0 0) 0x7f0cfc0abdf0 con 0x7f0cfc071d30 2024-09-13T17:58:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.000+0000 7f0cf9ffb640 1 -- 172.21.15.111:0/179205882 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ce8003520 con 0x7f0cfc071d30 2024-09-13T17:58:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cf9ffb640 1 -- 172.21.15.111:0/179205882 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 730839301 0 0) 0x7f0ce8003520 con 0x7f0cfc071d30 2024-09-13T17:58:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cf9ffb640 1 -- 172.21.15.111:0/179205882 >> v2:172.21.15.188:3300/0 conn(0x7f0cfc072110 msgr2=0x7f0cfc070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:48.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cf9ffb640 1 --2- 172.21.15.111:0/179205882 >> v2:172.21.15.188:3300/0 conn(0x7f0cfc072110 0x7f0cfc070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cf9ffb640 1 -- 172.21.15.111:0/179205882 >> v1:172.21.15.188:6789/0 conn(0x7f0cfc0c8920 legacy=0x7f0cfc0ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:48.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cf9ffb640 1 -- 172.21.15.111:0/179205882 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc0ab4c0 con 0x7f0cfc071d30 2024-09-13T17:58:48.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cf9ffb640 1 -- 172.21.15.111:0/179205882 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f0cec0019f0 con 0x7f0cfc071d30 2024-09-13T17:58:48.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cf9ffb640 1 -- 172.21.15.111:0/179205882 >> v1:172.21.15.111:6789/0 conn(0x7f0cfc071d30 legacy=0x7f0cfc0b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:48.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cf9ffb640 1 --2- 172.21.15.111:0/179205882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ce8003970 0x7f0ce8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cf9ffb640 1 --2- 172.21.15.111:0/179205882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ce80047c0 0x7f0ce8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cf9ffb640 1 --2- 172.21.15.111:0/179205882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ce8005180 0x7f0ce800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cf9ffb640 1 -- 172.21.15.111:0/179205882 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc0ab1e0 con 0x7f0ce8005180 2024-09-13T17:58:48.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cf9ffb640 1 -- 172.21.15.111:0/179205882 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce800c020 con 0x7f0ce80047c0 2024-09-13T17:58:48.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cf9ffb640 1 -- 172.21.15.111:0/179205882 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce800c1e0 con 0x7f0ce8003970 2024-09-13T17:58:48.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cfb7fe640 1 --2- 172.21.15.111:0/179205882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ce8005180 0x7f0ce800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cfa7fc640 1 --2- 172.21.15.111:0/179205882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ce80047c0 0x7f0ce8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cfaffd640 1 --2- 172.21.15.111:0/179205882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ce8003970 0x7f0ce8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cfb7fe640 1 -- 172.21.15.111:0/179205882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ce8003970 msgr2=0x7f0ce8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:48.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cfb7fe640 1 --2- 172.21.15.111:0/179205882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ce8003970 0x7f0ce8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cfb7fe640 1 -- 172.21.15.111:0/179205882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ce80047c0 msgr2=0x7f0ce8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:48.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cfb7fe640 1 --2- 172.21.15.111:0/179205882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ce80047c0 0x7f0ce8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cfb7fe640 1 -- 172.21.15.111:0/179205882 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce8003520 con 0x7f0ce8005180 2024-09-13T17:58:48.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.001+0000 7f0cfaffd640 1 --2- 172.21.15.111:0/179205882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ce8003970 0x7f0ce8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:48.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.002+0000 7f0cfb7fe640 1 -- 172.21.15.111:0/179205882 reap_dead start 2024-09-13T17:58:48.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.002+0000 7f0cfb7fe640 1 --2- 172.21.15.111:0/179205882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ce8005180 0x7f0ce800b980 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4001750 tx=0x7f0ce80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5038ac2230a28f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:48.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.002+0000 7f0cf9ffb640 1 -- 172.21.15.111:0/179205882 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf40052d0 con 0x7f0ce8005180 2024-09-13T17:58:48.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.002+0000 7f0cf9ffb640 1 -- 172.21.15.111:0/179205882 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf4005450 con 0x7f0ce8005180 2024-09-13T17:58:48.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.002+0000 7f0cf9ffb640 1 -- 172.21.15.111:0/179205882 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf402abe0 con 0x7f0ce8005180 2024-09-13T17:58:48.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.002+0000 7f0d01412640 1 -- 172.21.15.111:0/179205882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ce8005180 msgr2=0x7f0ce800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:48.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.002+0000 7f0d01412640 1 --2- 172.21.15.111:0/179205882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ce8005180 0x7f0ce800b980 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4001750 tx=0x7f0ce80057f0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.002+0000 7f0d01412640 1 -- 172.21.15.111:0/179205882 shutdown_connections 2024-09-13T17:58:48.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.002+0000 7f0d01412640 1 --2- 172.21.15.111:0/179205882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ce8005180 0x7f0ce800b980 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.002+0000 7f0d01412640 1 -- 172.21.15.111:0/179205882 >> 172.21.15.111:0/179205882 conn(0x7f0cfc069ed0 msgr2=0x7f0cfc06a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:48.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.003+0000 7f0d01412640 1 -- 172.21.15.111:0/179205882 shutdown_connections 2024-09-13T17:58:48.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.003+0000 7f0d01412640 1 -- 172.21.15.111:0/179205882 wait complete. 2024-09-13T17:58:48.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.004+0000 7f0d01412640 1 Processor -- start 2024-09-13T17:58:48.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.004+0000 7f0d01412640 1 -- start start 2024-09-13T17:58:48.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0d01412640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ce8005180 0x7f0cfc158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0d01412640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc0c2610 0x7f0cfc0c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0d01412640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cfc15cd40 0x7f0cfc15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0d01412640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc15d700 con 0x7f0cfc15cd40 2024-09-13T17:58:48.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0d01412640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc1f6620 con 0x7f0cfc0c2610 2024-09-13T17:58:48.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0d01412640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc1f5860 con 0x7f0ce8005180 2024-09-13T17:58:48.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0cfb7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cfc15cd40 0x7f0cfc15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0cfb7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cfc15cd40 0x7f0cfc15d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40324/0 (socket says 172.21.15.111:40324) 2024-09-13T17:58:48.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0cfb7fe640 1 -- 172.21.15.111:0/759089972 learned_addr learned my addr 172.21.15.111:0/759089972 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:48.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0cfa7fc640 1 --2- 172.21.15.111:0/759089972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc0c2610 0x7f0cfc0c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0cfaffd640 1 --2- 172.21.15.111:0/759089972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ce8005180 0x7f0cfc158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0cfb7fe640 1 -- 172.21.15.111:0/759089972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ce8005180 msgr2=0x7f0cfc158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:48.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0cfb7fe640 1 --2- 172.21.15.111:0/759089972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ce8005180 0x7f0cfc158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0cfb7fe640 1 -- 172.21.15.111:0/759089972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc0c2610 msgr2=0x7f0cfc0c2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:48.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0cfb7fe640 1 --2- 172.21.15.111:0/759089972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc0c2610 0x7f0cfc0c2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0cfb7fe640 1 -- 172.21.15.111:0/759089972 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc1f5b40 con 0x7f0cfc15cd40 2024-09-13T17:58:48.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0cfb7fe640 1 --2- 172.21.15.111:0/759089972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cfc15cd40 0x7f0cfc15d1c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4001840 tx=0x7f0cf4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:48.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0cf37fe640 1 -- 172.21.15.111:0/759089972 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4038070 con 0x7f0cfc15cd40 2024-09-13T17:58:48.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0cf37fe640 1 -- 172.21.15.111:0/759089972 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf4007a30 con 0x7f0cfc15cd40 2024-09-13T17:58:48.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0cf37fe640 1 -- 172.21.15.111:0/759089972 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4004db0 con 0x7f0cfc15cd40 2024-09-13T17:58:48.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0d01412640 1 -- 172.21.15.111:0/759089972 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cfc1f5e20 con 0x7f0cfc15cd40 2024-09-13T17:58:48.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.005+0000 7f0d01412640 1 -- 172.21.15.111:0/759089972 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cfc0c3140 con 0x7f0cfc15cd40 2024-09-13T17:58:48.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.007+0000 7f0cf37fe640 1 -- 172.21.15.111:0/759089972 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0cf4007030 con 0x7f0cfc15cd40 2024-09-13T17:58:48.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.007+0000 7f0cf37fe640 1 --2- 172.21.15.111:0/759089972 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0cdc06ac30 0x7f0cdc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.007+0000 7f0cfaffd640 1 --2- 172.21.15.111:0/759089972 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0cdc06ac30 0x7f0cdc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.007+0000 7f0cf37fe640 1 -- 172.21.15.111:0/759089972 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0cf40a8430 con 0x7f0cfc15cd40 2024-09-13T17:58:48.014 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.008+0000 7f0cfaffd640 1 --2- 172.21.15.111:0/759089972 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0cdc06ac30 0x7f0cdc06d0f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0012e0 tx=0x7f0cec001d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:48.014 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.008+0000 7f0d04212740 1 Processor -- start 2024-09-13T17:58:48.014 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.008+0000 7f0d04212740 1 -- start start 2024-09-13T17:58:48.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.009+0000 7f0d04212740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6cd0a0 0x55716d6cd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.009+0000 7f0d04212740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6ca860 0x55716d71a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.009+0000 7f0d04212740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6f4550 0x55716d71aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.009+0000 7f0d04212740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716d6277c0 con 0x55716d6ca860 2024-09-13T17:58:48.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.009+0000 7f0d04212740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716d636b70 con 0x55716d6f4550 2024-09-13T17:58:48.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.009+0000 7f0d04212740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55716d634300 con 0x55716d6cd0a0 2024-09-13T17:58:48.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.009+0000 7f0cfa7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6ca860 0x55716d71a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.009+0000 7f0cfa7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6ca860 0x55716d71a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40334/0 (socket says 172.21.15.111:40334) 2024-09-13T17:58:48.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.009+0000 7f0cfa7fc640 1 -- 172.21.15.111:0/3365927952 learned_addr learned my addr 172.21.15.111:0/3365927952 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:48.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.009+0000 7f0cfaffd640 1 --2- 172.21.15.111:0/3365927952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6cd0a0 0x55716d6cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.009+0000 7f0cfb7fe640 1 --2- 172.21.15.111:0/3365927952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6f4550 0x55716d71aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.009+0000 7f0cfa7fc640 1 -- 172.21.15.111:0/3365927952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6cd0a0 msgr2=0x55716d6cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:48.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.010+0000 7f0cfa7fc640 1 --2- 172.21.15.111:0/3365927952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6cd0a0 0x55716d6cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.010+0000 7f0cfa7fc640 1 -- 172.21.15.111:0/3365927952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6f4550 msgr2=0x55716d71aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:48.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.010+0000 7f0cfa7fc640 1 --2- 172.21.15.111:0/3365927952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6f4550 0x55716d71aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.010+0000 7f0cfa7fc640 1 -- 172.21.15.111:0/3365927952 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55716d5bc4b0 con 0x55716d6ca860 2024-09-13T17:58:48.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.010+0000 7f0cfb7fe640 1 --2- 172.21.15.111:0/3365927952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6f4550 0x55716d71aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:48.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.010+0000 7f0cfaffd640 1 --2- 172.21.15.111:0/3365927952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6cd0a0 0x55716d6cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:48.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.010+0000 7f0cfa7fc640 1 --2- 172.21.15.111:0/3365927952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6ca860 0x55716d71a570 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009c10 tx=0x7f0ce4014780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a7d698675683f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:48.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.010+0000 7f0cf0ff9640 1 -- 172.21.15.111:0/3365927952 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce401d440 con 0x55716d6ca860 2024-09-13T17:58:48.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.010+0000 7f0cf0ff9640 1 -- 172.21.15.111:0/3365927952 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce401d5a0 con 0x55716d6ca860 2024-09-13T17:58:48.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.010+0000 7f0cf0ff9640 1 -- 172.21.15.111:0/3365927952 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce401d910 con 0x55716d6ca860 2024-09-13T17:58:48.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.010+0000 7f0d04212740 1 -- 172.21.15.111:0/3365927952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6ca860 msgr2=0x55716d71a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:48.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.010+0000 7f0d04212740 1 --2- 172.21.15.111:0/3365927952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6ca860 0x55716d71a570 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009c10 tx=0x7f0ce4014780 comp rx=0 tx=0).stop 2024-09-13T17:58:48.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.011+0000 7f0d04212740 1 -- 172.21.15.111:0/3365927952 shutdown_connections 2024-09-13T17:58:48.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.011+0000 7f0d04212740 1 --2- 172.21.15.111:0/3365927952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6f4550 0x55716d71aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.011+0000 7f0d04212740 1 --2- 172.21.15.111:0/3365927952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6ca860 0x55716d71a570 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.011+0000 7f0d04212740 1 --2- 172.21.15.111:0/3365927952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6cd0a0 0x55716d6cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.011+0000 7f0d04212740 1 -- 172.21.15.111:0/3365927952 >> 172.21.15.111:0/3365927952 conn(0x55716d6c97d0 msgr2=0x55716d711c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:48.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.011+0000 7f0d04212740 1 -- 172.21.15.111:0/3365927952 shutdown_connections 2024-09-13T17:58:48.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.011+0000 7f0d04212740 1 -- 172.21.15.111:0/3365927952 wait complete. 2024-09-13T17:58:48.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.011+0000 7f0d04212740 1 -- start start 2024-09-13T17:58:48.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.011+0000 7f0d04212740 1 Processor -- start 2024-09-13T17:58:48.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0d04212740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6cd0a0 0x55716d6c7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0d04212740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6f4550 0x55716d6f2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0d04212740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6c7bb0 0x55716d6f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0d04212740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716d62fbf0 con 0x55716d6c7bb0 2024-09-13T17:58:48.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0d04212740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716d5ee250 con 0x55716d6f4550 2024-09-13T17:58:48.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0d04212740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55716d5edef0 con 0x55716d6cd0a0 2024-09-13T17:58:48.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0d04212740 10 client.0 _ll_register_callbacks cb 0x55716d620740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:58:48.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0cfb7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6c7bb0 0x55716d6f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0cfb7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6c7bb0 0x55716d6f2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40340/0 (socket says 172.21.15.111:40340) 2024-09-13T17:58:48.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0cfb7fe640 1 -- 172.21.15.111:0/2887441641 learned_addr learned my addr 172.21.15.111:0/2887441641 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:48.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0cfa7fc640 1 --2- 172.21.15.111:0/2887441641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6f4550 0x55716d6f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0cfaffd640 1 --2- 172.21.15.111:0/2887441641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6cd0a0 0x55716d6c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0cfb7fe640 1 -- 172.21.15.111:0/2887441641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6cd0a0 msgr2=0x55716d6c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:48.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0cfb7fe640 1 --2- 172.21.15.111:0/2887441641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6cd0a0 0x55716d6c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0cfb7fe640 1 -- 172.21.15.111:0/2887441641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6f4550 msgr2=0x55716d6f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:48.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0cfb7fe640 1 --2- 172.21.15.111:0/2887441641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6f4550 0x55716d6f2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0cfb7fe640 1 -- 172.21.15.111:0/2887441641 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce4009840 con 0x55716d6c7bb0 2024-09-13T17:58:48.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.012+0000 7f0cfa7fc640 1 --2- 172.21.15.111:0/2887441641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6f4550 0x55716d6f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:48.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.013+0000 7f0cfb7fe640 1 --2- 172.21.15.111:0/2887441641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6c7bb0 0x55716d6f2ba0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40a6750 tx=0x7f0cf4004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acef4511ed84ebc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:48.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.013+0000 7f0cd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-13T17:58:48.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.013+0000 7f0cd9ffb640 1 -- 172.21.15.111:0/2887441641 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf407afb0 con 0x55716d6c7bb0 2024-09-13T17:58:48.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.013+0000 7f0cd9ffb640 1 -- 172.21.15.111:0/2887441641 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0cf4079040 con 0x55716d6c7bb0 2024-09-13T17:58:48.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.013+0000 7f0d04212740 1 -- 172.21.15.111:0/2887441641 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55716d3233c0 con 0x55716d6c7bb0 2024-09-13T17:58:48.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.013+0000 7f0cd9ffb640 1 -- 172.21.15.111:0/2887441641 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cf407f570 con 0x55716d6c7bb0 2024-09-13T17:58:48.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.013+0000 7f0d04212740 10 client.5664 fetch_fsmap learned FSMap version 15 2024-09-13T17:58:48.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.013+0000 7f0d04212740 1 -- 172.21.15.111:0/2887441641 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55716d62e270 con 0x55716d6c7bb0 2024-09-13T17:58:48.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.013+0000 7f0cd9ffb640 1 -- 172.21.15.111:0/2887441641 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55716d3233c0 con 0x55716d6c7bb0 2024-09-13T17:58:48.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.013+0000 7f0cd9ffb640 1 -- 172.21.15.111:0/2887441641 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 15) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0cf40a80c0 con 0x55716d6c7bb0 2024-09-13T17:58:48.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.013+0000 7f0d04212740 10 client.5664 fetch_fsmap finished waiting for FSMap version 15 2024-09-13T17:58:48.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.013+0000 7f0d04212740 10 client.5664 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T17:58:48.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.013+0000 7f0d04212740 20 client.5664 populate_metadata read hostname 'smithi111' 2024-09-13T17:58:48.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.013+0000 7f0d04212740 1 --2- 172.21.15.111:0/2887441641 >> [v2:172.21.15.111:6836/982359636,v1:172.21.15.111:6837/982359636] conn(0x55716d724d40 0x55716d725120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.013+0000 7f0d04212740 4 client.5664 mds_command: new command op to 5490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:58:48.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.013+0000 7f0d04212740 1 -- 172.21.15.111:0/2887441641 --> [v2:172.21.15.111:6836/982359636,v1:172.21.15.111:6837/982359636] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55716d3346c0 con 0x55716d724d40 2024-09-13T17:58:48.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.013+0000 7f0cfaffd640 1 --2- 172.21.15.111:0/2887441641 >> [v2:172.21.15.111:6836/982359636,v1:172.21.15.111:6837/982359636] conn(0x55716d724d40 0x55716d725120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.014+0000 7f0cd9ffb640 10 client.5664 ms_handle_connect on v2:172.21.15.111:6836/982359636 2024-09-13T17:58:48.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.014+0000 7f0cfaffd640 1 --2- 172.21.15.111:0/2887441641 >> [v2:172.21.15.111:6836/982359636,v1:172.21.15.111:6837/982359636] conn(0x55716d724d40 0x55716d725120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bec6be96d910279 server_cookie=ab89cbf919381e1f in_seq=0 out_seq=0 2024-09-13T17:58:48.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.014+0000 7f0cd9ffb640 1 -- 172.21.15.111:0/2887441641 <== mds.0 v2:172.21.15.111:6836/982359636 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55716d3346c0 con 0x55716d724d40 2024-09-13T17:58:48.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.014+0000 7f0cd9ffb640 10 client.5664 handle_command_reply: tid=0 2024-09-13T17:58:48.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.014+0000 7f0d04212740 1 client.5664 shutdown 2024-09-13T17:58:48.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.014+0000 7f0d04212740 1 -- 172.21.15.111:0/2887441641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6c7bb0 msgr2=0x55716d6f2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:48.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.014+0000 7f0d04212740 1 --2- 172.21.15.111:0/2887441641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6c7bb0 0x55716d6f2ba0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40a6750 tx=0x7f0cf4004f10 comp rx=0 tx=0).stop 2024-09-13T17:58:48.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.014+0000 7f0d04212740 1 -- 172.21.15.111:0/2887441641 shutdown_connections 2024-09-13T17:58:48.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.014+0000 7f0d04212740 1 --2- 172.21.15.111:0/2887441641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6c7bb0 0x55716d6f2ba0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.014+0000 7f0d04212740 1 --2- 172.21.15.111:0/2887441641 >> [v2:172.21.15.111:6836/982359636,v1:172.21.15.111:6837/982359636] conn(0x55716d724d40 0x55716d725120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.014+0000 7f0d04212740 1 --2- 172.21.15.111:0/2887441641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6f4550 0x55716d6f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.014+0000 7f0d04212740 1 --2- 172.21.15.111:0/2887441641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6cd0a0 0x55716d6c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.014+0000 7f0cd9ffb640 0 client.5664 ms_handle_reset on v2:172.21.15.111:6836/982359636 2024-09-13T17:58:48.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.014+0000 7f0d04212740 1 -- 172.21.15.111:0/2887441641 >> 172.21.15.111:0/2887441641 conn(0x55716d6ca860 msgr2=0x55716d715140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:48.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.014+0000 7f0d04212740 1 -- 172.21.15.111:0/2887441641 shutdown_connections 2024-09-13T17:58:48.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.014+0000 7f0d04212740 1 -- 172.21.15.111:0/2887441641 wait complete. 2024-09-13T17:58:48.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.015+0000 7f0d04212740 20 client.5664 trim_cache size 0 max 16384 2024-09-13T17:58:48.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.029+0000 7f0d04212740 1 Processor -- start 2024-09-13T17:58:48.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.029+0000 7f0d04212740 1 -- start start 2024-09-13T17:58:48.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0d04212740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6c7bb0 0x55716d711a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0d04212740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6cd0a0 0x55716d6cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0d04212740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6f4550 0x55716d6cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0d04212740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716d5b3460 con 0x55716d6cd0a0 2024-09-13T17:58:48.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0d04212740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716d6f7ec0 con 0x55716d6c7bb0 2024-09-13T17:58:48.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0d04212740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55716d664e50 con 0x55716d6f4550 2024-09-13T17:58:48.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0cfa7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6cd0a0 0x55716d6cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0cfa7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6cd0a0 0x55716d6cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40342/0 (socket says 172.21.15.111:40342) 2024-09-13T17:58:48.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0cfa7fc640 1 -- 172.21.15.111:0/430236537 learned_addr learned my addr 172.21.15.111:0/430236537 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:48.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0cfaffd640 1 --2- 172.21.15.111:0/430236537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6c7bb0 0x55716d711a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0cfb7fe640 1 --2- 172.21.15.111:0/430236537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6f4550 0x55716d6cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0cfa7fc640 1 -- 172.21.15.111:0/430236537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6f4550 msgr2=0x55716d6cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:48.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0cfa7fc640 1 --2- 172.21.15.111:0/430236537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6f4550 0x55716d6cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0cfa7fc640 1 -- 172.21.15.111:0/430236537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6c7bb0 msgr2=0x55716d711a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:48.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0cfa7fc640 1 --2- 172.21.15.111:0/430236537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6c7bb0 0x55716d711a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0cfa7fc640 1 -- 172.21.15.111:0/430236537 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55716d5d05f0 con 0x55716d6cd0a0 2024-09-13T17:58:48.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0cfaffd640 1 --2- 172.21.15.111:0/430236537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6c7bb0 0x55716d711a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:48.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0cfb7fe640 1 --2- 172.21.15.111:0/430236537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6f4550 0x55716d6cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:58:48.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0cfa7fc640 1 --2- 172.21.15.111:0/430236537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6cd0a0 0x55716d6cbfb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x55716d714b80 tx=0x7f0ce4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb25eead021d58c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:48.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.030+0000 7f0cf17fa640 1 -- 172.21.15.111:0/430236537 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce402a070 con 0x55716d6cd0a0 2024-09-13T17:58:48.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.031+0000 7f0cf17fa640 1 -- 172.21.15.111:0/430236537 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce4004040 con 0x55716d6cd0a0 2024-09-13T17:58:48.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.031+0000 7f0cf17fa640 1 -- 172.21.15.111:0/430236537 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce401d440 con 0x55716d6cd0a0 2024-09-13T17:58:48.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.031+0000 7f0d04212740 1 -- 172.21.15.111:0/430236537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6cd0a0 msgr2=0x55716d6cbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:48.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.031+0000 7f0d04212740 1 --2- 172.21.15.111:0/430236537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6cd0a0 0x55716d6cbfb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x55716d714b80 tx=0x7f0ce4017000 comp rx=0 tx=0).stop 2024-09-13T17:58:48.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.031+0000 7f0d04212740 1 -- 172.21.15.111:0/430236537 shutdown_connections 2024-09-13T17:58:48.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.031+0000 7f0d04212740 1 --2- 172.21.15.111:0/430236537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6f4550 0x55716d6cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.031+0000 7f0d04212740 1 --2- 172.21.15.111:0/430236537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6cd0a0 0x55716d6cbfb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.031+0000 7f0d04212740 1 --2- 172.21.15.111:0/430236537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6c7bb0 0x55716d711a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.031+0000 7f0d04212740 1 -- 172.21.15.111:0/430236537 >> 172.21.15.111:0/430236537 conn(0x55716d6ca860 msgr2=0x55716d7243f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:48.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.031+0000 7f0d04212740 1 -- 172.21.15.111:0/430236537 shutdown_connections 2024-09-13T17:58:48.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.031+0000 7f0d04212740 1 -- 172.21.15.111:0/430236537 wait complete. 2024-09-13T17:58:48.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.032+0000 7f0d04212740 1 -- start start 2024-09-13T17:58:48.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.032+0000 7f0d04212740 1 Processor -- start 2024-09-13T17:58:48.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.032+0000 7f0d04212740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6c7bb0 0x55716d77b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.032+0000 7f0d04212740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6cd0a0 0x55716d732d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.032+0000 7f0d04212740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6f4550 0x55716d733270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.032+0000 7f0d04212740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716d72bb50 con 0x55716d6f4550 2024-09-13T17:58:48.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.032+0000 7f0d04212740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716d72b870 con 0x55716d6c7bb0 2024-09-13T17:58:48.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.032+0000 7f0d04212740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55716d77ca30 con 0x55716d6cd0a0 2024-09-13T17:58:48.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.032+0000 7f0d04212740 10 client.0 _ll_register_callbacks cb 0x7f0ce40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:58:48.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0cfb7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6f4550 0x55716d733270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0cfb7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6f4550 0x55716d733270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40358/0 (socket says 172.21.15.111:40358) 2024-09-13T17:58:48.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0cfb7fe640 1 -- 172.21.15.111:0/3492513650 learned_addr learned my addr 172.21.15.111:0/3492513650 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:48.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0cfa7fc640 1 --2- 172.21.15.111:0/3492513650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6cd0a0 0x55716d732d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0cfaffd640 1 --2- 172.21.15.111:0/3492513650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6c7bb0 0x55716d77b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0cfb7fe640 1 -- 172.21.15.111:0/3492513650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6cd0a0 msgr2=0x55716d732d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:48.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0cfb7fe640 1 --2- 172.21.15.111:0/3492513650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6cd0a0 0x55716d732d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0cfb7fe640 1 -- 172.21.15.111:0/3492513650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6c7bb0 msgr2=0x55716d77b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:48.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0cfb7fe640 1 --2- 172.21.15.111:0/3492513650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6c7bb0 0x55716d77b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0cfb7fe640 1 -- 172.21.15.111:0/3492513650 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55716d5bc4b0 con 0x55716d6f4550 2024-09-13T17:58:48.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0cfb7fe640 1 --2- 172.21.15.111:0/3492513650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6f4550 0x55716d733270 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0cf407f040 tx=0x7f0cf40a66e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec82a34d377c0bc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:48.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0cd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-13T17:58:48.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0cd9ffb640 1 -- 172.21.15.111:0/3492513650 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf407f2e0 con 0x55716d6f4550 2024-09-13T17:58:48.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0cd9ffb640 1 -- 172.21.15.111:0/3492513650 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0cf4079070 con 0x55716d6f4550 2024-09-13T17:58:48.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0d04212740 1 -- 172.21.15.111:0/3492513650 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ce401d5a0 con 0x55716d6f4550 2024-09-13T17:58:48.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0cd9ffb640 1 -- 172.21.15.111:0/3492513650 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cf4076d70 con 0x55716d6f4550 2024-09-13T17:58:48.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0d04212740 10 client.5670 fetch_fsmap learned FSMap version 15 2024-09-13T17:58:48.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0d04212740 1 -- 172.21.15.111:0/3492513650 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55716d5f5d00 con 0x55716d6f4550 2024-09-13T17:58:48.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0cd9ffb640 1 -- 172.21.15.111:0/3492513650 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce401d5a0 con 0x55716d6f4550 2024-09-13T17:58:48.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0cd9ffb640 1 -- 172.21.15.111:0/3492513650 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 15) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0cf4077490 con 0x55716d6f4550 2024-09-13T17:58:48.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0d04212740 10 client.5670 fetch_fsmap finished waiting for FSMap version 15 2024-09-13T17:58:48.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0d04212740 10 client.5670 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T17:58:48.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0d04212740 20 client.5670 populate_metadata read hostname 'smithi111' 2024-09-13T17:58:48.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0d04212740 1 --2- 172.21.15.111:0/3492513650 >> [v2:172.21.15.111:6836/982359636,v1:172.21.15.111:6837/982359636] conn(0x55716d77cfa0 0x55716d77d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0d04212740 4 client.5670 mds_command: new command op to 5490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:58:48.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.033+0000 7f0d04212740 1 -- 172.21.15.111:0/3492513650 --> [v2:172.21.15.111:6836/982359636,v1:172.21.15.111:6837/982359636] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55716d387d10 con 0x55716d77cfa0 2024-09-13T17:58:48.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0cfaffd640 1 --2- 172.21.15.111:0/3492513650 >> [v2:172.21.15.111:6836/982359636,v1:172.21.15.111:6837/982359636] conn(0x55716d77cfa0 0x55716d77d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0cd9ffb640 10 client.5670 ms_handle_connect on v2:172.21.15.111:6836/982359636 2024-09-13T17:58:48.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0cfaffd640 1 --2- 172.21.15.111:0/3492513650 >> [v2:172.21.15.111:6836/982359636,v1:172.21.15.111:6837/982359636] conn(0x55716d77cfa0 0x55716d77d380 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=febfa36a340893bb server_cookie=b0546514282135d7 in_seq=0 out_seq=0 2024-09-13T17:58:48.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0cd9ffb640 1 -- 172.21.15.111:0/3492513650 <== mds.0 v2:172.21.15.111:6836/982359636 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55716d387d10 con 0x55716d77cfa0 2024-09-13T17:58:48.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0cd9ffb640 10 client.5670 handle_command_reply: tid=0 2024-09-13T17:58:48.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0d04212740 1 client.5670 shutdown 2024-09-13T17:58:48.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0d04212740 1 -- 172.21.15.111:0/3492513650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6f4550 msgr2=0x55716d733270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:48.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0d04212740 1 --2- 172.21.15.111:0/3492513650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6f4550 0x55716d733270 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0cf407f040 tx=0x7f0cf40a66e0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0d04212740 1 -- 172.21.15.111:0/3492513650 shutdown_connections 2024-09-13T17:58:48.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0d04212740 1 --2- 172.21.15.111:0/3492513650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55716d6f4550 0x55716d733270 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0d04212740 1 --2- 172.21.15.111:0/3492513650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55716d6cd0a0 0x55716d732d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0d04212740 1 --2- 172.21.15.111:0/3492513650 >> [v2:172.21.15.111:6836/982359636,v1:172.21.15.111:6837/982359636] conn(0x55716d77cfa0 0x55716d77d380 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0d04212740 1 --2- 172.21.15.111:0/3492513650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55716d6c7bb0 0x55716d77b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0cd9ffb640 0 client.5670 ms_handle_reset on v2:172.21.15.111:6836/982359636 2024-09-13T17:58:48.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0d04212740 1 -- 172.21.15.111:0/3492513650 >> 172.21.15.111:0/3492513650 conn(0x55716d6ca860 msgr2=0x55716d719080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:48.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0d04212740 1 -- 172.21.15.111:0/3492513650 shutdown_connections 2024-09-13T17:58:48.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0d04212740 1 -- 172.21.15.111:0/3492513650 wait complete. 2024-09-13T17:58:48.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.034+0000 7f0d04212740 20 client.5670 trim_cache size 0 max 16384 2024-09-13T17:58:48.045 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T17:58:48.045 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "2a8c13ef-d427-4b48-90b3-65459ec79a47", 2024-09-13T17:58:48.045 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2024-09-13T17:58:48.045 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 5490, 2024-09-13T17:58:48.045 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-09-13T17:58:48.045 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-09-13T17:58:48.045 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-09-13T17:58:48.045 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 4.4997445379999998, 2024-09-13T17:58:48.045 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 15, 2024-09-13T17:58:48.045 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 39, 2024-09-13T17:58:48.045 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 39, 2024-09-13T17:58:48.045 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 11.003914493 2024-09-13T17:58:48.045 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T17:58:48.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.035+0000 7f0d01412640 1 -- 172.21.15.111:0/759089972 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0cdc06ac30 msgr2=0x7f0cdc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:48.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.035+0000 7f0d01412640 1 --2- 172.21.15.111:0/759089972 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0cdc06ac30 0x7f0cdc06d0f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0012e0 tx=0x7f0cec001d80 comp rx=0 tx=0).stop 2024-09-13T17:58:48.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.035+0000 7f0d01412640 1 -- 172.21.15.111:0/759089972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cfc15cd40 msgr2=0x7f0cfc15d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:48.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.035+0000 7f0d01412640 1 --2- 172.21.15.111:0/759089972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cfc15cd40 0x7f0cfc15d1c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4001840 tx=0x7f0cf4001120 comp rx=0 tx=0).stop 2024-09-13T17:58:48.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.035+0000 7f0d01412640 1 -- 172.21.15.111:0/759089972 shutdown_connections 2024-09-13T17:58:48.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.035+0000 7f0d01412640 1 --2- 172.21.15.111:0/759089972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cfc15cd40 0x7f0cfc15d1c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.035+0000 7f0d01412640 1 --2- 172.21.15.111:0/759089972 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f0cdc06ac30 0x7f0cdc06d0f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.035+0000 7f0d01412640 1 --2- 172.21.15.111:0/759089972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc0c2610 0x7f0cfc0c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.035+0000 7f0d01412640 1 --2- 172.21.15.111:0/759089972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ce8005180 0x7f0cfc158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.035+0000 7f0d01412640 1 -- 172.21.15.111:0/759089972 >> 172.21.15.111:0/759089972 conn(0x7f0cfc069ed0 msgr2=0x7f0cfc0ad010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:48.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.035+0000 7f0d01412640 1 -- 172.21.15.111:0/759089972 shutdown_connections 2024-09-13T17:58:48.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.035+0000 7f0d01412640 1 -- 172.21.15.111:0/759089972 wait complete. 2024-09-13T17:58:48.048 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:58:48.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.142+0000 7f17af95e640 1 Processor -- start 2024-09-13T17:58:48.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.142+0000 7f17af95e640 1 -- start start 2024-09-13T17:58:48.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.143+0000 7f17af95e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f17a815c690 0x7f17a80b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.143+0000 7f17af95e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f17a80b26e0 0x7f17a80cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.143+0000 7f17af95e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a80b2b10 con 0x7f17a80cace0 2024-09-13T17:58:48.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.143+0000 7f17af95e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a80b2310 con 0x7f17a815c690 2024-09-13T17:58:48.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.143+0000 7f17af95e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a80c9ac0 con 0x7f17a80b26e0 2024-09-13T17:58:48.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.143+0000 7f17acef9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f17a815c690 0x7f17a80b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.143+0000 7f17acef9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f17a815c690 0x7f17a80b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40374/0 (socket says 172.21.15.111:40374) 2024-09-13T17:58:48.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.143+0000 7f17acef9640 1 -- 172.21.15.111:0/1810063523 learned_addr learned my addr 172.21.15.111:0/1810063523 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:48.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.143+0000 7f17adefb640 1 --2- 172.21.15.111:0/1810063523 >> v2:172.21.15.188:3300/0 conn(0x7f17a80b26e0 0x7f17a80cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.143+0000 7f17acef9640 1 -- 172.21.15.111:0/1810063523 >> v2:172.21.15.188:3300/0 conn(0x7f17a80b26e0 msgr2=0x7f17a80cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:48.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.143+0000 7f17acef9640 1 --2- 172.21.15.111:0/1810063523 >> v2:172.21.15.188:3300/0 conn(0x7f17a80b26e0 0x7f17a80cdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.143+0000 7f17acef9640 1 -- 172.21.15.111:0/1810063523 >> v1:172.21.15.188:6789/0 conn(0x7f17a80cace0 legacy=0x7f17a80cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:48.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.143+0000 7f17acef9640 1 -- 172.21.15.111:0/1810063523 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a80c9da0 con 0x7f17a815c690 2024-09-13T17:58:48.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.144+0000 7f17acef9640 1 --2- 172.21.15.111:0/1810063523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17a815c690 0x7f17a80b1ca0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f17a80b15a0 tx=0x7f179802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c5aee6ffa87f8c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:48.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.144+0000 7f179ffff640 1 -- 172.21.15.111:0/1810063523 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1798032440 con 0x7f17a815c690 2024-09-13T17:58:48.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.144+0000 7f179ffff640 1 -- 172.21.15.111:0/1810063523 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17980325a0 con 0x7f17a815c690 2024-09-13T17:58:48.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.144+0000 7f179ffff640 1 -- 172.21.15.111:0/1810063523 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1798032930 con 0x7f17a815c690 2024-09-13T17:58:48.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.145+0000 7f17af95e640 1 -- 172.21.15.111:0/1810063523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17a815c690 msgr2=0x7f17a80b1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:48.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.145+0000 7f17af95e640 1 --2- 172.21.15.111:0/1810063523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17a815c690 0x7f17a80b1ca0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f17a80b15a0 tx=0x7f179802a1a0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.145+0000 7f17af95e640 1 -- 172.21.15.111:0/1810063523 shutdown_connections 2024-09-13T17:58:48.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.145+0000 7f17af95e640 1 --2- 172.21.15.111:0/1810063523 >> v2:172.21.15.188:3300/0 conn(0x7f17a80b26e0 0x7f17a80cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.145+0000 7f17af95e640 1 --2- 172.21.15.111:0/1810063523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17a815c690 0x7f17a80b1ca0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.145+0000 7f17af95e640 1 -- 172.21.15.111:0/1810063523 >> 172.21.15.111:0/1810063523 conn(0x7f17a8069ed0 msgr2=0x7f17a806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:48.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.145+0000 7f17af95e640 1 -- 172.21.15.111:0/1810063523 shutdown_connections 2024-09-13T17:58:48.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.145+0000 7f17af95e640 1 -- 172.21.15.111:0/1810063523 wait complete. 2024-09-13T17:58:48.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.146+0000 7f17af95e640 1 Processor -- start 2024-09-13T17:58:48.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.146+0000 7f17af95e640 1 -- start start 2024-09-13T17:58:48.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.147+0000 7f17af95e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17a80b26e0 0x7f17a80acad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.147+0000 7f17af95e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17a80cace0 0x7f17a806d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.147+0000 7f17af95e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17a806a2b0 0x7f17a80c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.147+0000 7f17af95e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a806d380 con 0x7f17a80cace0 2024-09-13T17:58:48.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.147+0000 7f17af95e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a81529e0 con 0x7f17a806a2b0 2024-09-13T17:58:48.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.147+0000 7f17af95e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a80b05f0 con 0x7f17a80b26e0 2024-09-13T17:58:48.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.147+0000 7f17acef9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17a80cace0 0x7f17a806d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.147+0000 7f17acef9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17a80cace0 0x7f17a806d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40380/0 (socket says 172.21.15.111:40380) 2024-09-13T17:58:48.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.147+0000 7f17acef9640 1 -- 172.21.15.111:0/4152887023 learned_addr learned my addr 172.21.15.111:0/4152887023 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:48.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.147+0000 7f17adefb640 1 --2- 172.21.15.111:0/4152887023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17a806a2b0 0x7f17a80c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.147+0000 7f17acef9640 1 -- 172.21.15.111:0/4152887023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17a80b26e0 msgr2=0x7f17a80acad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:48.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.147+0000 7f17ad6fa640 1 --2- 172.21.15.111:0/4152887023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17a80b26e0 0x7f17a80acad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.147+0000 7f17acef9640 1 --2- 172.21.15.111:0/4152887023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17a80b26e0 0x7f17a80acad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.147+0000 7f17acef9640 1 -- 172.21.15.111:0/4152887023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17a806a2b0 msgr2=0x7f17a80c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:48.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.147+0000 7f17acef9640 1 --2- 172.21.15.111:0/4152887023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17a806a2b0 0x7f17a80c53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.147+0000 7f17acef9640 1 -- 172.21.15.111:0/4152887023 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17980094d0 con 0x7f17a80cace0 2024-09-13T17:58:48.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.148+0000 7f17acef9640 1 --2- 172.21.15.111:0/4152887023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17a80cace0 0x7f17a806d7a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1798000c00 tx=0x7f17980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:48.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.148+0000 7f179dffb640 1 -- 172.21.15.111:0/4152887023 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1798032440 con 0x7f17a80cace0 2024-09-13T17:58:48.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.148+0000 7f179dffb640 1 -- 172.21.15.111:0/4152887023 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1798030b80 con 0x7f17a80cace0 2024-09-13T17:58:48.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.148+0000 7f17af95e640 1 -- 172.21.15.111:0/4152887023 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a8152150 con 0x7f17a80cace0 2024-09-13T17:58:48.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.148+0000 7f179dffb640 1 -- 172.21.15.111:0/4152887023 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179803d420 con 0x7f17a80cace0 2024-09-13T17:58:48.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.148+0000 7f17af95e640 1 -- 172.21.15.111:0/4152887023 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a80ad1f0 con 0x7f17a80cace0 2024-09-13T17:58:48.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.149+0000 7f17af95e640 1 -- 172.21.15.111:0/4152887023 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a806d0e0 con 0x7f17a80cace0 2024-09-13T17:58:48.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.149+0000 7f179dffb640 1 -- 172.21.15.111:0/4152887023 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1798039040 con 0x7f17a80cace0 2024-09-13T17:58:48.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.150+0000 7f179dffb640 1 --2- 172.21.15.111:0/4152887023 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f178c06ac00 0x7f178c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:48.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.150+0000 7f179dffb640 1 -- 172.21.15.111:0/4152887023 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f17980a9fa0 con 0x7f17a80cace0 2024-09-13T17:58:48.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.150+0000 7f17ad6fa640 1 --2- 172.21.15.111:0/4152887023 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f178c06ac00 0x7f178c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:48.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.150+0000 7f17ad6fa640 1 --2- 172.21.15.111:0/4152887023 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f178c06ac00 0x7f178c06d0c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f17a40015d0 tx=0x7f17a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:48.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.154+0000 7f179dffb640 1 -- 172.21.15.111:0/4152887023 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f17980dd9c0 con 0x7f17a80cace0 2024-09-13T17:58:48.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.353+0000 7f17af95e640 1 -- 172.21.15.111:0/4152887023 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f17a806d4c0 con 0x7f17a80cace0 2024-09-13T17:58:48.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.353+0000 7f179dffb640 1 -- 172.21.15.111:0/4152887023 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f17a806d4c0 con 0x7f17a80cace0 2024-09-13T17:58:48.356 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T17:58:48.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.355+0000 7f17af95e640 1 -- 172.21.15.111:0/4152887023 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f178c06ac00 msgr2=0x7f178c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:48.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.355+0000 7f17af95e640 1 --2- 172.21.15.111:0/4152887023 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f178c06ac00 0x7f178c06d0c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f17a40015d0 tx=0x7f17a4009040 comp rx=0 tx=0).stop 2024-09-13T17:58:48.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.355+0000 7f17af95e640 1 -- 172.21.15.111:0/4152887023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17a80cace0 msgr2=0x7f17a806d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:48.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.355+0000 7f17af95e640 1 --2- 172.21.15.111:0/4152887023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17a80cace0 0x7f17a806d7a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1798000c00 tx=0x7f17980046c0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.355+0000 7f17af95e640 1 -- 172.21.15.111:0/4152887023 shutdown_connections 2024-09-13T17:58:48.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.355+0000 7f17af95e640 1 --2- 172.21.15.111:0/4152887023 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f178c06ac00 0x7f178c06d0c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.355+0000 7f17af95e640 1 --2- 172.21.15.111:0/4152887023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17a806a2b0 0x7f17a80c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.355+0000 7f17af95e640 1 --2- 172.21.15.111:0/4152887023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17a80cace0 0x7f17a806d7a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.355+0000 7f17af95e640 1 --2- 172.21.15.111:0/4152887023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17a80b26e0 0x7f17a80acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:48.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.355+0000 7f17af95e640 1 -- 172.21.15.111:0/4152887023 >> 172.21.15.111:0/4152887023 conn(0x7f17a8069ed0 msgr2=0x7f17a80afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:48.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.355+0000 7f17af95e640 1 -- 172.21.15.111:0/4152887023 shutdown_connections 2024-09-13T17:58:48.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:48.355+0000 7f17af95e640 1 -- 172.21.15.111:0/4152887023 wait complete. 2024-09-13T17:58:48.369 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:58:48.369 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:58:48.369 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2024-09-13T17:58:48.369 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T17:58:48.369 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T17:58:48.369 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:58:48.369 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:58:48.369 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:58:48.369 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:58:48.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:48.370 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2024-09-13T17:58:48.385 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:58:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:58:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:58:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-09-13T17:58:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:48.386 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:58:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:58:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2024-09-13T17:58:48.386 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:58:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:58:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2024-09-13T17:58:48.386 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:58:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2024-09-13T17:58:48.387 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:58:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2024-09-13T17:58:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-13T17:58:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42731sec preferred_lft 42731sec 2024-09-13T17:58:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link noprefixroute 2024-09-13T17:58:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:48.387 INFO:teuthology.orchestra.run.smithi111.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:58:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 4a:ef:99:2a:0b:cf brd ff:ff:ff:ff:ff:ff 2024-09-13T17:58:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T17:58:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::d4ae:99ff:fea4:a069/64 scope link 2024-09-13T17:58:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:48.388 INFO:teuthology.orchestra.run.smithi111.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:58:48.388 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether da:3f:2a:41:1d:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:58:48.388 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::d83f:2aff:fe41:1ddc/64 scope link 2024-09-13T17:58:48.388 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:48.388 INFO:teuthology.orchestra.run.smithi111.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:58:48.388 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 4a:ef:99:2a:0b:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T17:58:48.388 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::48ef:99ff:fe2a:bcf/64 scope link 2024-09-13T17:58:48.388 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:58:48.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:48.388 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:58:48.449 INFO:teuthology.orchestra.run.smithi111.stdout:1 2024-09-13T17:58:48.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:48.450 DEBUG:teuthology.orchestra.run.smithi111:> route 2024-09-13T17:58:48.507 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2024-09-13T17:58:48.507 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:58:48.507 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:58:48.507 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:58:48.507 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:58:48.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:48.508 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2024-09-13T17:58:48.508 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-13T17:58:48.508 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:58:48.508 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:58:48.508 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:58:48.508 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-13T17:58:48.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:48.594 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-09-13T17:58:48.609 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T17:58:48.609 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:58:48.609 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:58:48.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:48.610 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2024-09-13T17:58:48.675 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-13T17:58:48.675 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:58:48.675 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T17:58:48.730 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:58:48.730 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:58:48.730 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:58:48.785 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2024-09-13T17:58:48.851 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:58:48.851 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:58:48.915 INFO:teuthology.orchestra.run.smithi111.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:58:48.916 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:58:48.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:48.916 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2024-09-13T17:58:48.972 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T17:58:48.972 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-13T17:58:49.014 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2024-09-13T17:58:49.038 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:58:49.038 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:58:49.095 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:2024-09-13T17:58:49.092+0000 7f71e694d480 -1 init, newargv = 0x55ebbf40ce90 newargc=15 2024-09-13T17:58:49.095 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[53181]: starting ceph client 2024-09-13T17:58:49.099 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[53181]: starting fuse 2024-09-13T17:58:49.104 INFO:teuthology.orchestra.run.smithi111.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:58:49.105 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:58:49.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:49.106 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2024-09-13T17:58:49.162 INFO:teuthology.orchestra.run.smithi111.stdout:41 2024-09-13T17:58:49.162 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-13T17:58:49.162 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:58:49.162 DEBUG:teuthology.orchestra.run.smithi111:> import glob 2024-09-13T17:58:49.162 DEBUG:teuthology.orchestra.run.smithi111:> import re 2024-09-13T17:58:49.163 DEBUG:teuthology.orchestra.run.smithi111:> import os 2024-09-13T17:58:49.163 DEBUG:teuthology.orchestra.run.smithi111:> import subprocess 2024-09-13T17:58:49.163 DEBUG:teuthology.orchestra.run.smithi111:> 2024-09-13T17:58:49.163 DEBUG:teuthology.orchestra.run.smithi111:> def _find_admin_socket(client_name): 2024-09-13T17:58:49.163 DEBUG:teuthology.orchestra.run.smithi111:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T17:58:49.163 DEBUG:teuthology.orchestra.run.smithi111:> files = glob.glob(asok_path) 2024-09-13T17:58:49.163 DEBUG:teuthology.orchestra.run.smithi111:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T17:58:49.163 DEBUG:teuthology.orchestra.run.smithi111:> 2024-09-13T17:58:49.163 DEBUG:teuthology.orchestra.run.smithi111:> # Given a non-glob path, it better be there 2024-09-13T17:58:49.163 DEBUG:teuthology.orchestra.run.smithi111:> if "*" not in asok_path: 2024-09-13T17:58:49.163 DEBUG:teuthology.orchestra.run.smithi111:> assert(len(files) == 1) 2024-09-13T17:58:49.163 DEBUG:teuthology.orchestra.run.smithi111:> return files[0] 2024-09-13T17:58:49.163 DEBUG:teuthology.orchestra.run.smithi111:> 2024-09-13T17:58:49.163 DEBUG:teuthology.orchestra.run.smithi111:> for f in files: 2024-09-13T17:58:49.163 DEBUG:teuthology.orchestra.run.smithi111:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:58:49.163 DEBUG:teuthology.orchestra.run.smithi111:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:58:49.164 DEBUG:teuthology.orchestra.run.smithi111:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:58:49.164 DEBUG:teuthology.orchestra.run.smithi111:> contents = proc_f.read() 2024-09-13T17:58:49.164 DEBUG:teuthology.orchestra.run.smithi111:> if mountpoint in contents: 2024-09-13T17:58:49.164 DEBUG:teuthology.orchestra.run.smithi111:> return f 2024-09-13T17:58:49.164 DEBUG:teuthology.orchestra.run.smithi111:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:58:49.164 DEBUG:teuthology.orchestra.run.smithi111:> 2024-09-13T17:58:49.164 DEBUG:teuthology.orchestra.run.smithi111:> print(_find_admin_socket("client.0")) 2024-09-13T17:58:49.164 DEBUG:teuthology.orchestra.run.smithi111:> ' 2024-09-13T17:58:49.276 INFO:teuthology.orchestra.run.smithi111.stdout:/var/run/ceph/ceph-client.0.53181.asok 2024-09-13T17:58:49.466 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53181.asok 2024-09-13T17:58:49.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:49.466 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53181.asok status 2024-09-13T17:58:49.600 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T17:58:49.600 INFO:teuthology.orchestra.run.smithi111.stdout: "metadata": { 2024-09-13T17:58:49.600 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:58:49.601 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:58:49.601 INFO:teuthology.orchestra.run.smithi111.stdout: "entity_id": "0", 2024-09-13T17:58:49.601 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": "smithi111", 2024-09-13T17:58:49.601 INFO:teuthology.orchestra.run.smithi111.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T17:58:49.601 INFO:teuthology.orchestra.run.smithi111.stdout: "pid": "53181", 2024-09-13T17:58:49.601 INFO:teuthology.orchestra.run.smithi111.stdout: "root": "/" 2024-09-13T17:58:49.601 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:49.601 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_count": 0, 2024-09-13T17:58:49.601 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_pinned_count": 0, 2024-09-13T17:58:49.601 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 5685, 2024-09-13T17:58:49.601 INFO:teuthology.orchestra.run.smithi111.stdout: "inst": { 2024-09-13T17:58:49.601 INFO:teuthology.orchestra.run.smithi111.stdout: "name": { 2024-09-13T17:58:49.601 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "client", 2024-09-13T17:58:49.602 INFO:teuthology.orchestra.run.smithi111.stdout: "num": 5685 2024-09-13T17:58:49.602 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:49.602 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2024-09-13T17:58:49.602 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2024-09-13T17:58:49.602 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:58:49.602 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 2807546433 2024-09-13T17:58:49.602 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T17:58:49.602 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:49.602 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2024-09-13T17:58:49.602 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2024-09-13T17:58:49.602 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:58:49.602 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 2807546433 2024-09-13T17:58:49.602 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T17:58:49.603 INFO:teuthology.orchestra.run.smithi111.stdout: "inst_str": "client.5685 192.168.0.1:0/2807546433", 2024-09-13T17:58:49.603 INFO:teuthology.orchestra.run.smithi111.stdout: "addr_str": "192.168.0.1:0/2807546433", 2024-09-13T17:58:49.603 INFO:teuthology.orchestra.run.smithi111.stdout: "inode_count": 1, 2024-09-13T17:58:49.603 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_epoch": 15, 2024-09-13T17:58:49.603 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch": 39, 2024-09-13T17:58:49.603 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:58:49.603 INFO:teuthology.orchestra.run.smithi111.stdout: "blocklisted": false, 2024-09-13T17:58:49.603 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs" 2024-09-13T17:58:49.603 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T17:58:49.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:49.608 DEBUG:teuthology.orchestra.run.smithi111:> stat --file-system '--printf=%T 2024-09-13T17:58:49.608 DEBUG:teuthology.orchestra.run.smithi111:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T17:58:49.624 INFO:teuthology.orchestra.run.smithi111.stdout:fuseblk 2024-09-13T17:58:49.625 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T17:58:49.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:49.625 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:58:49.693 DEBUG:teuthology.orchestra.run.smithi111:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:58:49.693 DEBUG:teuthology.orchestra.run.smithi111:> import os 2024-09-13T17:58:49.693 DEBUG:teuthology.orchestra.run.smithi111:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir") 2024-09-13T17:58:49.693 DEBUG:teuthology.orchestra.run.smithi111:> fd = os.open("/home/ubuntu/cephtest/mnt.0/testdir", os.O_RDONLY) 2024-09-13T17:58:49.693 DEBUG:teuthology.orchestra.run.smithi111:> os.fchmod(fd, 0o777) 2024-09-13T17:58:49.693 DEBUG:teuthology.orchestra.run.smithi111:> os.fsync(fd) 2024-09-13T17:58:49.694 DEBUG:teuthology.orchestra.run.smithi111:> ' 2024-09-13T17:58:49.984 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:58:50.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.121+0000 7f255b545640 1 Processor -- start 2024-09-13T17:58:50.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.122+0000 7f255b545640 1 -- start start 2024-09-13T17:58:50.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.122+0000 7f255b545640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2554071d30 0x7f2554070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.122+0000 7f255b545640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2554072110 0x7f25540b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.122+0000 7f255b545640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25540abdf0 con 0x7f25540c8920 2024-09-13T17:58:50.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.122+0000 7f255b545640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25540ab1e0 con 0x7f2554071d30 2024-09-13T17:58:50.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.122+0000 7f255b545640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25540ab4c0 con 0x7f2554072110 2024-09-13T17:58:50.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.123+0000 7f25592e1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2554071d30 0x7f2554070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.123+0000 7f25592e1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2554071d30 0x7f2554070190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.111:43774/0 (socket says 172.21.15.111:43774) 2024-09-13T17:58:50.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.123+0000 7f25592e1640 1 -- 172.21.15.111:0/2576247974 learned_addr learned my addr 172.21.15.111:0/2576247974 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:50.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.123+0000 7f2559ae2640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f25540c8920 0x7f25540ac280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:46718/0 (socket says 172.21.15.111:46718) 2024-09-13T17:58:50.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.123+0000 7f2558ae0640 1 --2- 172.21.15.111:0/2576247974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2554072110 0x7f25540b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.123+0000 7f25592e1640 1 -- 172.21.15.111:0/2576247974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2554072110 msgr2=0x7f25540b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:50.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.123+0000 7f25592e1640 1 --2- 172.21.15.111:0/2576247974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2554072110 0x7f25540b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.123+0000 7f25592e1640 1 -- 172.21.15.111:0/2576247974 >> v1:172.21.15.188:6789/0 conn(0x7f25540c8920 legacy=0x7f25540ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.123+0000 7f25592e1640 1 -- 172.21.15.111:0/2576247974 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25540ab7a0 con 0x7f2554071d30 2024-09-13T17:58:50.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.124+0000 7f25592e1640 1 --2- 172.21.15.111:0/2576247974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2554071d30 0x7f2554070190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f254c0098a0 tx=0x7f254c02c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=129c96e1027914c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:50.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.124+0000 7f2553fff640 1 -- 172.21.15.111:0/2576247974 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c031710 con 0x7f2554071d30 2024-09-13T17:58:50.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.124+0000 7f2553fff640 1 -- 172.21.15.111:0/2576247974 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254c02f050 con 0x7f2554071d30 2024-09-13T17:58:50.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.124+0000 7f2553fff640 1 -- 172.21.15.111:0/2576247974 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c031a90 con 0x7f2554071d30 2024-09-13T17:58:50.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.125+0000 7f255b545640 1 -- 172.21.15.111:0/2576247974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2554071d30 msgr2=0x7f2554070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.125+0000 7f255b545640 1 --2- 172.21.15.111:0/2576247974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2554071d30 0x7f2554070190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f254c0098a0 tx=0x7f254c02c4a0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.125+0000 7f255b545640 1 -- 172.21.15.111:0/2576247974 shutdown_connections 2024-09-13T17:58:50.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.125+0000 7f255b545640 1 --2- 172.21.15.111:0/2576247974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2554072110 0x7f25540b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.125+0000 7f255b545640 1 --2- 172.21.15.111:0/2576247974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2554071d30 0x7f2554070190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.125+0000 7f255b545640 1 -- 172.21.15.111:0/2576247974 >> 172.21.15.111:0/2576247974 conn(0x7f2554069ed0 msgr2=0x7f255406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:50.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.125+0000 7f255b545640 1 -- 172.21.15.111:0/2576247974 shutdown_connections 2024-09-13T17:58:50.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.125+0000 7f255b545640 1 -- 172.21.15.111:0/2576247974 wait complete. 2024-09-13T17:58:50.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.126+0000 7f255b545640 1 Processor -- start 2024-09-13T17:58:50.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.126+0000 7f255b545640 1 -- start start 2024-09-13T17:58:50.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f255b545640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2554071d30 0x7f25540c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f255b545640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2554072110 0x7f25540979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f255b545640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25540c8920 0x7f255406d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f255b545640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2554097ef0 con 0x7f25540c8920 2024-09-13T17:58:50.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f255b545640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25540c2690 con 0x7f2554071d30 2024-09-13T17:58:50.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f255b545640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25540c1a80 con 0x7f2554072110 2024-09-13T17:58:50.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f2559ae2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25540c8920 0x7f255406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f2559ae2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25540c8920 0x7f255406d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40384/0 (socket says 172.21.15.111:40384) 2024-09-13T17:58:50.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f2559ae2640 1 -- 172.21.15.111:0/3938099546 learned_addr learned my addr 172.21.15.111:0/3938099546 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:50.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f25592e1640 1 --2- 172.21.15.111:0/3938099546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2554071d30 0x7f25540c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f2558ae0640 1 --2- 172.21.15.111:0/3938099546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2554072110 0x7f25540979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f2559ae2640 1 -- 172.21.15.111:0/3938099546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2554072110 msgr2=0x7f25540979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f2559ae2640 1 --2- 172.21.15.111:0/3938099546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2554072110 0x7f25540979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f2559ae2640 1 -- 172.21.15.111:0/3938099546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2554071d30 msgr2=0x7f25540c2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f2559ae2640 1 --2- 172.21.15.111:0/3938099546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2554071d30 0x7f25540c2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f2559ae2640 1 -- 172.21.15.111:0/3938099546 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c0094d0 con 0x7f25540c8920 2024-09-13T17:58:50.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f25592e1640 1 --2- 172.21.15.111:0/3938099546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2554071d30 0x7f25540c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:50.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f2559ae2640 1 --2- 172.21.15.111:0/3938099546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25540c8920 0x7f255406d5f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f254400c7b0 tx=0x7f254400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:50.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f2551ffb640 1 -- 172.21.15.111:0/3938099546 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254400a640 con 0x7f25540c8920 2024-09-13T17:58:50.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f2551ffb640 1 -- 172.21.15.111:0/3938099546 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254400a7a0 con 0x7f25540c8920 2024-09-13T17:58:50.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f2551ffb640 1 -- 172.21.15.111:0/3938099546 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254400ab10 con 0x7f25540c8920 2024-09-13T17:58:50.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.127+0000 7f255b545640 1 -- 172.21.15.111:0/3938099546 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25540c1d60 con 0x7f25540c8920 2024-09-13T17:58:50.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.128+0000 7f255b545640 1 -- 172.21.15.111:0/3938099546 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25540c2320 con 0x7f25540c8920 2024-09-13T17:58:50.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.129+0000 7f255b545640 1 -- 172.21.15.111:0/3938099546 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2554070190 con 0x7f25540c8920 2024-09-13T17:58:50.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.129+0000 7f2551ffb640 1 -- 172.21.15.111:0/3938099546 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f254400ac70 con 0x7f25540c8920 2024-09-13T17:58:50.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.129+0000 7f2551ffb640 1 --2- 172.21.15.111:0/3938099546 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f254006ac00 0x7f254006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.130+0000 7f2551ffb640 1 -- 172.21.15.111:0/3938099546 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2544088c80 con 0x7f25540c8920 2024-09-13T17:58:50.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.130+0000 7f25592e1640 1 --2- 172.21.15.111:0/3938099546 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f254006ac00 0x7f254006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.130+0000 7f25592e1640 1 --2- 172.21.15.111:0/3938099546 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f254006ac00 0x7f254006d0c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f254c035000 tx=0x7f254c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:50.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.134+0000 7f2551ffb640 1 -- 172.21.15.111:0/3938099546 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f2544053d40 con 0x7f25540c8920 2024-09-13T17:58:50.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.327+0000 7f255b545640 1 -- 172.21.15.111:0/3938099546 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25540c27d0 con 0x7f25540c8920 2024-09-13T17:58:50.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.328+0000 7f2551ffb640 1 -- 172.21.15.111:0/3938099546 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f2544017070 con 0x7f25540c8920 2024-09-13T17:58:50.331 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:50.331 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4217138481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4217138481},{"type":"v1","addr":"172.21.15.188:6834","nonce":4217138481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3099948287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3099948287},{"type":"v1","addr":"172.21.15.188:6835","nonce":3099948287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/214034224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":214034224},{"type":"v1","addr":"172.21.15.111:6835","nonce":214034224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:58:43.534567+0000","modified":"2024-09-13T17:58:45.542773+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5490":{"gid":5490,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6837/982359636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":982359636},{"type":"v1","addr":"172.21.15.111:6837","nonce":982359636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:58:50.331 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 15 2024-09-13T17:58:50.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.330+0000 7f255b545640 1 -- 172.21.15.111:0/3938099546 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f254006ac00 msgr2=0x7f254006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.330+0000 7f255b545640 1 --2- 172.21.15.111:0/3938099546 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f254006ac00 0x7f254006d0c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f254c035000 tx=0x7f254c008030 comp rx=0 tx=0).stop 2024-09-13T17:58:50.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.330+0000 7f255b545640 1 -- 172.21.15.111:0/3938099546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25540c8920 msgr2=0x7f255406d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.330+0000 7f255b545640 1 --2- 172.21.15.111:0/3938099546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25540c8920 0x7f255406d5f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f254400c7b0 tx=0x7f254400cc70 comp rx=0 tx=0).stop 2024-09-13T17:58:50.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.330+0000 7f255b545640 1 -- 172.21.15.111:0/3938099546 shutdown_connections 2024-09-13T17:58:50.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.330+0000 7f255b545640 1 --2- 172.21.15.111:0/3938099546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25540c8920 0x7f255406d5f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.330+0000 7f255b545640 1 --2- 172.21.15.111:0/3938099546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2554072110 0x7f25540979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.330+0000 7f255b545640 1 --2- 172.21.15.111:0/3938099546 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f254006ac00 0x7f254006d0c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.330+0000 7f255b545640 1 --2- 172.21.15.111:0/3938099546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2554071d30 0x7f25540c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.330+0000 7f255b545640 1 -- 172.21.15.111:0/3938099546 >> 172.21.15.111:0/3938099546 conn(0x7f2554069ed0 msgr2=0x7f25540afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:50.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.330+0000 7f255b545640 1 -- 172.21.15.111:0/3938099546 shutdown_connections 2024-09-13T17:58:50.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.330+0000 7f255b545640 1 -- 172.21.15.111:0/3938099546 wait complete. 2024-09-13T17:58:50.344 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-09-13T17:58:50.462 INFO:teuthology.orchestra.run.smithi111.stdout:{"message":"","return_code":0} 2024-09-13T17:58:50.462 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-13T17:58:50.463 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:58:50.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.557+0000 7f6b14d9c640 1 Processor -- start 2024-09-13T17:58:50.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.557+0000 7f6b14d9c640 1 -- start start 2024-09-13T17:58:50.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.558+0000 7f6b14d9c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6b1015c6e0 0x7f6b100b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.558+0000 7f6b14d9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b100ccb20 0x7f6b100b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.558+0000 7f6b14d9c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b100d1840 con 0x7f6b100d07b0 2024-09-13T17:58:50.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.558+0000 7f6b14d9c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1006bd20 con 0x7f6b1015c6e0 2024-09-13T17:58:50.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.558+0000 7f6b14d9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100d0c30 con 0x7f6b100ccb20 2024-09-13T17:58:50.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.558+0000 7f6b0e59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6b1015c6e0 0x7f6b100b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.559+0000 7f6b0e59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6b1015c6e0 0x7f6b100b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40394/0 (socket says 172.21.15.111:40394) 2024-09-13T17:58:50.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.559+0000 7f6b0e59c640 1 -- 172.21.15.111:0/1136485285 learned_addr learned my addr 172.21.15.111:0/1136485285 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:50.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.559+0000 7f6b0ed9d640 1 --2- 172.21.15.111:0/1136485285 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b100ccb20 0x7f6b100b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.559+0000 7f6b0e59c640 1 -- 172.21.15.111:0/1136485285 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b100ccb20 msgr2=0x7f6b100b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:50.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.559+0000 7f6b0e59c640 1 --2- 172.21.15.111:0/1136485285 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b100ccb20 0x7f6b100b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.559+0000 7f6b0e59c640 1 -- 172.21.15.111:0/1136485285 >> v1:172.21.15.188:6789/0 conn(0x7f6b100d07b0 legacy=0x7f6b10160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:50.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.559+0000 7f6b0e59c640 1 -- 172.21.15.111:0/1136485285 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b100d0f10 con 0x7f6b1015c6e0 2024-09-13T17:58:50.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.559+0000 7f6b0e59c640 1 --2- 172.21.15.111:0/1136485285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b1015c6e0 0x7f6b100b3f50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009e50 tx=0x7f6b0002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32ba39d1f59675b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:50.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.560+0000 7f6b0d59a640 1 -- 172.21.15.111:0/1136485285 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00031600 con 0x7f6b1015c6e0 2024-09-13T17:58:50.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.560+0000 7f6b0d59a640 1 -- 172.21.15.111:0/1136485285 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0002d050 con 0x7f6b1015c6e0 2024-09-13T17:58:50.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.560+0000 7f6b0d59a640 1 -- 172.21.15.111:0/1136485285 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00031980 con 0x7f6b1015c6e0 2024-09-13T17:58:50.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.560+0000 7f6b14d9c640 1 -- 172.21.15.111:0/1136485285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b1015c6e0 msgr2=0x7f6b100b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.560+0000 7f6b14d9c640 1 --2- 172.21.15.111:0/1136485285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b1015c6e0 0x7f6b100b3f50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009e50 tx=0x7f6b0002a630 comp rx=0 tx=0).stop 2024-09-13T17:58:50.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.560+0000 7f6b14d9c640 1 -- 172.21.15.111:0/1136485285 shutdown_connections 2024-09-13T17:58:50.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.560+0000 7f6b14d9c640 1 --2- 172.21.15.111:0/1136485285 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b100ccb20 0x7f6b100b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.560+0000 7f6b14d9c640 1 --2- 172.21.15.111:0/1136485285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b1015c6e0 0x7f6b100b3f50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.560+0000 7f6b14d9c640 1 -- 172.21.15.111:0/1136485285 >> 172.21.15.111:0/1136485285 conn(0x7f6b10089f60 msgr2=0x7f6b100d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:50.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.560+0000 7f6b14d9c640 1 -- 172.21.15.111:0/1136485285 shutdown_connections 2024-09-13T17:58:50.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.560+0000 7f6b14d9c640 1 -- 172.21.15.111:0/1136485285 wait complete. 2024-09-13T17:58:50.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.562+0000 7f6b14d9c640 1 Processor -- start 2024-09-13T17:58:50.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.562+0000 7f6b14d9c640 1 -- start start 2024-09-13T17:58:50.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.562+0000 7f6b14d9c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b100ccb20 0x7f6b10154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.562+0000 7f6b14d9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b100d07b0 0x7f6b100b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.562+0000 7f6b14d9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1015c6e0 0x7f6b100b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.562+0000 7f6b14d9c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b10154fb0 con 0x7f6b100ccb20 2024-09-13T17:58:50.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.562+0000 7f6b14d9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100b6b10 con 0x7f6b1015c6e0 2024-09-13T17:58:50.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.562+0000 7f6b14d9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100cafd0 con 0x7f6b100d07b0 2024-09-13T17:58:50.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.562+0000 7f6b0e59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b100ccb20 0x7f6b10154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.562+0000 7f6b0e59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b100ccb20 0x7f6b10154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40398/0 (socket says 172.21.15.111:40398) 2024-09-13T17:58:50.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.562+0000 7f6b0e59c640 1 -- 172.21.15.111:0/829542750 learned_addr learned my addr 172.21.15.111:0/829542750 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:58:50.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.563+0000 7f6b0ed9d640 1 --2- 172.21.15.111:0/829542750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1015c6e0 0x7f6b100b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.563+0000 7f6b0dd9b640 1 --2- 172.21.15.111:0/829542750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b100d07b0 0x7f6b100b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.563+0000 7f6b0e59c640 1 -- 172.21.15.111:0/829542750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b100d07b0 msgr2=0x7f6b100b78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.563+0000 7f6b0e59c640 1 --2- 172.21.15.111:0/829542750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b100d07b0 0x7f6b100b78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.563+0000 7f6b0e59c640 1 -- 172.21.15.111:0/829542750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1015c6e0 msgr2=0x7f6b100b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.563+0000 7f6b0e59c640 1 --2- 172.21.15.111:0/829542750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1015c6e0 0x7f6b100b7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.563+0000 7f6b0e59c640 1 -- 172.21.15.111:0/829542750 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b00009b30 con 0x7f6b100ccb20 2024-09-13T17:58:50.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.563+0000 7f6b0dd9b640 1 --2- 172.21.15.111:0/829542750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b100d07b0 0x7f6b100b78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:50.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.563+0000 7f6b0e59c640 1 --2- 172.21.15.111:0/829542750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b100ccb20 0x7f6b10154a70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6b00034040 tx=0x7f6b00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:50.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.563+0000 7f6afeffd640 1 -- 172.21.15.111:0/829542750 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0003b810 con 0x7f6b100ccb20 2024-09-13T17:58:50.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.563+0000 7f6afeffd640 1 -- 172.21.15.111:0/829542750 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0003b970 con 0x7f6b100ccb20 2024-09-13T17:58:50.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.563+0000 7f6b14d9c640 1 -- 172.21.15.111:0/829542750 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b100b5f00 con 0x7f6b100ccb20 2024-09-13T17:58:50.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.563+0000 7f6afeffd640 1 -- 172.21.15.111:0/829542750 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0003bd20 con 0x7f6b100ccb20 2024-09-13T17:58:50.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.563+0000 7f6b14d9c640 1 -- 172.21.15.111:0/829542750 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b100b64c0 con 0x7f6b100ccb20 2024-09-13T17:58:50.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.564+0000 7f6b14d9c640 1 -- 172.21.15.111:0/829542750 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b100cdd40 con 0x7f6b100ccb20 2024-09-13T17:58:50.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.564+0000 7f6afeffd640 1 -- 172.21.15.111:0/829542750 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6b0003a3a0 con 0x7f6b100ccb20 2024-09-13T17:58:50.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.565+0000 7f6afeffd640 1 --2- 172.21.15.111:0/829542750 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6af006ac00 0x7f6af006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.565+0000 7f6afeffd640 1 -- 172.21.15.111:0/829542750 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6b000a9fd0 con 0x7f6b100ccb20 2024-09-13T17:58:50.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.565+0000 7f6b0dd9b640 1 --2- 172.21.15.111:0/829542750 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6af006ac00 0x7f6af006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.565+0000 7f6b0dd9b640 1 --2- 172.21.15.111:0/829542750 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6af006ac00 0x7f6af006d0c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6b101f5970 tx=0x7f6b04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:50.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.569+0000 7f6afeffd640 1 -- 172.21.15.111:0/829542750 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f6b000751b0 con 0x7f6b100ccb20 2024-09-13T17:58:50.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.765+0000 7f6b14d9c640 1 -- 172.21.15.111:0/829542750 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b100b4c80 con 0x7f6b100ccb20 2024-09-13T17:58:50.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.765+0000 7f6afeffd640 1 -- 172.21.15.111:0/829542750 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f6b00009510 con 0x7f6b100ccb20 2024-09-13T17:58:50.768 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:58:50.768 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4217138481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4217138481},{"type":"v1","addr":"172.21.15.188:6834","nonce":4217138481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3099948287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3099948287},{"type":"v1","addr":"172.21.15.188:6835","nonce":3099948287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/214034224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":214034224},{"type":"v1","addr":"172.21.15.111:6835","nonce":214034224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:58:43.534567+0000","modified":"2024-09-13T17:58:45.542773+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5490":{"gid":5490,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6837/982359636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":982359636},{"type":"v1","addr":"172.21.15.111:6837","nonce":982359636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:58:50.768 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 15 2024-09-13T17:58:50.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.767+0000 7f6b14d9c640 1 -- 172.21.15.111:0/829542750 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6af006ac00 msgr2=0x7f6af006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.767+0000 7f6b14d9c640 1 --2- 172.21.15.111:0/829542750 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6af006ac00 0x7f6af006d0c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6b101f5970 tx=0x7f6b04009040 comp rx=0 tx=0).stop 2024-09-13T17:58:50.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.767+0000 7f6b14d9c640 1 -- 172.21.15.111:0/829542750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b100ccb20 msgr2=0x7f6b10154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.767+0000 7f6b14d9c640 1 --2- 172.21.15.111:0/829542750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b100ccb20 0x7f6b10154a70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6b00034040 tx=0x7f6b00004a40 comp rx=0 tx=0).stop 2024-09-13T17:58:50.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.767+0000 7f6b14d9c640 1 -- 172.21.15.111:0/829542750 shutdown_connections 2024-09-13T17:58:50.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.767+0000 7f6b14d9c640 1 --2- 172.21.15.111:0/829542750 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6af006ac00 0x7f6af006d0c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.767+0000 7f6b14d9c640 1 --2- 172.21.15.111:0/829542750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1015c6e0 0x7f6b100b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.767+0000 7f6b14d9c640 1 --2- 172.21.15.111:0/829542750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b100d07b0 0x7f6b100b78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.767+0000 7f6b14d9c640 1 --2- 172.21.15.111:0/829542750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b100ccb20 0x7f6b10154a70 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.767+0000 7f6b14d9c640 1 -- 172.21.15.111:0/829542750 >> 172.21.15.111:0/829542750 conn(0x7f6b10089f60 msgr2=0x7f6b1006ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:50.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.767+0000 7f6b14d9c640 1 -- 172.21.15.111:0/829542750 shutdown_connections 2024-09-13T17:58:50.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:58:50.767+0000 7f6b14d9c640 1 -- 172.21.15.111:0/829542750 wait complete. 2024-09-13T17:58:50.781 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_log_pause 1 2024-09-13T17:58:50.889 INFO:teuthology.orchestra.run.smithi111.stdout:{"success":"mds_log_pause = '' "} 2024-09-13T17:58:50.890 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_log_pause = '' " } 2024-09-13T17:58:50.890 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:58:50.890 DEBUG:teuthology.orchestra.run.smithi111:> import os 2024-09-13T17:58:50.890 DEBUG:teuthology.orchestra.run.smithi111:> os.chdir("/home/ubuntu/cephtest/mnt.0/testdir") 2024-09-13T17:58:50.890 DEBUG:teuthology.orchestra.run.smithi111:> os.setgid(65534) 2024-09-13T17:58:50.890 DEBUG:teuthology.orchestra.run.smithi111:> os.setuid(65534) 2024-09-13T17:58:50.890 DEBUG:teuthology.orchestra.run.smithi111:> fd = os.open("testfile", os.O_CREAT | os.O_RDWR, 0o644) 2024-09-13T17:58:50.890 DEBUG:teuthology.orchestra.run.smithi111:> os.fchmod(fd, 0o640) 2024-09-13T17:58:50.890 DEBUG:teuthology.orchestra.run.smithi111:> ' 2024-09-13T17:58:51.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:58:51.151 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod 600 /home/ubuntu/cephtest/mnt.0/testdir/testfile) 2024-09-13T17:58:56.585 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:58:56.588 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:59:01.194 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 2:0 2024-09-13T17:59:01.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.305+0000 7fa59c23a640 1 Processor -- start 2024-09-13T17:59:01.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.305+0000 7fa59c23a640 1 -- start start 2024-09-13T17:59:01.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.306+0000 7fa59c23a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa594159310 0x7fa5940ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:01.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.306+0000 7fa59c23a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa5940b2b40 0x7fa5940ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:01.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.306+0000 7fa59c23a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5940ccbe0 con 0x7fa5940ab680 2024-09-13T17:59:01.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.306+0000 7fa59c23a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5940ad940 con 0x7fa594159310 2024-09-13T17:59:01.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.306+0000 7fa59c23a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5940b1810 con 0x7fa5940b2b40 2024-09-13T17:59:01.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.306+0000 7fa599fd6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa594159310 0x7fa5940ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:01.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.306+0000 7fa599fd6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa594159310 0x7fa5940ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50704/0 (socket says 172.21.15.111:50704) 2024-09-13T17:59:01.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.306+0000 7fa599fd6640 1 -- 172.21.15.111:0/3356167108 learned_addr learned my addr 172.21.15.111:0/3356167108 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:01.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.306+0000 7fa5997d5640 1 --2- 172.21.15.111:0/3356167108 >> v2:172.21.15.188:3300/0 conn(0x7fa5940b2b40 0x7fa5940ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:01.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.307+0000 7fa599fd6640 1 -- 172.21.15.111:0/3356167108 >> v2:172.21.15.188:3300/0 conn(0x7fa5940b2b40 msgr2=0x7fa5940ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:01.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.307+0000 7fa599fd6640 1 --2- 172.21.15.111:0/3356167108 >> v2:172.21.15.188:3300/0 conn(0x7fa5940b2b40 0x7fa5940ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.307+0000 7fa599fd6640 1 -- 172.21.15.111:0/3356167108 >> v1:172.21.15.188:6789/0 conn(0x7fa5940ab680 legacy=0x7fa5940ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:01.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.307+0000 7fa599fd6640 1 -- 172.21.15.111:0/3356167108 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59406d9e0 con 0x7fa594159310 2024-09-13T17:59:01.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.307+0000 7fa599fd6640 1 --2- 172.21.15.111:0/3356167108 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa594159310 0x7fa5940ad400 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa5940b16a0 tx=0x7fa59002c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799ab8b3533f035e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:01.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.307+0000 7fa598fd4640 1 -- 172.21.15.111:0/3356167108 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59000b2d0 con 0x7fa594159310 2024-09-13T17:59:01.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.307+0000 7fa598fd4640 1 -- 172.21.15.111:0/3356167108 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59002f070 con 0x7fa594159310 2024-09-13T17:59:01.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.307+0000 7fa598fd4640 1 -- 172.21.15.111:0/3356167108 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590009820 con 0x7fa594159310 2024-09-13T17:59:01.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.308+0000 7fa59c23a640 1 -- 172.21.15.111:0/3356167108 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa594159310 msgr2=0x7fa5940ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:01.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.308+0000 7fa59c23a640 1 --2- 172.21.15.111:0/3356167108 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa594159310 0x7fa5940ad400 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa5940b16a0 tx=0x7fa59002c8b0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.308+0000 7fa59c23a640 1 -- 172.21.15.111:0/3356167108 shutdown_connections 2024-09-13T17:59:01.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.308+0000 7fa59c23a640 1 --2- 172.21.15.111:0/3356167108 >> v2:172.21.15.188:3300/0 conn(0x7fa5940b2b40 0x7fa5940ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.308+0000 7fa59c23a640 1 --2- 172.21.15.111:0/3356167108 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa594159310 0x7fa5940ad400 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.308+0000 7fa59c23a640 1 -- 172.21.15.111:0/3356167108 >> 172.21.15.111:0/3356167108 conn(0x7fa59406a0b0 msgr2=0x7fa59406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:01.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.308+0000 7fa59c23a640 1 -- 172.21.15.111:0/3356167108 shutdown_connections 2024-09-13T17:59:01.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.308+0000 7fa59c23a640 1 -- 172.21.15.111:0/3356167108 wait complete. 2024-09-13T17:59:01.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.309+0000 7fa59c23a640 1 Processor -- start 2024-09-13T17:59:01.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.309+0000 7fa59c23a640 1 -- start start 2024-09-13T17:59:01.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.310+0000 7fa59c23a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5940ab680 0x7fa5940c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:01.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.310+0000 7fa59c23a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5940b2b40 0x7fa59414fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:01.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.310+0000 7fa59c23a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa594159310 0x7fa5940710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:01.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.310+0000 7fa59c23a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59415b320 con 0x7fa594159310 2024-09-13T17:59:01.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.310+0000 7fa59c23a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5941503e0 con 0x7fa5940ab680 2024-09-13T17:59:01.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.310+0000 7fa59c23a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59415b040 con 0x7fa5940b2b40 2024-09-13T17:59:01.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.310+0000 7fa59a7d7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa594159310 0x7fa5940710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:01.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.310+0000 7fa59a7d7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa594159310 0x7fa5940710e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50714/0 (socket says 172.21.15.111:50714) 2024-09-13T17:59:01.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.310+0000 7fa59a7d7640 1 -- 172.21.15.111:0/3632416247 learned_addr learned my addr 172.21.15.111:0/3632416247 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:01.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.310+0000 7fa599fd6640 1 --2- 172.21.15.111:0/3632416247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5940ab680 0x7fa5940c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:01.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.310+0000 7fa5997d5640 1 --2- 172.21.15.111:0/3632416247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5940b2b40 0x7fa59414fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:01.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.310+0000 7fa59a7d7640 1 -- 172.21.15.111:0/3632416247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5940b2b40 msgr2=0x7fa59414fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:01.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.310+0000 7fa59a7d7640 1 --2- 172.21.15.111:0/3632416247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5940b2b40 0x7fa59414fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.310+0000 7fa59a7d7640 1 -- 172.21.15.111:0/3632416247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5940ab680 msgr2=0x7fa5940c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:01.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.310+0000 7fa59a7d7640 1 --2- 172.21.15.111:0/3632416247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5940ab680 0x7fa5940c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.310+0000 7fa59a7d7640 1 -- 172.21.15.111:0/3632416247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59000bd40 con 0x7fa594159310 2024-09-13T17:59:01.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.310+0000 7fa5997d5640 1 --2- 172.21.15.111:0/3632416247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5940b2b40 0x7fa59414fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:01.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.311+0000 7fa59a7d7640 1 --2- 172.21.15.111:0/3632416247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa594159310 0x7fa5940710e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00c3a0 tx=0x7fa58c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:01.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.311+0000 7fa58a7fc640 1 -- 172.21.15.111:0/3632416247 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c0052d0 con 0x7fa594159310 2024-09-13T17:59:01.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.311+0000 7fa58a7fc640 1 -- 172.21.15.111:0/3632416247 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c005430 con 0x7fa594159310 2024-09-13T17:59:01.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.311+0000 7fa59c23a640 1 -- 172.21.15.111:0/3632416247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa594159890 con 0x7fa594159310 2024-09-13T17:59:01.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.311+0000 7fa58a7fc640 1 -- 172.21.15.111:0/3632416247 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c003830 con 0x7fa594159310 2024-09-13T17:59:01.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.311+0000 7fa59c23a640 1 -- 172.21.15.111:0/3632416247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa594159e50 con 0x7fa594159310 2024-09-13T17:59:01.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.312+0000 7fa59c23a640 1 -- 172.21.15.111:0/3632416247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5941596f0 con 0x7fa594159310 2024-09-13T17:59:01.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.312+0000 7fa58a7fc640 1 -- 172.21.15.111:0/3632416247 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa58c003990 con 0x7fa594159310 2024-09-13T17:59:01.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.313+0000 7fa58a7fc640 1 --2- 172.21.15.111:0/3632416247 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa57806ac30 0x7fa57806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:01.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.313+0000 7fa58a7fc640 1 -- 172.21.15.111:0/3632416247 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa58c0888e0 con 0x7fa594159310 2024-09-13T17:59:01.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.313+0000 7fa599fd6640 1 --2- 172.21.15.111:0/3632416247 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa57806ac30 0x7fa57806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:01.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.313+0000 7fa599fd6640 1 --2- 172.21.15.111:0/3632416247 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa57806ac30 0x7fa57806d0f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa590036000 tx=0x7fa59000b000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:01.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.317+0000 7fa58a7fc640 1 -- 172.21.15.111:0/3632416247 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fa5941596f0 con 0x7fa594159310 2024-09-13T17:59:01.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.541+0000 7fa59c23a640 1 -- 172.21.15.111:0/3632416247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "2:0"} v 0) v1 -- 0x7fa5941596f0 con 0x7fa594159310 2024-09-13T17:59:01.543 INFO:tasks.ceph.mon.a.smithi111.stderr:2024-09-13T17:59:01.541+0000 7f3bedaae640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T17:59:01.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.548+0000 7fa58a7fc640 1 -- 172.21.15.111:0/3632416247 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "2:0"}]=0 failed mds gid 5490 v16) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa5941596f0 con 0x7fa594159310 2024-09-13T17:59:01.550 INFO:teuthology.orchestra.run.smithi111.stderr:failed mds gid 5490 2024-09-13T17:59:01.552 INFO:tasks.ceph.mds.a.smithi111.stderr: -1731> 2024-09-13T17:58:47.871+0000 7f33429de640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:59:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.551+0000 7fa59c23a640 1 -- 172.21.15.111:0/3632416247 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa57806ac30 msgr2=0x7fa57806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.551+0000 7fa59c23a640 1 --2- 172.21.15.111:0/3632416247 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa57806ac30 0x7fa57806d0f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa590036000 tx=0x7fa59000b000 comp rx=0 tx=0).stop 2024-09-13T17:59:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.551+0000 7fa59c23a640 1 -- 172.21.15.111:0/3632416247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa594159310 msgr2=0x7fa5940710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.551+0000 7fa59c23a640 1 --2- 172.21.15.111:0/3632416247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa594159310 0x7fa5940710e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00c3a0 tx=0x7fa58c00c860 comp rx=0 tx=0).stop 2024-09-13T17:59:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.551+0000 7fa59c23a640 1 -- 172.21.15.111:0/3632416247 shutdown_connections 2024-09-13T17:59:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.551+0000 7fa59c23a640 1 --2- 172.21.15.111:0/3632416247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa594159310 0x7fa5940710e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.551+0000 7fa59c23a640 1 --2- 172.21.15.111:0/3632416247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5940b2b40 0x7fa59414fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.551+0000 7fa59c23a640 1 --2- 172.21.15.111:0/3632416247 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa57806ac30 0x7fa57806d0f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.551+0000 7fa59c23a640 1 --2- 172.21.15.111:0/3632416247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5940ab680 0x7fa5940c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.551+0000 7fa59c23a640 1 -- 172.21.15.111:0/3632416247 >> 172.21.15.111:0/3632416247 conn(0x7fa59406a0b0 msgr2=0x7fa5940acf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:01.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.551+0000 7fa59c23a640 1 -- 172.21.15.111:0/3632416247 shutdown_connections 2024-09-13T17:59:01.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.551+0000 7fa59c23a640 1 -- 172.21.15.111:0/3632416247 wait complete. 2024-09-13T17:59:01.568 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:59:01.583 INFO:tasks.ceph.mds.a.smithi111.stdout:starting mds.a at 2024-09-13T17:59:01.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.665+0000 7fa2fa7bc640 1 Processor -- start 2024-09-13T17:59:01.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.666+0000 7fa2fa7bc640 1 -- start start 2024-09-13T17:59:01.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.666+0000 7fa2fa7bc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa2f406c4c0 0x7fa2f406c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:01.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.666+0000 7fa2fa7bc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2f40b2590 0x7fa2f406d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:01.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.666+0000 7fa2fa7bc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa2f40cbea0 0x7fa2f40cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:01.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.666+0000 7fa2fa7bc640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f406deb0 con 0x7fa2f406c4c0 2024-09-13T17:59:01.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.666+0000 7fa2fa7bc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f40cc890 con 0x7fa2f40cbea0 2024-09-13T17:59:01.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.666+0000 7fa2fa7bc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f406e0c0 con 0x7fa2f40b2590 2024-09-13T17:59:01.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.666+0000 7fa2f3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa2f406c4c0 0x7fa2f406c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:01.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.666+0000 7fa2f3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa2f406c4c0 0x7fa2f406c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50746/0 (socket says 172.21.15.111:50746) 2024-09-13T17:59:01.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.666+0000 7fa2f3fff640 1 -- 172.21.15.111:0/3849445648 learned_addr learned my addr 172.21.15.111:0/3849445648 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:01.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.667+0000 7fa2f37fe640 1 --2- 172.21.15.111:0/3849445648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2f40b2590 0x7fa2f406d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:01.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.667+0000 7fa2f8d59640 1 --2- 172.21.15.111:0/3849445648 >> v2:172.21.15.188:3300/0 conn(0x7fa2f40cbea0 0x7fa2f40cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:01.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.667+0000 7fa2f3fff640 1 -- 172.21.15.111:0/3849445648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2f40b2590 msgr2=0x7fa2f406d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:01.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.667+0000 7fa2f3fff640 1 --2- 172.21.15.111:0/3849445648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2f40b2590 0x7fa2f406d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.667+0000 7fa2f3fff640 1 -- 172.21.15.111:0/3849445648 >> v2:172.21.15.188:3300/0 conn(0x7fa2f40cbea0 msgr2=0x7fa2f40cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:01.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.667+0000 7fa2f3fff640 1 --2- 172.21.15.111:0/3849445648 >> v2:172.21.15.188:3300/0 conn(0x7fa2f40cbea0 0x7fa2f40cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.667+0000 7fa2f3fff640 1 -- 172.21.15.111:0/3849445648 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f406e3a0 con 0x7fa2f406c4c0 2024-09-13T17:59:01.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.667+0000 7fa2f3fff640 1 --2- 172.21.15.111:0/3849445648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2f406c4c0 0x7fa2f406c8a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009870 tx=0x7fa2e402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b193f9768d708318 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:01.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.668+0000 7fa2f2ffd640 1 -- 172.21.15.111:0/3849445648 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4032440 con 0x7fa2f406c4c0 2024-09-13T17:59:01.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.668+0000 7fa2f2ffd640 1 -- 172.21.15.111:0/3849445648 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e40325a0 con 0x7fa2f406c4c0 2024-09-13T17:59:01.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.668+0000 7fa2f2ffd640 1 -- 172.21.15.111:0/3849445648 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4032930 con 0x7fa2f406c4c0 2024-09-13T17:59:01.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.668+0000 7fa2fa7bc640 1 -- 172.21.15.111:0/3849445648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2f406c4c0 msgr2=0x7fa2f406c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:01.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.668+0000 7fa2fa7bc640 1 --2- 172.21.15.111:0/3849445648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2f406c4c0 0x7fa2f406c8a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009870 tx=0x7fa2e402a210 comp rx=0 tx=0).stop 2024-09-13T17:59:01.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.668+0000 7fa2fa7bc640 1 -- 172.21.15.111:0/3849445648 shutdown_connections 2024-09-13T17:59:01.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.668+0000 7fa2fa7bc640 1 --2- 172.21.15.111:0/3849445648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2f40b2590 0x7fa2f406d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.668+0000 7fa2fa7bc640 1 --2- 172.21.15.111:0/3849445648 >> v2:172.21.15.188:3300/0 conn(0x7fa2f40cbea0 0x7fa2f40cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.668+0000 7fa2fa7bc640 1 --2- 172.21.15.111:0/3849445648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2f406c4c0 0x7fa2f406c8a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.668+0000 7fa2fa7bc640 1 -- 172.21.15.111:0/3849445648 >> 172.21.15.111:0/3849445648 conn(0x7fa2f4069ed0 msgr2=0x7fa2f406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:01.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.668+0000 7fa2fa7bc640 1 -- 172.21.15.111:0/3849445648 shutdown_connections 2024-09-13T17:59:01.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.668+0000 7fa2fa7bc640 1 -- 172.21.15.111:0/3849445648 wait complete. 2024-09-13T17:59:01.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.669+0000 7fa2fa7bc640 1 Processor -- start 2024-09-13T17:59:01.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.670+0000 7fa2fa7bc640 1 -- start start 2024-09-13T17:59:01.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.670+0000 7fa2fa7bc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2f406c4c0 0x7fa2f415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:01.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.670+0000 7fa2fa7bc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2f40b2590 0x7fa2f415a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:01.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.670+0000 7fa2fa7bc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2f40cbea0 0x7fa2f40c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:01.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.670+0000 7fa2fa7bc640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f40c5ec0 con 0x7fa2f40b2590 2024-09-13T17:59:01.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.670+0000 7fa2fa7bc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f40c4cc0 con 0x7fa2f40cbea0 2024-09-13T17:59:01.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.670+0000 7fa2fa7bc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f415bd00 con 0x7fa2f406c4c0 2024-09-13T17:59:01.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.670+0000 7fa2f37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2f40b2590 0x7fa2f415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:01.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.670+0000 7fa2f37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2f40b2590 0x7fa2f415a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50754/0 (socket says 172.21.15.111:50754) 2024-09-13T17:59:01.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.670+0000 7fa2f37fe640 1 -- 172.21.15.111:0/85090417 learned_addr learned my addr 172.21.15.111:0/85090417 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:01.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.671+0000 7fa2f3fff640 1 --2- 172.21.15.111:0/85090417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2f406c4c0 0x7fa2f415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:01.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.671+0000 7fa2f8d59640 1 --2- 172.21.15.111:0/85090417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2f40cbea0 0x7fa2f40c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:01.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.671+0000 7fa2f37fe640 1 -- 172.21.15.111:0/85090417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2f406c4c0 msgr2=0x7fa2f415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:01.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.671+0000 7fa2f37fe640 1 --2- 172.21.15.111:0/85090417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2f406c4c0 0x7fa2f415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.671+0000 7fa2f37fe640 1 -- 172.21.15.111:0/85090417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2f40cbea0 msgr2=0x7fa2f40c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:01.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.671+0000 7fa2f37fe640 1 --2- 172.21.15.111:0/85090417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2f40cbea0 0x7fa2f40c4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.671+0000 7fa2f37fe640 1 -- 172.21.15.111:0/85090417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e40094d0 con 0x7fa2f40b2590 2024-09-13T17:59:01.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.671+0000 7fa2f37fe640 1 --2- 172.21.15.111:0/85090417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2f40b2590 0x7fa2f415a700 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800d6a0 tx=0x7fa2e800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:01.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.671+0000 7fa2f0ff9640 1 -- 172.21.15.111:0/85090417 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8004220 con 0x7fa2f40b2590 2024-09-13T17:59:01.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.671+0000 7fa2f0ff9640 1 -- 172.21.15.111:0/85090417 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e8004380 con 0x7fa2f40b2590 2024-09-13T17:59:01.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.671+0000 7fa2fa7bc640 1 -- 172.21.15.111:0/85090417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f40c5800 con 0x7fa2f40b2590 2024-09-13T17:59:01.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.671+0000 7fa2f0ff9640 1 -- 172.21.15.111:0/85090417 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8007920 con 0x7fa2f40b2590 2024-09-13T17:59:01.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.671+0000 7fa2fa7bc640 1 -- 172.21.15.111:0/85090417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f41590e0 con 0x7fa2f40b2590 2024-09-13T17:59:01.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.672+0000 7fa2fa7bc640 1 -- 172.21.15.111:0/85090417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f40c5660 con 0x7fa2f40b2590 2024-09-13T17:59:01.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.672+0000 7fa2f0ff9640 1 -- 172.21.15.111:0/85090417 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa2e8007a80 con 0x7fa2f40b2590 2024-09-13T17:59:01.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.673+0000 7fa2f0ff9640 1 --2- 172.21.15.111:0/85090417 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa2e006ac00 0x7fa2e006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:01.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.673+0000 7fa2f0ff9640 1 -- 172.21.15.111:0/85090417 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fa2e8089310 con 0x7fa2f40b2590 2024-09-13T17:59:01.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.673+0000 7fa2f3fff640 1 --2- 172.21.15.111:0/85090417 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa2e006ac00 0x7fa2e006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:01.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.673+0000 7fa2f3fff640 1 --2- 172.21.15.111:0/85090417 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa2e006ac00 0x7fa2e006d0c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4034000 tx=0x7fa2e402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:01.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.677+0000 7fa2f0ff9640 1 -- 172.21.15.111:0/85090417 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fa2f40c5660 con 0x7fa2f40b2590 2024-09-13T17:59:01.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.872+0000 7fa2fa7bc640 1 -- 172.21.15.111:0/85090417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2f40c5660 con 0x7fa2f40b2590 2024-09-13T17:59:01.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.872+0000 7fa2f0ff9640 1 -- 172.21.15.111:0/85090417 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+3984 (secure 0 0 0) 0x7fa2f40c5660 con 0x7fa2f40b2590 2024-09-13T17:59:01.875 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:01.875 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4217138481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4217138481},{"type":"v1","addr":"172.21.15.188:6834","nonce":4217138481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3099948287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3099948287},{"type":"v1","addr":"172.21.15.188:6835","nonce":3099948287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:58:43.534567+0000","modified":"2024-09-13T17:59:01.549576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5487":{"gid":5487,"name":"c","rank":0,"incarnation":17,"state":"up:replay","state_seq":1,"addr":"172.21.15.111:6835/214034224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":214034224},{"type":"v1","addr":"172.21.15.111:6835","nonce":214034224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:59:01.875 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 17 2024-09-13T17:59:01.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.874+0000 7fa2fa7bc640 1 -- 172.21.15.111:0/85090417 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa2e006ac00 msgr2=0x7fa2e006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:01.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.874+0000 7fa2fa7bc640 1 --2- 172.21.15.111:0/85090417 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa2e006ac00 0x7fa2e006d0c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4034000 tx=0x7fa2e402e000 comp rx=0 tx=0).stop 2024-09-13T17:59:01.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.874+0000 7fa2fa7bc640 1 -- 172.21.15.111:0/85090417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2f40b2590 msgr2=0x7fa2f415a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:01.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.874+0000 7fa2fa7bc640 1 --2- 172.21.15.111:0/85090417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2f40b2590 0x7fa2f415a700 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800d6a0 tx=0x7fa2e800db60 comp rx=0 tx=0).stop 2024-09-13T17:59:01.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.874+0000 7fa2fa7bc640 1 -- 172.21.15.111:0/85090417 shutdown_connections 2024-09-13T17:59:01.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.874+0000 7fa2fa7bc640 1 --2- 172.21.15.111:0/85090417 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa2e006ac00 0x7fa2e006d0c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.874+0000 7fa2fa7bc640 1 --2- 172.21.15.111:0/85090417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2f40cbea0 0x7fa2f40c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.874+0000 7fa2fa7bc640 1 --2- 172.21.15.111:0/85090417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2f40b2590 0x7fa2f415a700 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.874+0000 7fa2fa7bc640 1 --2- 172.21.15.111:0/85090417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2f406c4c0 0x7fa2f415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:01.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.874+0000 7fa2fa7bc640 1 -- 172.21.15.111:0/85090417 >> 172.21.15.111:0/85090417 conn(0x7fa2f4069ed0 msgr2=0x7fa2f406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:01.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.875+0000 7fa2fa7bc640 1 -- 172.21.15.111:0/85090417 shutdown_connections 2024-09-13T17:59:01.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:01.875+0000 7fa2fa7bc640 1 -- 172.21.15.111:0/85090417 wait complete. 2024-09-13T17:59:02.291 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:58:43.534567+0000', 'modified': '2024-09-13T17:59:01.549576+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 40, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5487': {'gid': 5487, 'name': 'c', 'rank': 0, 'incarnation': 17, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.111:6835/214034224', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 214034224}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 214034224}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T17:59:02.291 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5487:up:replay 2024-09-13T17:59:03.292 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:59:03.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.400+0000 7fcfd523a640 1 Processor -- start 2024-09-13T17:59:03.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.400+0000 7fcfd523a640 1 -- start start 2024-09-13T17:59:03.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.401+0000 7fcfd523a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcfd00cace0 0x7fcfd00cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:03.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.401+0000 7fcfd523a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfd006d030 0x7fcfd006d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:03.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.401+0000 7fcfd523a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd00c8920 con 0x7fcfd015a2e0 2024-09-13T17:59:03.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.401+0000 7fcfd523a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd00cb690 con 0x7fcfd00cace0 2024-09-13T17:59:03.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.401+0000 7fcfd523a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd015a6c0 con 0x7fcfd006d030 2024-09-13T17:59:03.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.401+0000 7fcfced9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fcfd015a2e0 0x7fcfd00c9a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33502/0 (socket says 172.21.15.111:33502) 2024-09-13T17:59:03.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.401+0000 7fcfced9d640 1 -- 172.21.15.111:0/2797349715 learned_addr learned my addr 172.21.15.111:0/2797349715 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:03.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.401+0000 7fcfcf59e640 1 --2- 172.21.15.111:0/2797349715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfd006d030 0x7fcfd006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:03.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.401+0000 7fcfce59c640 1 --2- 172.21.15.111:0/2797349715 >> v2:172.21.15.188:3300/0 conn(0x7fcfd00cace0 0x7fcfd00cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:03.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.401+0000 7fcfcdd9b640 1 -- 172.21.15.111:0/2797349715 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2631117653 0 0) 0x7fcfd00c8920 con 0x7fcfd015a2e0 2024-09-13T17:59:03.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.401+0000 7fcfcdd9b640 1 -- 172.21.15.111:0/2797349715 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfb8003520 con 0x7fcfd015a2e0 2024-09-13T17:59:03.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.402+0000 7fcfcdd9b640 1 -- 172.21.15.111:0/2797349715 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2990750605 0 0) 0x7fcfb8003520 con 0x7fcfd015a2e0 2024-09-13T17:59:03.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.402+0000 7fcfcf59e640 1 -- 172.21.15.111:0/2797349715 >> v2:172.21.15.188:3300/0 conn(0x7fcfd00cace0 msgr2=0x7fcfd00cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:03.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.402+0000 7fcfcf59e640 1 --2- 172.21.15.111:0/2797349715 >> v2:172.21.15.188:3300/0 conn(0x7fcfd00cace0 0x7fcfd00cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:03.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.402+0000 7fcfcf59e640 1 -- 172.21.15.111:0/2797349715 >> v1:172.21.15.111:6789/0 conn(0x7fcfd015a2e0 legacy=0x7fcfd00c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:03.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.402+0000 7fcfcf59e640 1 -- 172.21.15.111:0/2797349715 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd00ca500 con 0x7fcfd006d030 2024-09-13T17:59:03.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.402+0000 7fcfcf59e640 1 --2- 172.21.15.111:0/2797349715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfd006d030 0x7fcfd006d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80097a0 tx=0x7fcfc802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=719dd0e519854dfb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:03.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.403+0000 7fcfcdd9b640 1 -- 172.21.15.111:0/2797349715 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc00019f0 con 0x7fcfd006d030 2024-09-13T17:59:03.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.403+0000 7fcfcdd9b640 1 -- 172.21.15.111:0/2797349715 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc802aba0 con 0x7fcfd006d030 2024-09-13T17:59:03.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.403+0000 7fcfcdd9b640 1 -- 172.21.15.111:0/2797349715 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8032440 con 0x7fcfd006d030 2024-09-13T17:59:03.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.403+0000 7fcfd523a640 1 -- 172.21.15.111:0/2797349715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfd006d030 msgr2=0x7fcfd006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:03.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.403+0000 7fcfd523a640 1 --2- 172.21.15.111:0/2797349715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfd006d030 0x7fcfd006d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80097a0 tx=0x7fcfc802a110 comp rx=0 tx=0).stop 2024-09-13T17:59:03.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.403+0000 7fcfd523a640 1 -- 172.21.15.111:0/2797349715 shutdown_connections 2024-09-13T17:59:03.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.403+0000 7fcfd523a640 1 --2- 172.21.15.111:0/2797349715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfd006d030 0x7fcfd006d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:03.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.403+0000 7fcfd523a640 1 --2- 172.21.15.111:0/2797349715 >> v2:172.21.15.188:3300/0 conn(0x7fcfd00cace0 0x7fcfd00cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:03.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.403+0000 7fcfd523a640 1 -- 172.21.15.111:0/2797349715 >> 172.21.15.111:0/2797349715 conn(0x7fcfd0069ed0 msgr2=0x7fcfd006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:03.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.403+0000 7fcfd523a640 1 -- 172.21.15.111:0/2797349715 shutdown_connections 2024-09-13T17:59:03.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.403+0000 7fcfd523a640 1 -- 172.21.15.111:0/2797349715 wait complete. 2024-09-13T17:59:03.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.405+0000 7fcfd523a640 1 Processor -- start 2024-09-13T17:59:03.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.405+0000 7fcfd523a640 1 -- start start 2024-09-13T17:59:03.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfd523a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfd006d030 0x7fcfd00c23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:03.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfd523a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfd00cace0 0x7fcfd0158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:03.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfd523a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfd015a2e0 0x7fcfd00ab140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:03.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfd523a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd00c2920 con 0x7fcfd00cace0 2024-09-13T17:59:03.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfd523a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd0159680 con 0x7fcfd006d030 2024-09-13T17:59:03.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfd523a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd00c5f80 con 0x7fcfd015a2e0 2024-09-13T17:59:03.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfce59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfd00cace0 0x7fcfd0158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:03.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfce59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfd00cace0 0x7fcfd0158b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52318/0 (socket says 172.21.15.111:52318) 2024-09-13T17:59:03.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfce59c640 1 -- 172.21.15.111:0/2397203489 learned_addr learned my addr 172.21.15.111:0/2397203489 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:03.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfced9d640 1 --2- 172.21.15.111:0/2397203489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfd006d030 0x7fcfd00c23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:03.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfcf59e640 1 --2- 172.21.15.111:0/2397203489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfd015a2e0 0x7fcfd00ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:03.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfce59c640 1 -- 172.21.15.111:0/2397203489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfd015a2e0 msgr2=0x7fcfd00ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:03.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfce59c640 1 --2- 172.21.15.111:0/2397203489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfd015a2e0 0x7fcfd00ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:03.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfce59c640 1 -- 172.21.15.111:0/2397203489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfd006d030 msgr2=0x7fcfd00c23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:03.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfce59c640 1 --2- 172.21.15.111:0/2397203489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfd006d030 0x7fcfd00c23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:03.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfce59c640 1 -- 172.21.15.111:0/2397203489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc8009400 con 0x7fcfd00cace0 2024-09-13T17:59:03.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfcf59e640 1 --2- 172.21.15.111:0/2397203489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfd015a2e0 0x7fcfd00ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:03.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfce59c640 1 --2- 172.21.15.111:0/2397203489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfd00cace0 0x7fcfd0158b00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400e9b0 tx=0x7fcfc400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:03.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfb77fe640 1 -- 172.21.15.111:0/2397203489 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4009670 con 0x7fcfd00cace0 2024-09-13T17:59:03.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfb77fe640 1 -- 172.21.15.111:0/2397203489 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc400c9a0 con 0x7fcfd00cace0 2024-09-13T17:59:03.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfd523a640 1 -- 172.21.15.111:0/2397203489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd00c6260 con 0x7fcfd00cace0 2024-09-13T17:59:03.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.406+0000 7fcfb77fe640 1 -- 172.21.15.111:0/2397203489 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc400cd50 con 0x7fcfd00cace0 2024-09-13T17:59:03.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.407+0000 7fcfd523a640 1 -- 172.21.15.111:0/2397203489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd00c6820 con 0x7fcfd00cace0 2024-09-13T17:59:03.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.408+0000 7fcfd523a640 1 -- 172.21.15.111:0/2397203489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd00c9630 con 0x7fcfd00cace0 2024-09-13T17:59:03.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.408+0000 7fcfb77fe640 1 -- 172.21.15.111:0/2397203489 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcfc4013470 con 0x7fcfd00cace0 2024-09-13T17:59:03.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.408+0000 7fcfb77fe640 1 --2- 172.21.15.111:0/2397203489 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcfbc06ac00 0x7fcfbc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:03.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.409+0000 7fcfb77fe640 1 -- 172.21.15.111:0/2397203489 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fcfc4089730 con 0x7fcfd00cace0 2024-09-13T17:59:03.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.409+0000 7fcfced9d640 1 --2- 172.21.15.111:0/2397203489 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcfbc06ac00 0x7fcfbc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:03.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.409+0000 7fcfced9d640 1 --2- 172.21.15.111:0/2397203489 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcfbc06ac00 0x7fcfbc06d0c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0002230 tx=0x7fcfc0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:03.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.413+0000 7fcfb77fe640 1 -- 172.21.15.111:0/2397203489 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fcfc4054830 con 0x7fcfd00cace0 2024-09-13T17:59:03.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.610+0000 7fcfd523a640 1 -- 172.21.15.111:0/2397203489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfd00724c0 con 0x7fcfd00cace0 2024-09-13T17:59:03.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.610+0000 7fcfb77fe640 1 -- 172.21.15.111:0/2397203489 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4741 (secure 0 0 0) 0x7fcfc4057e50 con 0x7fcfd00cace0 2024-09-13T17:59:03.612 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:03.612 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4217138481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4217138481},{"type":"v1","addr":"172.21.15.188:6834","nonce":4217138481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3099948287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3099948287},{"type":"v1","addr":"172.21.15.188:6835","nonce":3099948287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/904506495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":904506495},{"type":"v1","addr":"172.21.15.111:6837","nonce":904506495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:58:43.534567+0000","modified":"2024-09-13T17:59:03.551959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5487":{"gid":5487,"name":"c","rank":0,"incarnation":17,"state":"up:rejoin","state_seq":9,"addr":"172.21.15.111:6835/214034224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":214034224},{"type":"v1","addr":"172.21.15.111:6835","nonce":214034224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:59:03.612 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 19 2024-09-13T17:59:03.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.612+0000 7fcfd523a640 1 -- 172.21.15.111:0/2397203489 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcfbc06ac00 msgr2=0x7fcfbc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:03.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.612+0000 7fcfd523a640 1 --2- 172.21.15.111:0/2397203489 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcfbc06ac00 0x7fcfbc06d0c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0002230 tx=0x7fcfc0009040 comp rx=0 tx=0).stop 2024-09-13T17:59:03.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.612+0000 7fcfd523a640 1 -- 172.21.15.111:0/2397203489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfd00cace0 msgr2=0x7fcfd0158b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:03.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.612+0000 7fcfd523a640 1 --2- 172.21.15.111:0/2397203489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfd00cace0 0x7fcfd0158b00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400e9b0 tx=0x7fcfc400ee70 comp rx=0 tx=0).stop 2024-09-13T17:59:03.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.612+0000 7fcfd523a640 1 -- 172.21.15.111:0/2397203489 shutdown_connections 2024-09-13T17:59:03.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.612+0000 7fcfd523a640 1 --2- 172.21.15.111:0/2397203489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfd015a2e0 0x7fcfd00ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:03.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.612+0000 7fcfd523a640 1 --2- 172.21.15.111:0/2397203489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfd00cace0 0x7fcfd0158b00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:03.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.612+0000 7fcfd523a640 1 --2- 172.21.15.111:0/2397203489 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fcfbc06ac00 0x7fcfbc06d0c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:03.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.612+0000 7fcfd523a640 1 --2- 172.21.15.111:0/2397203489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfd006d030 0x7fcfd00c23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:03.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.612+0000 7fcfd523a640 1 -- 172.21.15.111:0/2397203489 >> 172.21.15.111:0/2397203489 conn(0x7fcfd0069ed0 msgr2=0x7fcfd0070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:03.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.612+0000 7fcfd523a640 1 -- 172.21.15.111:0/2397203489 shutdown_connections 2024-09-13T17:59:03.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:03.612+0000 7fcfd523a640 1 -- 172.21.15.111:0/2397203489 wait complete. 2024-09-13T17:59:04.028 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:58:43.534567+0000', 'modified': '2024-09-13T17:59:03.551959+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 40, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5487': {'gid': 5487, 'name': 'c', 'rank': 0, 'incarnation': 17, 'state': 'up:rejoin', 'state_seq': 9, 'addr': '172.21.15.111:6835/214034224', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 214034224}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 214034224}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T17:59:04.028 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5487:up:rejoin 2024-09-13T17:59:05.029 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:59:05.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.138+0000 7f4b0f1b5640 1 Processor -- start 2024-09-13T17:59:05.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.138+0000 7f4b0f1b5640 1 -- start start 2024-09-13T17:59:05.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.138+0000 7f4b0f1b5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b080b4130 0x7f4b080d2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:05.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.138+0000 7f4b0f1b5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4b0808a370 0x7f4b080d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:05.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.138+0000 7f4b0f1b5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b080b3540 con 0x7f4b080b4510 2024-09-13T17:59:05.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.138+0000 7f4b0f1b5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b080b2ff0 con 0x7f4b0808a370 2024-09-13T17:59:05.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.138+0000 7f4b0f1b5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080cdc70 con 0x7f4b080b4130 2024-09-13T17:59:05.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b0d752640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4b080b4510 0x7f4b080d1450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33506/0 (socket says 172.21.15.111:33506) 2024-09-13T17:59:05.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b0d752640 1 -- 172.21.15.111:0/2856440277 learned_addr learned my addr 172.21.15.111:0/2856440277 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:05.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b0cf51640 1 --2- 172.21.15.111:0/2856440277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b080b4130 0x7f4b080d2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:05.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b077fe640 1 -- 172.21.15.111:0/2856440277 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476934329 0 0) 0x7f4b080b3540 con 0x7f4b080b4510 2024-09-13T17:59:05.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b07fff640 1 --2- 172.21.15.111:0/2856440277 >> v2:172.21.15.188:3300/0 conn(0x7f4b0808a370 0x7f4b080d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:05.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b077fe640 1 -- 172.21.15.111:0/2856440277 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4af8003520 con 0x7f4b080b4510 2024-09-13T17:59:05.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b077fe640 1 -- 172.21.15.111:0/2856440277 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1977429462 0 0) 0x7f4af8003520 con 0x7f4b080b4510 2024-09-13T17:59:05.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b077fe640 1 -- 172.21.15.111:0/2856440277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b080b4130 msgr2=0x7f4b080d2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:05.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b077fe640 1 --2- 172.21.15.111:0/2856440277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b080b4130 0x7f4b080d2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:05.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b077fe640 1 -- 172.21.15.111:0/2856440277 >> v2:172.21.15.188:3300/0 conn(0x7f4b0808a370 msgr2=0x7f4b080d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:05.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b077fe640 1 --2- 172.21.15.111:0/2856440277 >> v2:172.21.15.188:3300/0 conn(0x7f4b0808a370 0x7f4b080d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:05.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b077fe640 1 -- 172.21.15.111:0/2856440277 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b080cd6f0 con 0x7f4b080b4510 2024-09-13T17:59:05.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b077fe640 1 -- 172.21.15.111:0/2856440277 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f4b000019f0 con 0x7f4b080b4510 2024-09-13T17:59:05.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b077fe640 1 -- 172.21.15.111:0/2856440277 >> v1:172.21.15.111:6789/0 conn(0x7f4b080b4510 legacy=0x7f4b080d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:05.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b077fe640 1 --2- 172.21.15.111:0/2856440277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af8003970 0x7f4af8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:05.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b077fe640 1 --2- 172.21.15.111:0/2856440277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b080b4130 0x7f4af80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:05.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b077fe640 1 --2- 172.21.15.111:0/2856440277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af8004d00 0x7f4af8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:05.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b077fe640 1 -- 172.21.15.111:0/2856440277 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080b2ff0 con 0x7f4af8003970 2024-09-13T17:59:05.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b077fe640 1 -- 172.21.15.111:0/2856440277 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080cdc70 con 0x7f4af8004d00 2024-09-13T17:59:05.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b077fe640 1 -- 172.21.15.111:0/2856440277 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af800b980 con 0x7f4b080b4130 2024-09-13T17:59:05.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b0cf51640 1 --2- 172.21.15.111:0/2856440277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af8003970 0x7f4af8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:05.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.139+0000 7f4b07fff640 1 --2- 172.21.15.111:0/2856440277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b080b4130 0x7f4af80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:05.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.140+0000 7f4b0d752640 1 --2- 172.21.15.111:0/2856440277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af8004d00 0x7f4af8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:05.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.140+0000 7f4b0cf51640 1 -- 172.21.15.111:0/2856440277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b080b4130 msgr2=0x7f4af80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:05.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.140+0000 7f4b0cf51640 1 --2- 172.21.15.111:0/2856440277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b080b4130 0x7f4af80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:05.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.140+0000 7f4b0cf51640 1 -- 172.21.15.111:0/2856440277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af8004d00 msgr2=0x7f4af8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:05.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.140+0000 7f4b0cf51640 1 --2- 172.21.15.111:0/2856440277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af8004d00 0x7f4af8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:05.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.140+0000 7f4b0cf51640 1 -- 172.21.15.111:0/2856440277 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af8003520 con 0x7f4af8003970 2024-09-13T17:59:05.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.140+0000 7f4b0cf51640 1 --2- 172.21.15.111:0/2856440277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af8003970 0x7f4af8003d90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009ac0 tx=0x7f4af80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15d08e100730adf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:05.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.140+0000 7f4b077fe640 1 -- 172.21.15.111:0/2856440277 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc031400 con 0x7f4af8003970 2024-09-13T17:59:05.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.141+0000 7f4b077fe640 1 -- 172.21.15.111:0/2856440277 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4afc02d050 con 0x7f4af8003970 2024-09-13T17:59:05.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.141+0000 7f4b077fe640 1 -- 172.21.15.111:0/2856440277 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc031780 con 0x7f4af8003970 2024-09-13T17:59:05.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.141+0000 7f4b0f1b5640 1 -- 172.21.15.111:0/2856440277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af8003970 msgr2=0x7f4af8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:05.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.141+0000 7f4b0f1b5640 1 --2- 172.21.15.111:0/2856440277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af8003970 0x7f4af8003d90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009ac0 tx=0x7f4af80057f0 comp rx=0 tx=0).stop 2024-09-13T17:59:05.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.141+0000 7f4b0d752640 1 -- 172.21.15.111:0/2856440277 reap_dead start 2024-09-13T17:59:05.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.141+0000 7f4b0f1b5640 1 -- 172.21.15.111:0/2856440277 shutdown_connections 2024-09-13T17:59:05.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.141+0000 7f4b0f1b5640 1 -- 172.21.15.111:0/2856440277 >> 172.21.15.111:0/2856440277 conn(0x7f4b08089f60 msgr2=0x7f4b080d24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:05.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.141+0000 7f4b0f1b5640 1 -- 172.21.15.111:0/2856440277 shutdown_connections 2024-09-13T17:59:05.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.141+0000 7f4b0f1b5640 1 -- 172.21.15.111:0/2856440277 wait complete. 2024-09-13T17:59:05.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.142+0000 7f4b0f1b5640 1 Processor -- start 2024-09-13T17:59:05.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.142+0000 7f4b0f1b5640 1 -- start start 2024-09-13T17:59:05.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.143+0000 7f4b0f1b5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af8003970 0x7f4b0814c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:05.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.143+0000 7f4b0f1b5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b080b0ee0 0x7f4b080b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:05.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.143+0000 7f4b0f1b5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b081f5420 0x7f4b081f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:05.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.143+0000 7f4b0f1b5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0815d4f0 con 0x7f4af8003970 2024-09-13T17:59:05.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.143+0000 7f4b0f1b5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080b1d10 con 0x7f4b080b0ee0 2024-09-13T17:59:05.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.143+0000 7f4b0f1b5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080b1ff0 con 0x7f4b081f5420 2024-09-13T17:59:05.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.143+0000 7f4b0cf51640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af8003970 0x7f4b0814c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:05.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.143+0000 7f4b0cf51640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af8003970 0x7f4b0814c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52338/0 (socket says 172.21.15.111:52338) 2024-09-13T17:59:05.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.143+0000 7f4b0cf51640 1 -- 172.21.15.111:0/547566351 learned_addr learned my addr 172.21.15.111:0/547566351 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:05.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.144+0000 7f4b07fff640 1 --2- 172.21.15.111:0/547566351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b080b0ee0 0x7f4b080b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:05.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.144+0000 7f4b0d752640 1 --2- 172.21.15.111:0/547566351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b081f5420 0x7f4b081f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:05.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.144+0000 7f4b0cf51640 1 -- 172.21.15.111:0/547566351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b081f5420 msgr2=0x7f4b081f58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:05.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.144+0000 7f4b0cf51640 1 --2- 172.21.15.111:0/547566351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b081f5420 0x7f4b081f58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:05.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.144+0000 7f4b0cf51640 1 -- 172.21.15.111:0/547566351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b080b0ee0 msgr2=0x7f4b080b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:05.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.144+0000 7f4b0cf51640 1 --2- 172.21.15.111:0/547566351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b080b0ee0 0x7f4b080b1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:05.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.144+0000 7f4b0cf51640 1 -- 172.21.15.111:0/547566351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b080b22d0 con 0x7f4af8003970 2024-09-13T17:59:05.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.144+0000 7f4b0cf51640 1 --2- 172.21.15.111:0/547566351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af8003970 0x7f4b0814c6f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009ac0 tx=0x7f4afc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:05.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.144+0000 7f4b057fa640 1 -- 172.21.15.111:0/547566351 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc003b90 con 0x7f4af8003970 2024-09-13T17:59:05.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.144+0000 7f4b057fa640 1 -- 172.21.15.111:0/547566351 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4afc003cf0 con 0x7f4af8003970 2024-09-13T17:59:05.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.144+0000 7f4b0f1b5640 1 -- 172.21.15.111:0/547566351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0814cc30 con 0x7f4af8003970 2024-09-13T17:59:05.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.144+0000 7f4b057fa640 1 -- 172.21.15.111:0/547566351 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc044650 con 0x7f4af8003970 2024-09-13T17:59:05.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.144+0000 7f4b0f1b5640 1 -- 172.21.15.111:0/547566351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b080b1a80 con 0x7f4af8003970 2024-09-13T17:59:05.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.145+0000 7f4b0f1b5640 1 -- 172.21.15.111:0/547566351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0815dab0 con 0x7f4af8003970 2024-09-13T17:59:05.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.145+0000 7f4b057fa640 1 -- 172.21.15.111:0/547566351 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4afc038070 con 0x7f4af8003970 2024-09-13T17:59:05.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.146+0000 7f4b057fa640 1 --2- 172.21.15.111:0/547566351 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4aec06ac00 0x7f4aec06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:05.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.146+0000 7f4b057fa640 1 -- 172.21.15.111:0/547566351 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f4afc0a99e0 con 0x7f4af8003970 2024-09-13T17:59:05.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.146+0000 7f4b07fff640 1 --2- 172.21.15.111:0/547566351 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4aec06ac00 0x7f4aec06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:05.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.146+0000 7f4b07fff640 1 --2- 172.21.15.111:0/547566351 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4aec06ac00 0x7f4aec06d0c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4b080b5ed0 tx=0x7f4af4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:05.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.150+0000 7f4b057fa640 1 -- 172.21.15.111:0/547566351 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4afc074b60 con 0x7f4af8003970 2024-09-13T17:59:05.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.344+0000 7f4b0f1b5640 1 -- 172.21.15.111:0/547566351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b080d3130 con 0x7f4af8003970 2024-09-13T17:59:05.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.345+0000 7f4b057fa640 1 -- 172.21.15.111:0/547566351 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f4afc036090 con 0x7f4af8003970 2024-09-13T17:59:05.347 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:05.347 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4217138481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4217138481},{"type":"v1","addr":"172.21.15.188:6834","nonce":4217138481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3099948287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3099948287},{"type":"v1","addr":"172.21.15.188:6835","nonce":3099948287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/904506495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":904506495},{"type":"v1","addr":"172.21.15.111:6837","nonce":904506495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:58:43.534567+0000","modified":"2024-09-13T17:59:04.553287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5487":{"gid":5487,"name":"c","rank":0,"incarnation":17,"state":"up:clientreplay","state_seq":10,"addr":"172.21.15.111:6835/214034224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":214034224},{"type":"v1","addr":"172.21.15.111:6835","nonce":214034224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:59:05.347 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 20 2024-09-13T17:59:05.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.347+0000 7f4b0f1b5640 1 -- 172.21.15.111:0/547566351 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4aec06ac00 msgr2=0x7f4aec06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:05.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.347+0000 7f4b0f1b5640 1 --2- 172.21.15.111:0/547566351 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4aec06ac00 0x7f4aec06d0c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4b080b5ed0 tx=0x7f4af4008040 comp rx=0 tx=0).stop 2024-09-13T17:59:05.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.347+0000 7f4b0f1b5640 1 -- 172.21.15.111:0/547566351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af8003970 msgr2=0x7f4b0814c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:05.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.347+0000 7f4b0f1b5640 1 --2- 172.21.15.111:0/547566351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af8003970 0x7f4b0814c6f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009ac0 tx=0x7f4afc004730 comp rx=0 tx=0).stop 2024-09-13T17:59:05.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.347+0000 7f4b0f1b5640 1 -- 172.21.15.111:0/547566351 shutdown_connections 2024-09-13T17:59:05.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.347+0000 7f4b0f1b5640 1 --2- 172.21.15.111:0/547566351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b081f5420 0x7f4b081f58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:05.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.347+0000 7f4b0f1b5640 1 --2- 172.21.15.111:0/547566351 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f4aec06ac00 0x7f4aec06d0c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:05.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.347+0000 7f4b0f1b5640 1 --2- 172.21.15.111:0/547566351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b080b0ee0 0x7f4b080b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:05.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.347+0000 7f4b0f1b5640 1 --2- 172.21.15.111:0/547566351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af8003970 0x7f4b0814c6f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:05.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.347+0000 7f4b0f1b5640 1 -- 172.21.15.111:0/547566351 >> 172.21.15.111:0/547566351 conn(0x7f4b08089f60 msgr2=0x7f4b080c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:05.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.347+0000 7f4b0f1b5640 1 -- 172.21.15.111:0/547566351 shutdown_connections 2024-09-13T17:59:05.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:05.347+0000 7f4b0f1b5640 1 -- 172.21.15.111:0/547566351 wait complete. 2024-09-13T17:59:05.763 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:58:43.534567+0000', 'modified': '2024-09-13T17:59:04.553287+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 40, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5487': {'gid': 5487, 'name': 'c', 'rank': 0, 'incarnation': 17, 'state': 'up:clientreplay', 'state_seq': 10, 'addr': '172.21.15.111:6835/214034224', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 214034224}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 214034224}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T17:59:05.763 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5487:up:clientreplay 2024-09-13T17:59:06.764 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:59:06.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.871+0000 7f5c2350f640 1 Processor -- start 2024-09-13T17:59:06.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.871+0000 7f5c2350f640 1 -- start start 2024-09-13T17:59:06.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.872+0000 7f5c2350f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5c1c15a2e0 0x7f5c1c0c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:06.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.872+0000 7f5c2350f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c1c0cb3f0 0x7f5c1c06d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:06.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.872+0000 7f5c2350f640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1c0c8920 con 0x7f5c1c0c9f80 2024-09-13T17:59:06.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.872+0000 7f5c2350f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1c15a6c0 con 0x7f5c1c15a2e0 2024-09-13T17:59:06.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.872+0000 7f5c2350f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0ca500 con 0x7f5c1c0cb3f0 2024-09-13T17:59:06.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.872+0000 7f5c20aaa640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5c1c0c9f80 0x7f5c1c0cace0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33512/0 (socket says 172.21.15.111:33512) 2024-09-13T17:59:06.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.872+0000 7f5c20aaa640 1 -- 172.21.15.111:0/2673224525 learned_addr learned my addr 172.21.15.111:0/2673224525 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:06.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.873+0000 7f5c21aac640 1 --2- 172.21.15.111:0/2673224525 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c1c0cb3f0 0x7f5c1c06d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:06.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.873+0000 7f5c212ab640 1 --2- 172.21.15.111:0/2673224525 >> v2:172.21.15.188:3300/0 conn(0x7f5c1c15a2e0 0x7f5c1c0c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:06.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.873+0000 7f5c13fff640 1 -- 172.21.15.111:0/2673224525 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3771036169 0 0) 0x7f5c1c0c8920 con 0x7f5c1c0c9f80 2024-09-13T17:59:06.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.873+0000 7f5c13fff640 1 -- 172.21.15.111:0/2673224525 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c08003520 con 0x7f5c1c0c9f80 2024-09-13T17:59:06.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.873+0000 7f5c212ab640 1 -- 172.21.15.111:0/2673224525 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c1c0cb3f0 msgr2=0x7f5c1c06d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:06.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.873+0000 7f5c212ab640 1 --2- 172.21.15.111:0/2673224525 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c1c0cb3f0 0x7f5c1c06d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:06.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.873+0000 7f5c212ab640 1 -- 172.21.15.111:0/2673224525 >> v1:172.21.15.111:6789/0 conn(0x7f5c1c0c9f80 legacy=0x7f5c1c0cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:06.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.873+0000 7f5c212ab640 1 -- 172.21.15.111:0/2673224525 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c0c8b70 con 0x7f5c1c15a2e0 2024-09-13T17:59:06.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.874+0000 7f5c212ab640 1 --2- 172.21.15.111:0/2673224525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c1c15a2e0 0x7f5c1c0c9a40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c180097a0 tx=0x7f5c1802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d61533053b3ffc2d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:06.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.874+0000 7f5c13fff640 1 -- 172.21.15.111:0/2673224525 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18032440 con 0x7f5c1c15a2e0 2024-09-13T17:59:06.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.874+0000 7f5c13fff640 1 -- 172.21.15.111:0/2673224525 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c180325a0 con 0x7f5c1c15a2e0 2024-09-13T17:59:06.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.874+0000 7f5c13fff640 1 -- 172.21.15.111:0/2673224525 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18032910 con 0x7f5c1c15a2e0 2024-09-13T17:59:06.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.874+0000 7f5c2350f640 1 -- 172.21.15.111:0/2673224525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c1c15a2e0 msgr2=0x7f5c1c0c9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:06.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.874+0000 7f5c2350f640 1 --2- 172.21.15.111:0/2673224525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c1c15a2e0 0x7f5c1c0c9a40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c180097a0 tx=0x7f5c1802a180 comp rx=0 tx=0).stop 2024-09-13T17:59:06.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.874+0000 7f5c2350f640 1 -- 172.21.15.111:0/2673224525 shutdown_connections 2024-09-13T17:59:06.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.874+0000 7f5c2350f640 1 --2- 172.21.15.111:0/2673224525 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c1c0cb3f0 0x7f5c1c06d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:06.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.874+0000 7f5c2350f640 1 --2- 172.21.15.111:0/2673224525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c1c15a2e0 0x7f5c1c0c9a40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:06.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.874+0000 7f5c2350f640 1 -- 172.21.15.111:0/2673224525 >> 172.21.15.111:0/2673224525 conn(0x7f5c1c069ed0 msgr2=0x7f5c1c06a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:06.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.875+0000 7f5c2350f640 1 -- 172.21.15.111:0/2673224525 shutdown_connections 2024-09-13T17:59:06.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.875+0000 7f5c2350f640 1 -- 172.21.15.111:0/2673224525 wait complete. 2024-09-13T17:59:06.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.876+0000 7f5c2350f640 1 Processor -- start 2024-09-13T17:59:06.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.876+0000 7f5c2350f640 1 -- start start 2024-09-13T17:59:06.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.876+0000 7f5c2350f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c1c0c9f80 0x7f5c1c1efae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:06.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.876+0000 7f5c2350f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c1c0cb3f0 0x7f5c1c159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:06.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.876+0000 7f5c2350f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c1c15a2e0 0x7f5c1c1f0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:06.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.876+0000 7f5c2350f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0b1500 con 0x7f5c1c0cb3f0 2024-09-13T17:59:06.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.876+0000 7f5c2350f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0c3ad0 con 0x7f5c1c15a2e0 2024-09-13T17:59:06.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.876+0000 7f5c2350f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c1f0020 con 0x7f5c1c0c9f80 2024-09-13T17:59:06.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.877+0000 7f5c20aaa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c1c0cb3f0 0x7f5c1c159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:06.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.877+0000 7f5c20aaa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c1c0cb3f0 0x7f5c1c159230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52354/0 (socket says 172.21.15.111:52354) 2024-09-13T17:59:06.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.877+0000 7f5c20aaa640 1 -- 172.21.15.111:0/2317173 learned_addr learned my addr 172.21.15.111:0/2317173 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:06.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.877+0000 7f5c212ab640 1 --2- 172.21.15.111:0/2317173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c1c0c9f80 0x7f5c1c1efae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:06.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.877+0000 7f5c21aac640 1 --2- 172.21.15.111:0/2317173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c1c15a2e0 0x7f5c1c1f0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:06.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.877+0000 7f5c20aaa640 1 -- 172.21.15.111:0/2317173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c1c0c9f80 msgr2=0x7f5c1c1efae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:06.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.877+0000 7f5c20aaa640 1 --2- 172.21.15.111:0/2317173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c1c0c9f80 0x7f5c1c1efae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:06.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.877+0000 7f5c20aaa640 1 -- 172.21.15.111:0/2317173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c1c15a2e0 msgr2=0x7f5c1c1f0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:06.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.877+0000 7f5c20aaa640 1 --2- 172.21.15.111:0/2317173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c1c15a2e0 0x7f5c1c1f0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:06.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.877+0000 7f5c20aaa640 1 -- 172.21.15.111:0/2317173 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c18009400 con 0x7f5c1c0cb3f0 2024-09-13T17:59:06.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.877+0000 7f5c21aac640 1 --2- 172.21.15.111:0/2317173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c1c15a2e0 0x7f5c1c1f0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:06.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.877+0000 7f5c20aaa640 1 --2- 172.21.15.111:0/2317173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c1c0cb3f0 0x7f5c1c159230 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00c390 tx=0x7f5c0c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:06.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.877+0000 7f5c11ffb640 1 -- 172.21.15.111:0/2317173 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c005ce0 con 0x7f5c1c0cb3f0 2024-09-13T17:59:06.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.877+0000 7f5c11ffb640 1 -- 172.21.15.111:0/2317173 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0c005e40 con 0x7f5c1c0cb3f0 2024-09-13T17:59:06.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.877+0000 7f5c2350f640 1 -- 172.21.15.111:0/2317173 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1c1f02c0 con 0x7f5c1c0cb3f0 2024-09-13T17:59:06.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.877+0000 7f5c11ffb640 1 -- 172.21.15.111:0/2317173 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c0072f0 con 0x7f5c1c0cb3f0 2024-09-13T17:59:06.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.877+0000 7f5c2350f640 1 -- 172.21.15.111:0/2317173 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c1f0880 con 0x7f5c1c0cb3f0 2024-09-13T17:59:06.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.878+0000 7f5c2350f640 1 -- 172.21.15.111:0/2317173 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1c066f30 con 0x7f5c1c0cb3f0 2024-09-13T17:59:06.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.879+0000 7f5c11ffb640 1 -- 172.21.15.111:0/2317173 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5c0c007450 con 0x7f5c1c0cb3f0 2024-09-13T17:59:06.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.879+0000 7f5c11ffb640 1 --2- 172.21.15.111:0/2317173 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5c0006ac00 0x7f5c0006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:06.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.880+0000 7f5c212ab640 1 --2- 172.21.15.111:0/2317173 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5c0006ac00 0x7f5c0006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:06.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.880+0000 7f5c11ffb640 1 -- 172.21.15.111:0/2317173 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f5c0c088a50 con 0x7f5c1c0cb3f0 2024-09-13T17:59:06.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.880+0000 7f5c212ab640 1 --2- 172.21.15.111:0/2317173 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5c0006ac00 0x7f5c0006d0c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5c18034000 tx=0x7f5c1802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:06.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:06.884+0000 7f5c11ffb640 1 -- 172.21.15.111:0/2317173 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f5c0c053b40 con 0x7f5c1c0cb3f0 2024-09-13T17:59:07.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.080+0000 7f5c2350f640 1 -- 172.21.15.111:0/2317173 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c1c06e1a0 con 0x7f5c1c0cb3f0 2024-09-13T17:59:07.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.080+0000 7f5c11ffb640 1 -- 172.21.15.111:0/2317173 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4742 (secure 0 0 0) 0x7f5c0c017020 con 0x7f5c1c0cb3f0 2024-09-13T17:59:07.082 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:07.082 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4217138481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4217138481},{"type":"v1","addr":"172.21.15.188:6834","nonce":4217138481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3099948287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3099948287},{"type":"v1","addr":"172.21.15.188:6835","nonce":3099948287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/904506495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":904506495},{"type":"v1","addr":"172.21.15.111:6837","nonce":904506495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:58:43.534567+0000","modified":"2024-09-13T17:59:05.554735+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5487":{"gid":5487,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6835/214034224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":214034224},{"type":"v1","addr":"172.21.15.111:6835","nonce":214034224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:59:07.083 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 21 2024-09-13T17:59:07.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.083+0000 7f5c2350f640 1 -- 172.21.15.111:0/2317173 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5c0006ac00 msgr2=0x7f5c0006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:07.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.083+0000 7f5c2350f640 1 --2- 172.21.15.111:0/2317173 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5c0006ac00 0x7f5c0006d0c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5c18034000 tx=0x7f5c1802e000 comp rx=0 tx=0).stop 2024-09-13T17:59:07.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.083+0000 7f5c2350f640 1 -- 172.21.15.111:0/2317173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c1c0cb3f0 msgr2=0x7f5c1c159230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:07.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.083+0000 7f5c2350f640 1 --2- 172.21.15.111:0/2317173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c1c0cb3f0 0x7f5c1c159230 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00c390 tx=0x7f5c0c00c850 comp rx=0 tx=0).stop 2024-09-13T17:59:07.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.083+0000 7f5c2350f640 1 -- 172.21.15.111:0/2317173 shutdown_connections 2024-09-13T17:59:07.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.083+0000 7f5c2350f640 1 --2- 172.21.15.111:0/2317173 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5c0006ac00 0x7f5c0006d0c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.083+0000 7f5c2350f640 1 --2- 172.21.15.111:0/2317173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c1c15a2e0 0x7f5c1c1f0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.083+0000 7f5c2350f640 1 --2- 172.21.15.111:0/2317173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c1c0cb3f0 0x7f5c1c159230 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.083+0000 7f5c2350f640 1 --2- 172.21.15.111:0/2317173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c1c0c9f80 0x7f5c1c1efae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.083+0000 7f5c2350f640 1 -- 172.21.15.111:0/2317173 >> 172.21.15.111:0/2317173 conn(0x7f5c1c069ed0 msgr2=0x7f5c1c0addd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:07.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.083+0000 7f5c2350f640 1 -- 172.21.15.111:0/2317173 shutdown_connections 2024-09-13T17:59:07.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.083+0000 7f5c2350f640 1 -- 172.21.15.111:0/2317173 wait complete. 2024-09-13T17:59:07.498 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:58:43.534567+0000', 'modified': '2024-09-13T17:59:05.554735+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 40, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5487': {'gid': 5487, 'name': 'c', 'rank': 0, 'incarnation': 17, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.111:6835/214034224', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 214034224}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 214034224}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T17:59:07.498 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T17:59:07.498 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-13T17:59:07.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.599+0000 7ff5135b5640 1 Processor -- start 2024-09-13T17:59:07.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.599+0000 7ff5135b5640 1 -- start start 2024-09-13T17:59:07.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.600+0000 7ff5135b5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff50c0b4660 0x7ff50c06b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.600+0000 7ff5135b5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50c0d1840 con 0x7ff50c0d07b0 2024-09-13T17:59:07.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.600+0000 7ff5135b5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50c0d0c30 con 0x7ff50c15c6e0 2024-09-13T17:59:07.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.600+0000 7ff5135b5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50c0d0f10 con 0x7ff50c0b4660 2024-09-13T17:59:07.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.600+0000 7ff511b52640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff50c0b4660 0x7ff50c06b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.600+0000 7ff510b50640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff50c0d07b0 0x7ff50c06ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33520/0 (socket says 172.21.15.111:33520) 2024-09-13T17:59:07.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.600+0000 7ff510b50640 1 -- 172.21.15.111:0/3314758950 learned_addr learned my addr 172.21.15.111:0/3314758950 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:07.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.601+0000 7ff511b52640 1 -- 172.21.15.111:0/3314758950 >> v1:172.21.15.188:6789/0 conn(0x7ff50c15c6e0 legacy=0x7ff50c0b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:07.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.601+0000 7ff511b52640 1 -- 172.21.15.111:0/3314758950 >> v1:172.21.15.111:6789/0 conn(0x7ff50c0d07b0 legacy=0x7ff50c06ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:07.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.601+0000 7ff511b52640 1 -- 172.21.15.111:0/3314758950 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c0d11f0 con 0x7ff50c0b4660 2024-09-13T17:59:07.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.601+0000 7ff503fff640 1 -- 172.21.15.111:0/3314758950 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058367634 0 0) 0x7ff50c0d1840 con 0x7ff50c0d07b0 2024-09-13T17:59:07.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.601+0000 7ff511b52640 1 --2- 172.21.15.111:0/3314758950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff50c0b4660 0x7ff50c06b420 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff504009d90 tx=0x7ff50402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a595a63971712a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:07.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.602+0000 7ff503fff640 1 -- 172.21.15.111:0/3314758950 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504031600 con 0x7ff50c0b4660 2024-09-13T17:59:07.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.602+0000 7ff503fff640 1 -- 172.21.15.111:0/3314758950 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50402d050 con 0x7ff50c0b4660 2024-09-13T17:59:07.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.602+0000 7ff503fff640 1 -- 172.21.15.111:0/3314758950 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504031980 con 0x7ff50c0b4660 2024-09-13T17:59:07.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.602+0000 7ff5135b5640 1 -- 172.21.15.111:0/3314758950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff50c0b4660 msgr2=0x7ff50c06b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:07.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.602+0000 7ff5135b5640 1 --2- 172.21.15.111:0/3314758950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff50c0b4660 0x7ff50c06b420 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff504009d90 tx=0x7ff50402a610 comp rx=0 tx=0).stop 2024-09-13T17:59:07.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.602+0000 7ff5135b5640 1 -- 172.21.15.111:0/3314758950 shutdown_connections 2024-09-13T17:59:07.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.602+0000 7ff5135b5640 1 --2- 172.21.15.111:0/3314758950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff50c0b4660 0x7ff50c06b420 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.602+0000 7ff5135b5640 1 -- 172.21.15.111:0/3314758950 >> 172.21.15.111:0/3314758950 conn(0x7ff50c089f60 msgr2=0x7ff50c0d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:07.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.602+0000 7ff5135b5640 1 -- 172.21.15.111:0/3314758950 shutdown_connections 2024-09-13T17:59:07.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.602+0000 7ff5135b5640 1 -- 172.21.15.111:0/3314758950 wait complete. 2024-09-13T17:59:07.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.603+0000 7ff5135b5640 1 Processor -- start 2024-09-13T17:59:07.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.603+0000 7ff5135b5640 1 -- start start 2024-09-13T17:59:07.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.604+0000 7ff5135b5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff50c0b4660 0x7ff50c0c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.604+0000 7ff5135b5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff50c0d07b0 0x7ff50c0c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.604+0000 7ff5135b5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff50c15c6e0 0x7ff50c0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.604+0000 7ff5135b5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c0c8bc0 con 0x7ff50c0d07b0 2024-09-13T17:59:07.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.604+0000 7ff5135b5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c15f510 con 0x7ff50c0b4660 2024-09-13T17:59:07.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.604+0000 7ff5135b5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c15e990 con 0x7ff50c15c6e0 2024-09-13T17:59:07.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.604+0000 7ff510b50640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff50c0d07b0 0x7ff50c0c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.604+0000 7ff510b50640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff50c0d07b0 0x7ff50c0c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52372/0 (socket says 172.21.15.111:52372) 2024-09-13T17:59:07.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.604+0000 7ff510b50640 1 -- 172.21.15.111:0/2891277867 learned_addr learned my addr 172.21.15.111:0/2891277867 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:07.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.604+0000 7ff511351640 1 --2- 172.21.15.111:0/2891277867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff50c0b4660 0x7ff50c0c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.604+0000 7ff510b50640 1 -- 172.21.15.111:0/2891277867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff50c15c6e0 msgr2=0x7ff50c0c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:07.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.605+0000 7ff510b50640 1 --2- 172.21.15.111:0/2891277867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff50c15c6e0 0x7ff50c0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.605+0000 7ff510b50640 1 -- 172.21.15.111:0/2891277867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff50c0b4660 msgr2=0x7ff50c0c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:07.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.605+0000 7ff510b50640 1 --2- 172.21.15.111:0/2891277867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff50c0b4660 0x7ff50c0c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.605+0000 7ff510b50640 1 -- 172.21.15.111:0/2891277867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff504009a70 con 0x7ff50c0d07b0 2024-09-13T17:59:07.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.605+0000 7ff510b50640 1 --2- 172.21.15.111:0/2891277867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff50c0d07b0 0x7ff50c0c98b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00c4d0 tx=0x7ff4fc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:07.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.605+0000 7ff501ffb640 1 -- 172.21.15.111:0/2891277867 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0074e0 con 0x7ff50c0d07b0 2024-09-13T17:59:07.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.605+0000 7ff501ffb640 1 -- 172.21.15.111:0/2891277867 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc007660 con 0x7ff50c0d07b0 2024-09-13T17:59:07.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.605+0000 7ff5135b5640 1 -- 172.21.15.111:0/2891277867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50c15ec70 con 0x7ff50c0d07b0 2024-09-13T17:59:07.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.605+0000 7ff501ffb640 1 -- 172.21.15.111:0/2891277867 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc004a20 con 0x7ff50c0d07b0 2024-09-13T17:59:07.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.605+0000 7ff5135b5640 1 -- 172.21.15.111:0/2891277867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50c15f230 con 0x7ff50c0d07b0 2024-09-13T17:59:07.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.607+0000 7ff501ffb640 1 -- 172.21.15.111:0/2891277867 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff4fc004b80 con 0x7ff50c0d07b0 2024-09-13T17:59:07.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.607+0000 7ff501ffb640 1 --2- 172.21.15.111:0/2891277867 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff4f006ac30 0x7ff4f006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.607+0000 7ff501ffb640 1 -- 172.21.15.111:0/2891277867 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6115+0+0 (secure 0 0 0) 0x7ff4fc088a00 con 0x7ff50c0d07b0 2024-09-13T17:59:07.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.607+0000 7ff511351640 1 --2- 172.21.15.111:0/2891277867 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff4f006ac30 0x7ff4f006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.608+0000 7ff511351640 1 --2- 172.21.15.111:0/2891277867 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff4f006ac30 0x7ff4f006d0f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff50800a750 tx=0x7ff508008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:07.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.608+0000 7ff5162d5740 1 Processor -- start 2024-09-13T17:59:07.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.608+0000 7ff5162d5740 1 -- start start 2024-09-13T17:59:07.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff5162d5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc0a0a0 0x55cf3bc0a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff5162d5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc07860 0x55cf3bc57570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff5162d5740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc31550 0x55cf3bc57ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff5162d5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf3bb647c0 con 0x55cf3bc0a0a0 2024-09-13T17:59:07.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff5162d5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf3bb73b70 con 0x55cf3bc07860 2024-09-13T17:59:07.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff5162d5740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cf3bb71300 con 0x55cf3bc31550 2024-09-13T17:59:07.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff511351640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc0a0a0 0x55cf3bc0a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff511351640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc0a0a0 0x55cf3bc0a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52376/0 (socket says 172.21.15.111:52376) 2024-09-13T17:59:07.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff511351640 1 -- 172.21.15.111:0/1659192126 learned_addr learned my addr 172.21.15.111:0/1659192126 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:07.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff510b50640 1 --2- 172.21.15.111:0/1659192126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc07860 0x55cf3bc57570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff511b52640 1 --2- 172.21.15.111:0/1659192126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc31550 0x55cf3bc57ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff511351640 1 -- 172.21.15.111:0/1659192126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc31550 msgr2=0x55cf3bc57ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:07.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff511351640 1 --2- 172.21.15.111:0/1659192126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc31550 0x55cf3bc57ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff511351640 1 -- 172.21.15.111:0/1659192126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc07860 msgr2=0x55cf3bc57570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:07.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff511351640 1 --2- 172.21.15.111:0/1659192126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc07860 0x55cf3bc57570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff511351640 1 -- 172.21.15.111:0/1659192126 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf3baf94b0 con 0x55cf3bc0a0a0 2024-09-13T17:59:07.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff511b52640 1 --2- 172.21.15.111:0/1659192126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc31550 0x55cf3bc57ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:07.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.609+0000 7ff511351640 1 --2- 172.21.15.111:0/1659192126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc0a0a0 0x55cf3bc0a480 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff50800c930 tx=0x7ff50800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51bd1cb3e65bc279 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:07.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.610+0000 7ff4f6ffd640 1 -- 172.21.15.111:0/1659192126 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508013b80 con 0x55cf3bc0a0a0 2024-09-13T17:59:07.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.610+0000 7ff4f6ffd640 1 -- 172.21.15.111:0/1659192126 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff508013ce0 con 0x55cf3bc0a0a0 2024-09-13T17:59:07.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.610+0000 7ff4f6ffd640 1 -- 172.21.15.111:0/1659192126 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50801d630 con 0x55cf3bc0a0a0 2024-09-13T17:59:07.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.610+0000 7ff5162d5740 1 -- 172.21.15.111:0/1659192126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc0a0a0 msgr2=0x55cf3bc0a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:07.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.610+0000 7ff5162d5740 1 --2- 172.21.15.111:0/1659192126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc0a0a0 0x55cf3bc0a480 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff50800c930 tx=0x7ff50800f000 comp rx=0 tx=0).stop 2024-09-13T17:59:07.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.610+0000 7ff5162d5740 1 -- 172.21.15.111:0/1659192126 shutdown_connections 2024-09-13T17:59:07.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.610+0000 7ff5162d5740 1 --2- 172.21.15.111:0/1659192126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc31550 0x55cf3bc57ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.610+0000 7ff5162d5740 1 --2- 172.21.15.111:0/1659192126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc07860 0x55cf3bc57570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.610+0000 7ff5162d5740 1 --2- 172.21.15.111:0/1659192126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc0a0a0 0x55cf3bc0a480 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.610+0000 7ff5162d5740 1 -- 172.21.15.111:0/1659192126 >> 172.21.15.111:0/1659192126 conn(0x55cf3bc067d0 msgr2=0x55cf3bc4ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:07.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.610+0000 7ff5162d5740 1 -- 172.21.15.111:0/1659192126 shutdown_connections 2024-09-13T17:59:07.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.610+0000 7ff5162d5740 1 -- 172.21.15.111:0/1659192126 wait complete. 2024-09-13T17:59:07.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.611+0000 7ff5162d5740 1 -- start start 2024-09-13T17:59:07.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.611+0000 7ff5162d5740 1 Processor -- start 2024-09-13T17:59:07.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff5162d5740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc0a0a0 0x55cf3bc56d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff5162d5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bc2f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff5162d5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc2fe60 0x55cf3bc61cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff5162d5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf3bb6cbf0 con 0x55cf3bc31550 2024-09-13T17:59:07.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff5162d5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf3bb2b250 con 0x55cf3bc2fe60 2024-09-13T17:59:07.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff5162d5740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cf3bb2aef0 con 0x55cf3bc0a0a0 2024-09-13T17:59:07.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff5162d5740 10 client.0 _ll_register_callbacks cb 0x55cf3bb5d740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:59:07.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff510b50640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bc2f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff510b50640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bc2f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52382/0 (socket says 172.21.15.111:52382) 2024-09-13T17:59:07.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff510b50640 1 -- 172.21.15.111:0/3040966211 learned_addr learned my addr 172.21.15.111:0/3040966211 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:07.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff511351640 1 --2- 172.21.15.111:0/3040966211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc0a0a0 0x55cf3bc56d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff511b52640 1 --2- 172.21.15.111:0/3040966211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc2fe60 0x55cf3bc61cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff510b50640 1 -- 172.21.15.111:0/3040966211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc0a0a0 msgr2=0x55cf3bc56d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:07.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff510b50640 1 --2- 172.21.15.111:0/3040966211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc0a0a0 0x55cf3bc56d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff510b50640 1 -- 172.21.15.111:0/3040966211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc2fe60 msgr2=0x55cf3bc61cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:07.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff510b50640 1 --2- 172.21.15.111:0/3040966211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc2fe60 0x55cf3bc61cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff510b50640 1 -- 172.21.15.111:0/3040966211 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50800c5e0 con 0x55cf3bc31550 2024-09-13T17:59:07.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff511351640 1 --2- 172.21.15.111:0/3040966211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc0a0a0 0x55cf3bc56d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:07.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff510b50640 1 --2- 172.21.15.111:0/3040966211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bc2f920 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00ce90 tx=0x7ff4fc057840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8edb8d43a4dffb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:07.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.612+0000 7ff4ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-13T17:59:07.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.615+0000 7ff4ebfff640 1 -- 172.21.15.111:0/3040966211 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc055b60 con 0x55cf3bc31550 2024-09-13T17:59:07.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.615+0000 7ff5162d5740 1 -- 172.21.15.111:0/3040966211 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf3b8603c0 con 0x55cf3bc31550 2024-09-13T17:59:07.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.615+0000 7ff4ebfff640 1 -- 172.21.15.111:0/3040966211 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6115+0+0 (secure 0 0 0) 0x7ff4fc05f660 con 0x55cf3bc31550 2024-09-13T17:59:07.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.615+0000 7ff4ebfff640 1 -- 172.21.15.111:0/3040966211 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4fc05f820 con 0x55cf3bc31550 2024-09-13T17:59:07.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.615+0000 7ff5162d5740 10 client.5745 fetch_fsmap learned FSMap version 21 2024-09-13T17:59:07.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.615+0000 7ff5162d5740 1 -- 172.21.15.111:0/3040966211 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55cf3bb6b270 con 0x55cf3bc31550 2024-09-13T17:59:07.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.615+0000 7ff4ebfff640 1 -- 172.21.15.111:0/3040966211 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cf3b8603c0 con 0x55cf3bc31550 2024-09-13T17:59:07.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.615+0000 7ff4ebfff640 1 -- 172.21.15.111:0/3040966211 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 21) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff4fc0534f0 con 0x55cf3bc31550 2024-09-13T17:59:07.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.615+0000 7ff5162d5740 10 client.5745 fetch_fsmap finished waiting for FSMap version 21 2024-09-13T17:59:07.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.615+0000 7ff5162d5740 10 client.5745 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T17:59:07.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.615+0000 7ff5162d5740 20 client.5745 populate_metadata read hostname 'smithi111' 2024-09-13T17:59:07.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.615+0000 7ff5162d5740 1 --2- 172.21.15.111:0/3040966211 >> [v2:172.21.15.111:6834/214034224,v1:172.21.15.111:6835/214034224] conn(0x55cf3bc626d0 0x55cf3bc552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.615+0000 7ff5162d5740 4 client.5745 mds_command: new command op to 5487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:59:07.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.615+0000 7ff5162d5740 1 -- 172.21.15.111:0/3040966211 --> [v2:172.21.15.111:6834/214034224,v1:172.21.15.111:6835/214034224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf3b8716c0 con 0x55cf3bc626d0 2024-09-13T17:59:07.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.615+0000 7ff511351640 1 --2- 172.21.15.111:0/3040966211 >> [v2:172.21.15.111:6834/214034224,v1:172.21.15.111:6835/214034224] conn(0x55cf3bc626d0 0x55cf3bc552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.616+0000 7ff4ebfff640 10 client.5745 ms_handle_connect on v2:172.21.15.111:6834/214034224 2024-09-13T17:59:07.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.616+0000 7ff511351640 1 --2- 172.21.15.111:0/3040966211 >> [v2:172.21.15.111:6834/214034224,v1:172.21.15.111:6835/214034224] conn(0x55cf3bc626d0 0x55cf3bc552d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34e39781c0520b21 server_cookie=eee5b64f9ab528aa in_seq=0 out_seq=0 2024-09-13T17:59:07.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.616+0000 7ff4ebfff640 1 -- 172.21.15.111:0/3040966211 <== mds.0 v2:172.21.15.111:6834/214034224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55cf3b8716c0 con 0x55cf3bc626d0 2024-09-13T17:59:07.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.616+0000 7ff4ebfff640 10 client.5745 handle_command_reply: tid=0 2024-09-13T17:59:07.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.616+0000 7ff5162d5740 1 client.5745 shutdown 2024-09-13T17:59:07.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.616+0000 7ff5162d5740 1 -- 172.21.15.111:0/3040966211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 msgr2=0x55cf3bc2f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:07.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.616+0000 7ff5162d5740 1 --2- 172.21.15.111:0/3040966211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bc2f920 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00ce90 tx=0x7ff4fc057840 comp rx=0 tx=0).stop 2024-09-13T17:59:07.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.616+0000 7ff5162d5740 1 -- 172.21.15.111:0/3040966211 shutdown_connections 2024-09-13T17:59:07.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.616+0000 7ff5162d5740 1 --2- 172.21.15.111:0/3040966211 >> [v2:172.21.15.111:6834/214034224,v1:172.21.15.111:6835/214034224] conn(0x55cf3bc626d0 0x55cf3bc552d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.616+0000 7ff4ebfff640 0 client.5745 ms_handle_reset on v2:172.21.15.111:6834/214034224 2024-09-13T17:59:07.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.617+0000 7ff5162d5740 1 --2- 172.21.15.111:0/3040966211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc2fe60 0x55cf3bc61cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.617+0000 7ff5162d5740 1 --2- 172.21.15.111:0/3040966211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bc2f920 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.617+0000 7ff5162d5740 1 --2- 172.21.15.111:0/3040966211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc0a0a0 0x55cf3bc56d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.617+0000 7ff5162d5740 1 -- 172.21.15.111:0/3040966211 >> 172.21.15.111:0/3040966211 conn(0x55cf3bc07860 msgr2=0x55cf3bc4e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:07.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.617+0000 7ff5162d5740 1 -- 172.21.15.111:0/3040966211 shutdown_connections 2024-09-13T17:59:07.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.617+0000 7ff5162d5740 1 -- 172.21.15.111:0/3040966211 wait complete. 2024-09-13T17:59:07.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.617+0000 7ff5162d5740 20 client.5745 trim_cache size 0 max 16384 2024-09-13T17:59:07.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.632+0000 7ff5162d5740 1 Processor -- start 2024-09-13T17:59:07.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.632+0000 7ff5162d5740 1 -- start start 2024-09-13T17:59:07.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.633+0000 7ff5162d5740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc0a0a0 0x55cf3bc35d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.633+0000 7ff5162d5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc2fe60 0x55cf3bc2f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.633+0000 7ff5162d5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bcaf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.633+0000 7ff5162d5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf3baf0460 con 0x55cf3bc31550 2024-09-13T17:59:07.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.633+0000 7ff5162d5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf3bc51c20 con 0x55cf3bc2fe60 2024-09-13T17:59:07.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.633+0000 7ff5162d5740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cf3bba1e20 con 0x55cf3bc0a0a0 2024-09-13T17:59:07.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.633+0000 7ff511b52640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bcaf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.633+0000 7ff511b52640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bcaf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52392/0 (socket says 172.21.15.111:52392) 2024-09-13T17:59:07.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.633+0000 7ff511b52640 1 -- 172.21.15.111:0/1196533317 learned_addr learned my addr 172.21.15.111:0/1196533317 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:07.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.634+0000 7ff511351640 1 --2- 172.21.15.111:0/1196533317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc0a0a0 0x55cf3bc35d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.634+0000 7ff511b52640 1 -- 172.21.15.111:0/1196533317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc0a0a0 msgr2=0x55cf3bc35d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:07.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.634+0000 7ff510b50640 1 --2- 172.21.15.111:0/1196533317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc2fe60 0x55cf3bc2f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.634+0000 7ff511b52640 1 --2- 172.21.15.111:0/1196533317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc0a0a0 0x55cf3bc35d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.634+0000 7ff511b52640 1 -- 172.21.15.111:0/1196533317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc2fe60 msgr2=0x55cf3bc2f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:07.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.634+0000 7ff511b52640 1 --2- 172.21.15.111:0/1196533317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc2fe60 0x55cf3bc2f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.634+0000 7ff511b52640 1 -- 172.21.15.111:0/1196533317 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf3bb0d5f0 con 0x55cf3bc31550 2024-09-13T17:59:07.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.634+0000 7ff511351640 1 --2- 172.21.15.111:0/1196533317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc0a0a0 0x55cf3bc35d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:59:07.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.634+0000 7ff510b50640 1 --2- 172.21.15.111:0/1196533317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc2fe60 0x55cf3bc2f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:59:07.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.634+0000 7ff511b52640 1 --2- 172.21.15.111:0/1196533317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bcaf630 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff504009eb0 tx=0x7ff50402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa9404070cf49cc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:07.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.634+0000 7ff4f77fe640 1 -- 172.21.15.111:0/1196533317 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504042070 con 0x55cf3bc31550 2024-09-13T17:59:07.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.634+0000 7ff4f77fe640 1 -- 172.21.15.111:0/1196533317 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff504031860 con 0x55cf3bc31550 2024-09-13T17:59:07.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.634+0000 7ff4f77fe640 1 -- 172.21.15.111:0/1196533317 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504031bf0 con 0x55cf3bc31550 2024-09-13T17:59:07.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.635+0000 7ff5162d5740 1 -- 172.21.15.111:0/1196533317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 msgr2=0x55cf3bcaf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:07.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.635+0000 7ff5162d5740 1 --2- 172.21.15.111:0/1196533317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bcaf630 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff504009eb0 tx=0x7ff50402c000 comp rx=0 tx=0).stop 2024-09-13T17:59:07.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.635+0000 7ff5162d5740 1 -- 172.21.15.111:0/1196533317 shutdown_connections 2024-09-13T17:59:07.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.635+0000 7ff5162d5740 1 --2- 172.21.15.111:0/1196533317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bcaf630 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.635+0000 7ff5162d5740 1 --2- 172.21.15.111:0/1196533317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc2fe60 0x55cf3bc2f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.635+0000 7ff5162d5740 1 --2- 172.21.15.111:0/1196533317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc0a0a0 0x55cf3bc35d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.635+0000 7ff5162d5740 1 -- 172.21.15.111:0/1196533317 >> 172.21.15.111:0/1196533317 conn(0x55cf3bc07860 msgr2=0x55cf3bc06000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:07.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.635+0000 7ff5162d5740 1 -- 172.21.15.111:0/1196533317 shutdown_connections 2024-09-13T17:59:07.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.635+0000 7ff5162d5740 1 -- 172.21.15.111:0/1196533317 wait complete. 2024-09-13T17:59:07.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.635+0000 7ff5162d5740 1 -- start start 2024-09-13T17:59:07.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.635+0000 7ff5162d5740 1 Processor -- start 2024-09-13T17:59:07.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.636+0000 7ff5162d5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc0a0a0 0x55cf3bc62af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.636+0000 7ff5162d5740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc2fe60 0x55cf3bcae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.636+0000 7ff5162d5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bc31a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.636+0000 7ff5162d5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf3bc68bb0 con 0x55cf3bc31550 2024-09-13T17:59:07.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.636+0000 7ff5162d5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf3bc640a0 con 0x55cf3bc0a0a0 2024-09-13T17:59:07.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.636+0000 7ff5162d5740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cf3bc32370 con 0x55cf3bc2fe60 2024-09-13T17:59:07.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.636+0000 7ff5162d5740 10 client.0 _ll_register_callbacks cb 0x7ff50800a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:59:07.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.636+0000 7ff511b52640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bc31a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.636+0000 7ff511b52640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bc31a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52408/0 (socket says 172.21.15.111:52408) 2024-09-13T17:59:07.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.636+0000 7ff511b52640 1 -- 172.21.15.111:0/3668823193 learned_addr learned my addr 172.21.15.111:0/3668823193 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:07.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.636+0000 7ff511351640 1 --2- 172.21.15.111:0/3668823193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc0a0a0 0x55cf3bc62af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.636+0000 7ff510b50640 1 --2- 172.21.15.111:0/3668823193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc2fe60 0x55cf3bcae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff511b52640 1 -- 172.21.15.111:0/3668823193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc2fe60 msgr2=0x55cf3bcae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:07.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff511b52640 1 --2- 172.21.15.111:0/3668823193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc2fe60 0x55cf3bcae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff511b52640 1 -- 172.21.15.111:0/3668823193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc0a0a0 msgr2=0x55cf3bc62af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:07.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff511b52640 1 --2- 172.21.15.111:0/3668823193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc0a0a0 0x55cf3bc62af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff511b52640 1 -- 172.21.15.111:0/3668823193 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50c0d11f0 con 0x55cf3bc31550 2024-09-13T17:59:07.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff511351640 1 --2- 172.21.15.111:0/3668823193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc0a0a0 0x55cf3bc62af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:07.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff510b50640 1 --2- 172.21.15.111:0/3668823193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc2fe60 0x55cf3bcae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:07.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff511b52640 1 --2- 172.21.15.111:0/3668823193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bc31a20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff504009e80 tx=0x7ff504004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d876654728183ed server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:07.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff4ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-13T17:59:07.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff4ebfff640 1 -- 172.21.15.111:0/3668823193 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50403dba0 con 0x55cf3bc31550 2024-09-13T17:59:07.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff4ebfff640 1 -- 172.21.15.111:0/3668823193 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6144+0+0 (secure 0 0 0) 0x7ff50403dd00 con 0x55cf3bc31550 2024-09-13T17:59:07.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff5162d5740 1 -- 172.21.15.111:0/3668823193 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff508013ce0 con 0x55cf3bc31550 2024-09-13T17:59:07.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff4ebfff640 1 -- 172.21.15.111:0/3668823193 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5040314e0 con 0x55cf3bc31550 2024-09-13T17:59:07.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff5162d5740 10 client.5751 fetch_fsmap learned FSMap version 21 2024-09-13T17:59:07.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff5162d5740 1 -- 172.21.15.111:0/3668823193 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55cf3bb32d00 con 0x55cf3bc31550 2024-09-13T17:59:07.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff4ebfff640 1 -- 172.21.15.111:0/3668823193 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff508013ce0 con 0x55cf3bc31550 2024-09-13T17:59:07.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff4ebfff640 1 -- 172.21.15.111:0/3668823193 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 21) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff504038040 con 0x55cf3bc31550 2024-09-13T17:59:07.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff5162d5740 10 client.5751 fetch_fsmap finished waiting for FSMap version 21 2024-09-13T17:59:07.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff5162d5740 10 client.5751 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T17:59:07.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff5162d5740 20 client.5751 populate_metadata read hostname 'smithi111' 2024-09-13T17:59:07.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff5162d5740 1 --2- 172.21.15.111:0/3668823193 >> [v2:172.21.15.111:6834/214034224,v1:172.21.15.111:6835/214034224] conn(0x55cf3bc4f740 0x55cf3bc4fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff5162d5740 4 client.5751 mds_command: new command op to 5487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:59:07.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff5162d5740 1 -- 172.21.15.111:0/3668823193 --> [v2:172.21.15.111:6834/214034224,v1:172.21.15.111:6835/214034224] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cf3b8c4d10 con 0x55cf3bc4f740 2024-09-13T17:59:07.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.637+0000 7ff511351640 1 --2- 172.21.15.111:0/3668823193 >> [v2:172.21.15.111:6834/214034224,v1:172.21.15.111:6835/214034224] conn(0x55cf3bc4f740 0x55cf3bc4fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.638+0000 7ff4ebfff640 10 client.5751 ms_handle_connect on v2:172.21.15.111:6834/214034224 2024-09-13T17:59:07.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.638+0000 7ff511351640 1 --2- 172.21.15.111:0/3668823193 >> [v2:172.21.15.111:6834/214034224,v1:172.21.15.111:6835/214034224] conn(0x55cf3bc4f740 0x55cf3bc4fb20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=681e7788affb1a00 server_cookie=27ec1d215d2c0ea in_seq=0 out_seq=0 2024-09-13T17:59:07.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.638+0000 7ff4ebfff640 1 -- 172.21.15.111:0/3668823193 <== mds.0 v2:172.21.15.111:6834/214034224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55cf3b8c4d10 con 0x55cf3bc4f740 2024-09-13T17:59:07.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.638+0000 7ff4ebfff640 10 client.5751 handle_command_reply: tid=0 2024-09-13T17:59:07.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.638+0000 7ff5162d5740 1 client.5751 shutdown 2024-09-13T17:59:07.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.638+0000 7ff5162d5740 1 -- 172.21.15.111:0/3668823193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 msgr2=0x55cf3bc31a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:07.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.638+0000 7ff5162d5740 1 --2- 172.21.15.111:0/3668823193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bc31a20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff504009e80 tx=0x7ff504004ab0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.638+0000 7ff5162d5740 1 -- 172.21.15.111:0/3668823193 shutdown_connections 2024-09-13T17:59:07.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.638+0000 7ff5162d5740 1 --2- 172.21.15.111:0/3668823193 >> [v2:172.21.15.111:6834/214034224,v1:172.21.15.111:6835/214034224] conn(0x55cf3bc4f740 0x55cf3bc4fb20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.638+0000 7ff5162d5740 1 --2- 172.21.15.111:0/3668823193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf3bc31550 0x55cf3bc31a20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.638+0000 7ff5162d5740 1 --2- 172.21.15.111:0/3668823193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf3bc2fe60 0x55cf3bcae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.638+0000 7ff5162d5740 1 --2- 172.21.15.111:0/3668823193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf3bc0a0a0 0x55cf3bc62af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.638+0000 7ff4ebfff640 0 client.5751 ms_handle_reset on v2:172.21.15.111:6834/214034224 2024-09-13T17:59:07.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.638+0000 7ff5162d5740 1 -- 172.21.15.111:0/3668823193 >> 172.21.15.111:0/3668823193 conn(0x55cf3bc07860 msgr2=0x55cf3bc06000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:07.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.638+0000 7ff5162d5740 1 -- 172.21.15.111:0/3668823193 shutdown_connections 2024-09-13T17:59:07.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.638+0000 7ff5162d5740 1 -- 172.21.15.111:0/3668823193 wait complete. 2024-09-13T17:59:07.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.638+0000 7ff5162d5740 20 client.5751 trim_cache size 0 max 16384 2024-09-13T17:59:07.647 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T17:59:07.647 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "2a8c13ef-d427-4b48-90b3-65459ec79a47", 2024-09-13T17:59:07.647 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2024-09-13T17:59:07.647 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 5487, 2024-09-13T17:59:07.647 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-09-13T17:59:07.647 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-09-13T17:59:07.647 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-09-13T17:59:07.647 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 6.0881480110000004, 2024-09-13T17:59:07.647 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 21, 2024-09-13T17:59:07.648 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 41, 2024-09-13T17:59:07.648 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 40, 2024-09-13T17:59:07.648 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 30.608154978000002 2024-09-13T17:59:07.648 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T17:59:07.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.639+0000 7ff5135b5640 1 -- 172.21.15.111:0/2891277867 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff4f006ac30 msgr2=0x7ff4f006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:07.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.639+0000 7ff5135b5640 1 --2- 172.21.15.111:0/2891277867 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff4f006ac30 0x7ff4f006d0f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff50800a750 tx=0x7ff508008040 comp rx=0 tx=0).stop 2024-09-13T17:59:07.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.639+0000 7ff5135b5640 1 -- 172.21.15.111:0/2891277867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff50c0d07b0 msgr2=0x7ff50c0c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:07.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.639+0000 7ff5135b5640 1 --2- 172.21.15.111:0/2891277867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff50c0d07b0 0x7ff50c0c98b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00c4d0 tx=0x7ff4fc00c990 comp rx=0 tx=0).stop 2024-09-13T17:59:07.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.639+0000 7ff5135b5640 1 -- 172.21.15.111:0/2891277867 shutdown_connections 2024-09-13T17:59:07.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.639+0000 7ff5135b5640 1 --2- 172.21.15.111:0/2891277867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff50c15c6e0 0x7ff50c0c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.639+0000 7ff5135b5640 1 --2- 172.21.15.111:0/2891277867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff50c0d07b0 0x7ff50c0c98b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.639+0000 7ff5135b5640 1 --2- 172.21.15.111:0/2891277867 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff4f006ac30 0x7ff4f006d0f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.639+0000 7ff5135b5640 1 --2- 172.21.15.111:0/2891277867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff50c0b4660 0x7ff50c0c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.639+0000 7ff5135b5640 1 -- 172.21.15.111:0/2891277867 >> 172.21.15.111:0/2891277867 conn(0x7ff50c089f60 msgr2=0x7ff50c0af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:07.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.639+0000 7ff5135b5640 1 -- 172.21.15.111:0/2891277867 shutdown_connections 2024-09-13T17:59:07.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.639+0000 7ff5135b5640 1 -- 172.21.15.111:0/2891277867 wait complete. 2024-09-13T17:59:07.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:59:07.651 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%a /home/ubuntu/cephtest/mnt.0/testdir/testfile) 2024-09-13T17:59:07.668 INFO:teuthology.orchestra.run.smithi111.stdout:600 2024-09-13T17:59:07.668 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T17:59:07.669 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T17:59:07.749 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T17:59:07.790 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T17:59:07.790 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:59:07.791 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:59:07.807 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:59:07.874 INFO:tasks.ceph.mds.c.smithi111.stderr:2024-09-13T17:59:07.873+0000 7f26fbac7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:59:07.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.941+0000 7f5cd0511640 1 Processor -- start 2024-09-13T17:59:07.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.941+0000 7f5cd0511640 1 -- start start 2024-09-13T17:59:07.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.942+0000 7f5cd0511640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5cc8072110 0x7f5cc8070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.942+0000 7f5cd0511640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5cc80c8920 0x7f5cc80c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.942+0000 7f5cd0511640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc80abdf0 con 0x7f5cc8071d30 2024-09-13T17:59:07.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.942+0000 7f5cd0511640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc80c92f0 con 0x7f5cc80c8920 2024-09-13T17:59:07.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.942+0000 7f5cd0511640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc80b2280 con 0x7f5cc8072110 2024-09-13T17:59:07.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.942+0000 7f5cceaae640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5cc80c8920 0x7f5cc80c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.942+0000 7f5cceaae640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5cc80c8920 0x7f5cc80c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52412/0 (socket says 172.21.15.111:52412) 2024-09-13T17:59:07.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.942+0000 7f5cceaae640 1 -- 172.21.15.111:0/3693951512 learned_addr learned my addr 172.21.15.111:0/3693951512 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:07.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.942+0000 7f5ccdaac640 1 --2- 172.21.15.111:0/3693951512 >> v2:172.21.15.188:3300/0 conn(0x7f5cc8072110 0x7f5cc8070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.942+0000 7f5cceaae640 1 -- 172.21.15.111:0/3693951512 >> v2:172.21.15.188:3300/0 conn(0x7f5cc8072110 msgr2=0x7f5cc8070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:07.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.942+0000 7f5cceaae640 1 --2- 172.21.15.111:0/3693951512 >> v2:172.21.15.188:3300/0 conn(0x7f5cc8072110 0x7f5cc8070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.942+0000 7f5cceaae640 1 -- 172.21.15.111:0/3693951512 >> v1:172.21.15.188:6789/0 conn(0x7f5cc8071d30 legacy=0x7f5cc80b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:07.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.942+0000 7f5cceaae640 1 -- 172.21.15.111:0/3693951512 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc80ab1e0 con 0x7f5cc80c8920 2024-09-13T17:59:07.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.943+0000 7f5cceaae640 1 --2- 172.21.15.111:0/3693951512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cc80c8920 0x7f5cc80c8db0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80cf000 tx=0x7f5cc002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c10c0242f2ca3ae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:07.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.943+0000 7f5ccd2ab640 1 -- 172.21.15.111:0/3693951512 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0031600 con 0x7f5cc80c8920 2024-09-13T17:59:07.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.943+0000 7f5ccd2ab640 1 -- 172.21.15.111:0/3693951512 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc002d050 con 0x7f5cc80c8920 2024-09-13T17:59:07.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.943+0000 7f5ccd2ab640 1 -- 172.21.15.111:0/3693951512 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0031980 con 0x7f5cc80c8920 2024-09-13T17:59:07.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.944+0000 7f5cd0511640 1 -- 172.21.15.111:0/3693951512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cc80c8920 msgr2=0x7f5cc80c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:07.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.944+0000 7f5cd0511640 1 --2- 172.21.15.111:0/3693951512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cc80c8920 0x7f5cc80c8db0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80cf000 tx=0x7f5cc002a730 comp rx=0 tx=0).stop 2024-09-13T17:59:07.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.944+0000 7f5cd0511640 1 -- 172.21.15.111:0/3693951512 shutdown_connections 2024-09-13T17:59:07.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.944+0000 7f5cd0511640 1 --2- 172.21.15.111:0/3693951512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cc80c8920 0x7f5cc80c8db0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.944+0000 7f5cd0511640 1 --2- 172.21.15.111:0/3693951512 >> v2:172.21.15.188:3300/0 conn(0x7f5cc8072110 0x7f5cc8070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.944+0000 7f5cd0511640 1 -- 172.21.15.111:0/3693951512 >> 172.21.15.111:0/3693951512 conn(0x7f5cc8069ed0 msgr2=0x7f5cc806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:07.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.944+0000 7f5cd0511640 1 -- 172.21.15.111:0/3693951512 shutdown_connections 2024-09-13T17:59:07.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.944+0000 7f5cd0511640 1 -- 172.21.15.111:0/3693951512 wait complete. 2024-09-13T17:59:07.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.945+0000 7f5cd0511640 1 Processor -- start 2024-09-13T17:59:07.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.945+0000 7f5cd0511640 1 -- start start 2024-09-13T17:59:07.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5cd0511640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cc8071d30 0x7f5cc80acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5cd0511640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cc8072110 0x7f5cc81f2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5cd0511640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc80c8920 0x7f5cc80cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5cd0511640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc80c51f0 con 0x7f5cc8071d30 2024-09-13T17:59:07.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5cd0511640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc81f27c0 con 0x7f5cc80c8920 2024-09-13T17:59:07.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5cd0511640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc80c4cc0 con 0x7f5cc8072110 2024-09-13T17:59:07.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5cce2ad640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cc8071d30 0x7f5cc80acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5cce2ad640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cc8071d30 0x7f5cc80acdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52424/0 (socket says 172.21.15.111:52424) 2024-09-13T17:59:07.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5cce2ad640 1 -- 172.21.15.111:0/709370292 learned_addr learned my addr 172.21.15.111:0/709370292 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:07.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5ccdaac640 1 --2- 172.21.15.111:0/709370292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cc8072110 0x7f5cc81f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5cceaae640 1 --2- 172.21.15.111:0/709370292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc80c8920 0x7f5cc80cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5cce2ad640 1 -- 172.21.15.111:0/709370292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cc8072110 msgr2=0x7f5cc81f2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:07.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5cce2ad640 1 --2- 172.21.15.111:0/709370292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cc8072110 0x7f5cc81f2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5cce2ad640 1 -- 172.21.15.111:0/709370292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc80c8920 msgr2=0x7f5cc80cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:07.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5cce2ad640 1 --2- 172.21.15.111:0/709370292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc80c8920 0x7f5cc80cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:07.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5cce2ad640 1 -- 172.21.15.111:0/709370292 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc0009b30 con 0x7f5cc8071d30 2024-09-13T17:59:07.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5ccdaac640 1 --2- 172.21.15.111:0/709370292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cc8072110 0x7f5cc81f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:07.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.946+0000 7f5cce2ad640 1 --2- 172.21.15.111:0/709370292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cc8071d30 0x7f5cc80acdc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400c480 tx=0x7f5cc400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:07.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.947+0000 7f5cbeffd640 1 -- 172.21.15.111:0/709370292 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc40052d0 con 0x7f5cc8071d30 2024-09-13T17:59:07.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.947+0000 7f5cbeffd640 1 -- 172.21.15.111:0/709370292 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc4005430 con 0x7f5cc8071d30 2024-09-13T17:59:07.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.947+0000 7f5cbeffd640 1 -- 172.21.15.111:0/709370292 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4012870 con 0x7f5cc8071d30 2024-09-13T17:59:07.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.947+0000 7f5cd0511640 1 -- 172.21.15.111:0/709370292 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc80caf80 con 0x7f5cc8071d30 2024-09-13T17:59:07.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.947+0000 7f5cd0511640 1 -- 172.21.15.111:0/709370292 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc806df10 con 0x7f5cc8071d30 2024-09-13T17:59:07.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.948+0000 7f5cd0511640 1 -- 172.21.15.111:0/709370292 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc8070190 con 0x7f5cc8071d30 2024-09-13T17:59:07.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.948+0000 7f5cbeffd640 1 -- 172.21.15.111:0/709370292 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5cc40129d0 con 0x7f5cc8071d30 2024-09-13T17:59:07.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.948+0000 7f5cbeffd640 1 --2- 172.21.15.111:0/709370292 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5cac06ac30 0x7f5cac06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:07.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.949+0000 7f5cbeffd640 1 -- 172.21.15.111:0/709370292 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f5cc4088f60 con 0x7f5cc8071d30 2024-09-13T17:59:07.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.949+0000 7f5ccdaac640 1 --2- 172.21.15.111:0/709370292 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5cac06ac30 0x7f5cac06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:07.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.949+0000 7f5ccdaac640 1 --2- 172.21.15.111:0/709370292 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5cac06ac30 0x7f5cac06d0f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5cc81f5db0 tx=0x7f5cb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:07.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:07.953+0000 7f5cbeffd640 1 -- 172.21.15.111:0/709370292 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f5cc4054020 con 0x7f5cc8071d30 2024-09-13T17:59:08.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.115+0000 7f5cd0511640 1 -- 172.21.15.111:0/709370292 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cc80cad50 con 0x7f5cc8071d30 2024-09-13T17:59:08.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.116+0000 7f5cbeffd640 1 -- 172.21.15.111:0/709370292 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+17012 (secure 0 0 0) 0x7f5cc4057640 con 0x7f5cc8071d30 2024-09-13T17:59:08.119 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:08.119 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":41,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:59:07.612858+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:58:41.698998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T17:58:42.499870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1c","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6837/982359636":"2024-09-14T17:59:01.542287+0000","172.21.15.111:6836/982359636":"2024-09-14T17:59:01.542287+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:59:08.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.118+0000 7f5cd0511640 1 -- 172.21.15.111:0/709370292 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5cac06ac30 msgr2=0x7f5cac06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.118+0000 7f5cd0511640 1 --2- 172.21.15.111:0/709370292 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5cac06ac30 0x7f5cac06d0f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5cc81f5db0 tx=0x7f5cb8008040 comp rx=0 tx=0).stop 2024-09-13T17:59:08.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.118+0000 7f5cd0511640 1 -- 172.21.15.111:0/709370292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cc8071d30 msgr2=0x7f5cc80acdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.118+0000 7f5cd0511640 1 --2- 172.21.15.111:0/709370292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cc8071d30 0x7f5cc80acdc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400c480 tx=0x7f5cc400c940 comp rx=0 tx=0).stop 2024-09-13T17:59:08.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.118+0000 7f5cd0511640 1 -- 172.21.15.111:0/709370292 shutdown_connections 2024-09-13T17:59:08.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.118+0000 7f5cd0511640 1 --2- 172.21.15.111:0/709370292 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5cac06ac30 0x7f5cac06d0f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.118+0000 7f5cd0511640 1 --2- 172.21.15.111:0/709370292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc80c8920 0x7f5cc80cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.118+0000 7f5cd0511640 1 --2- 172.21.15.111:0/709370292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cc8072110 0x7f5cc81f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.118+0000 7f5cd0511640 1 --2- 172.21.15.111:0/709370292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cc8071d30 0x7f5cc80acdc0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.118+0000 7f5cd0511640 1 -- 172.21.15.111:0/709370292 >> 172.21.15.111:0/709370292 conn(0x7f5cc8069ed0 msgr2=0x7f5cc80b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:08.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.118+0000 7f5cd0511640 1 -- 172.21.15.111:0/709370292 shutdown_connections 2024-09-13T17:59:08.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.119+0000 7f5cd0511640 1 -- 172.21.15.111:0/709370292 wait complete. 2024-09-13T17:59:08.132 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:58:41.698998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:58:42.499870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:59:08.132 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:59:08.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.270+0000 7fdb519ff640 1 Processor -- start 2024-09-13T17:59:08.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.270+0000 7fdb519ff640 1 -- start start 2024-09-13T17:59:08.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.271+0000 7fdb519ff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c143be0 0x7fdb4c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:08.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.271+0000 7fdb519ff640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4c06e9f0 con 0x7fdb4c06a5e0 2024-09-13T17:59:08.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.271+0000 7fdb519ff640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4c06ecd0 con 0x7fdb4c0ce700 2024-09-13T17:59:08.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.271+0000 7fdb519ff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c06efb0 con 0x7fdb4c143be0 2024-09-13T17:59:08.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.271+0000 7fdb4b7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fdb4c06a5e0 0x7fdb4c15a5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33536/0 (socket says 172.21.15.111:33536) 2024-09-13T17:59:08.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.271+0000 7fdb4b7fe640 1 -- 172.21.15.111:0/176902793 learned_addr learned my addr 172.21.15.111:0/176902793 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:08.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.271+0000 7fdb4a7fc640 1 --1- 172.21.15.111:0/176902793 >> v1:172.21.15.188:6789/0 conn(0x7fdb4c0ce700 0x7fdb4c069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.111:38476/0 (socket says 172.21.15.111:38476) 2024-09-13T17:59:08.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.271+0000 7fdb4affd640 1 --2- 172.21.15.111:0/176902793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c143be0 0x7fdb4c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:08.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.271+0000 7fdb49ffb640 1 -- 172.21.15.111:0/176902793 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027377787 0 0) 0x7fdb4c06e9f0 con 0x7fdb4c06a5e0 2024-09-13T17:59:08.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.271+0000 7fdb49ffb640 1 -- 172.21.15.111:0/176902793 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb38003520 con 0x7fdb4c06a5e0 2024-09-13T17:59:08.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.271+0000 7fdb49ffb640 1 -- 172.21.15.111:0/176902793 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474881906 0 0) 0x7fdb4c06ecd0 con 0x7fdb4c0ce700 2024-09-13T17:59:08.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.271+0000 7fdb49ffb640 1 -- 172.21.15.111:0/176902793 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb4c06e9f0 con 0x7fdb4c0ce700 2024-09-13T17:59:08.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.271+0000 7fdb4affd640 1 -- 172.21.15.111:0/176902793 >> v1:172.21.15.188:6789/0 conn(0x7fdb4c0ce700 legacy=0x7fdb4c069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.271+0000 7fdb4affd640 1 -- 172.21.15.111:0/176902793 >> v1:172.21.15.111:6789/0 conn(0x7fdb4c06a5e0 legacy=0x7fdb4c15a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.271+0000 7fdb4affd640 1 -- 172.21.15.111:0/176902793 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c06f290 con 0x7fdb4c143be0 2024-09-13T17:59:08.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.272+0000 7fdb4affd640 1 --2- 172.21.15.111:0/176902793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c143be0 0x7fdb4c06e410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009dc0 tx=0x7fdb3c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a5a0035ec21775b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:08.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.272+0000 7fdb49ffb640 1 -- 172.21.15.111:0/176902793 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c007740 con 0x7fdb4c143be0 2024-09-13T17:59:08.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.272+0000 7fdb49ffb640 1 -- 172.21.15.111:0/176902793 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c02d050 con 0x7fdb4c143be0 2024-09-13T17:59:08.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.272+0000 7fdb49ffb640 1 -- 172.21.15.111:0/176902793 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c007aa0 con 0x7fdb4c143be0 2024-09-13T17:59:08.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.273+0000 7fdb519ff640 1 -- 172.21.15.111:0/176902793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c143be0 msgr2=0x7fdb4c06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.273+0000 7fdb519ff640 1 --2- 172.21.15.111:0/176902793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c143be0 0x7fdb4c06e410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009dc0 tx=0x7fdb3c02a6d0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.273+0000 7fdb519ff640 1 -- 172.21.15.111:0/176902793 shutdown_connections 2024-09-13T17:59:08.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.273+0000 7fdb519ff640 1 --2- 172.21.15.111:0/176902793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c143be0 0x7fdb4c06e410 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.273+0000 7fdb519ff640 1 -- 172.21.15.111:0/176902793 >> 172.21.15.111:0/176902793 conn(0x7fdb4c0b0db0 msgr2=0x7fdb4c0b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:08.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.273+0000 7fdb519ff640 1 -- 172.21.15.111:0/176902793 shutdown_connections 2024-09-13T17:59:08.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.273+0000 7fdb519ff640 1 -- 172.21.15.111:0/176902793 wait complete. 2024-09-13T17:59:08.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.274+0000 7fdb519ff640 1 Processor -- start 2024-09-13T17:59:08.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.274+0000 7fdb519ff640 1 -- start start 2024-09-13T17:59:08.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb519ff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb4c06a5e0 0x7fdb4c1597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:08.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb519ff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c0ce700 0x7fdb4c1f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:08.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb519ff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb4c143be0 0x7fdb4c0ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:08.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb519ff640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c159d10 con 0x7fdb4c06a5e0 2024-09-13T17:59:08.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb519ff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c13fd60 con 0x7fdb4c143be0 2024-09-13T17:59:08.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb519ff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c1400d0 con 0x7fdb4c0ce700 2024-09-13T17:59:08.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb4affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb4c06a5e0 0x7fdb4c1597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:08.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb4affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb4c06a5e0 0x7fdb4c1597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52440/0 (socket says 172.21.15.111:52440) 2024-09-13T17:59:08.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb4affd640 1 -- 172.21.15.111:0/2413642038 learned_addr learned my addr 172.21.15.111:0/2413642038 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:08.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb4affd640 1 -- 172.21.15.111:0/2413642038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c0ce700 msgr2=0x7fdb4c1f00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb4b7fe640 1 --2- 172.21.15.111:0/2413642038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb4c143be0 0x7fdb4c0ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:08.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb4a7fc640 1 --2- 172.21.15.111:0/2413642038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c0ce700 0x7fdb4c1f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:08.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb4affd640 1 --2- 172.21.15.111:0/2413642038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c0ce700 0x7fdb4c1f00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb4affd640 1 -- 172.21.15.111:0/2413642038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb4c143be0 msgr2=0x7fdb4c0ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb4affd640 1 --2- 172.21.15.111:0/2413642038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb4c143be0 0x7fdb4c0ada00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb4affd640 1 -- 172.21.15.111:0/2413642038 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3c009a70 con 0x7fdb4c06a5e0 2024-09-13T17:59:08.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb4a7fc640 1 --2- 172.21.15.111:0/2413642038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c0ce700 0x7fdb4c1f00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:59:08.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.275+0000 7fdb4b7fe640 1 --2- 172.21.15.111:0/2413642038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb4c143be0 0x7fdb4c0ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:59:08.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.276+0000 7fdb4affd640 1 --2- 172.21.15.111:0/2413642038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb4c06a5e0 0x7fdb4c1597d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c000f80 tx=0x7fdb3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:08.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.276+0000 7fdb43fff640 1 -- 172.21.15.111:0/2413642038 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c03b820 con 0x7fdb4c06a5e0 2024-09-13T17:59:08.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.276+0000 7fdb43fff640 1 -- 172.21.15.111:0/2413642038 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c03b980 con 0x7fdb4c06a5e0 2024-09-13T17:59:08.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.276+0000 7fdb519ff640 1 -- 172.21.15.111:0/2413642038 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4c0ced50 con 0x7fdb4c06a5e0 2024-09-13T17:59:08.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.276+0000 7fdb43fff640 1 -- 172.21.15.111:0/2413642038 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c03bcf0 con 0x7fdb4c06a5e0 2024-09-13T17:59:08.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.276+0000 7fdb519ff640 1 -- 172.21.15.111:0/2413642038 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4c140690 con 0x7fdb4c06a5e0 2024-09-13T17:59:08.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.277+0000 7fdb519ff640 1 -- 172.21.15.111:0/2413642038 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4c06ecd0 con 0x7fdb4c06a5e0 2024-09-13T17:59:08.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.277+0000 7fdb43fff640 1 -- 172.21.15.111:0/2413642038 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdb3c043410 con 0x7fdb4c06a5e0 2024-09-13T17:59:08.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.278+0000 7fdb43fff640 1 --2- 172.21.15.111:0/2413642038 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdb2c06ac00 0x7fdb2c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:08.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.278+0000 7fdb43fff640 1 -- 172.21.15.111:0/2413642038 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fdb3c0aa980 con 0x7fdb4c06a5e0 2024-09-13T17:59:08.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.278+0000 7fdb4a7fc640 1 --2- 172.21.15.111:0/2413642038 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdb2c06ac00 0x7fdb2c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:08.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.278+0000 7fdb4a7fc640 1 --2- 172.21.15.111:0/2413642038 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdb2c06ac00 0x7fdb2c06d0c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c1e2400 tx=0x7fdb34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:08.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.282+0000 7fdb43fff640 1 -- 172.21.15.111:0/2413642038 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fdb3c0759c0 con 0x7fdb4c06a5e0 2024-09-13T17:59:08.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.442+0000 7fdb519ff640 1 -- 172.21.15.111:0/2413642038 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdb4c1e07f0 con 0x7fdb4c06a5e0 2024-09-13T17:59:08.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.442+0000 7fdb43fff640 1 -- 172.21.15.111:0/2413642038 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb3c078fe0 con 0x7fdb4c06a5e0 2024-09-13T17:59:08.444 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:59:08.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.444+0000 7fdb519ff640 1 -- 172.21.15.111:0/2413642038 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdb2c06ac00 msgr2=0x7fdb2c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.444+0000 7fdb519ff640 1 --2- 172.21.15.111:0/2413642038 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdb2c06ac00 0x7fdb2c06d0c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c1e2400 tx=0x7fdb34009040 comp rx=0 tx=0).stop 2024-09-13T17:59:08.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.444+0000 7fdb519ff640 1 -- 172.21.15.111:0/2413642038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb4c06a5e0 msgr2=0x7fdb4c1597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.444+0000 7fdb519ff640 1 --2- 172.21.15.111:0/2413642038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb4c06a5e0 0x7fdb4c1597d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c000f80 tx=0x7fdb3c004a40 comp rx=0 tx=0).stop 2024-09-13T17:59:08.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.444+0000 7fdb519ff640 1 -- 172.21.15.111:0/2413642038 shutdown_connections 2024-09-13T17:59:08.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.444+0000 7fdb519ff640 1 --2- 172.21.15.111:0/2413642038 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdb2c06ac00 0x7fdb2c06d0c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.444+0000 7fdb519ff640 1 --2- 172.21.15.111:0/2413642038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb4c143be0 0x7fdb4c0ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.444+0000 7fdb519ff640 1 --2- 172.21.15.111:0/2413642038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c0ce700 0x7fdb4c1f00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.444+0000 7fdb519ff640 1 --2- 172.21.15.111:0/2413642038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb4c06a5e0 0x7fdb4c1597d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.444+0000 7fdb519ff640 1 -- 172.21.15.111:0/2413642038 >> 172.21.15.111:0/2413642038 conn(0x7fdb4c0b0db0 msgr2=0x7fdb4c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:08.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.444+0000 7fdb519ff640 1 -- 172.21.15.111:0/2413642038 shutdown_connections 2024-09-13T17:59:08.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.444+0000 7fdb519ff640 1 -- 172.21.15.111:0/2413642038 wait complete. 2024-09-13T17:59:08.457 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:59:08.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.555+0000 7fdf4ffce640 1 Processor -- start 2024-09-13T17:59:08.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.555+0000 7fdf4ffce640 1 -- start start 2024-09-13T17:59:08.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.556+0000 7fdf4ffce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf48159310 0x7fdf480ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:08.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.556+0000 7fdf4ffce640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdf480ce4f0 0x7fdf480ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:08.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.556+0000 7fdf4ffce640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf480ccbe0 con 0x7fdf480b2b40 2024-09-13T17:59:08.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.556+0000 7fdf4ffce640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf480ad940 con 0x7fdf480ce4f0 2024-09-13T17:59:08.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.556+0000 7fdf4ffce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480cee80 con 0x7fdf48159310 2024-09-13T17:59:08.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.556+0000 7fdf4e56b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdf480ce4f0 0x7fdf480ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:08.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.556+0000 7fdf4e56b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdf480ce4f0 0x7fdf480ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52456/0 (socket says 172.21.15.111:52456) 2024-09-13T17:59:08.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.556+0000 7fdf4d569640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fdf480b2b40 0x7fdf480ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33544/0 (socket says 172.21.15.111:33544) 2024-09-13T17:59:08.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.556+0000 7fdf4e56b640 1 -- 172.21.15.111:0/4185553366 learned_addr learned my addr 172.21.15.111:0/4185553366 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:08.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.556+0000 7fdf4dd6a640 1 --2- 172.21.15.111:0/4185553366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf48159310 0x7fdf480ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:08.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.556+0000 7fdf4e56b640 1 -- 172.21.15.111:0/4185553366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf48159310 msgr2=0x7fdf480ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:08.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.556+0000 7fdf4e56b640 1 --2- 172.21.15.111:0/4185553366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf48159310 0x7fdf480ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.556+0000 7fdf4e56b640 1 -- 172.21.15.111:0/4185553366 >> v1:172.21.15.111:6789/0 conn(0x7fdf480b2b40 legacy=0x7fdf480ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.556+0000 7fdf4e56b640 1 -- 172.21.15.111:0/4185553366 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf480b1810 con 0x7fdf480ce4f0 2024-09-13T17:59:08.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.556+0000 7fdf4cd68640 1 -- 172.21.15.111:0/4185553366 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715611034 0 0) 0x7fdf480ccbe0 con 0x7fdf480b2b40 2024-09-13T17:59:08.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.557+0000 7fdf4e56b640 1 --2- 172.21.15.111:0/4185553366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf480ce4f0 0x7fdf480ce940 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000d150 tx=0x7fdf4002c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55883e0b716933 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:08.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.557+0000 7fdf4cd68640 1 -- 172.21.15.111:0/4185553366 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40009600 con 0x7fdf480ce4f0 2024-09-13T17:59:08.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.557+0000 7fdf4cd68640 1 -- 172.21.15.111:0/4185553366 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf40009760 con 0x7fdf480ce4f0 2024-09-13T17:59:08.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.557+0000 7fdf4cd68640 1 -- 172.21.15.111:0/4185553366 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40009af0 con 0x7fdf480ce4f0 2024-09-13T17:59:08.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.557+0000 7fdf4ffce640 1 -- 172.21.15.111:0/4185553366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf480ce4f0 msgr2=0x7fdf480ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.557+0000 7fdf4ffce640 1 --2- 172.21.15.111:0/4185553366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf480ce4f0 0x7fdf480ce940 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000d150 tx=0x7fdf4002c890 comp rx=0 tx=0).stop 2024-09-13T17:59:08.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.558+0000 7fdf4ffce640 1 -- 172.21.15.111:0/4185553366 shutdown_connections 2024-09-13T17:59:08.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.558+0000 7fdf4ffce640 1 --2- 172.21.15.111:0/4185553366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf480ce4f0 0x7fdf480ce940 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.558+0000 7fdf4ffce640 1 --2- 172.21.15.111:0/4185553366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf48159310 0x7fdf480ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.558+0000 7fdf4ffce640 1 -- 172.21.15.111:0/4185553366 >> 172.21.15.111:0/4185553366 conn(0x7fdf4806a0b0 msgr2=0x7fdf4806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:08.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.558+0000 7fdf4ffce640 1 -- 172.21.15.111:0/4185553366 shutdown_connections 2024-09-13T17:59:08.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.558+0000 7fdf4ffce640 1 -- 172.21.15.111:0/4185553366 wait complete. 2024-09-13T17:59:08.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.559+0000 7fdf4ffce640 1 Processor -- start 2024-09-13T17:59:08.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.559+0000 7fdf4ffce640 1 -- start start 2024-09-13T17:59:08.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.559+0000 7fdf4ffce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf480b2b40 0x7fdf481f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:08.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.559+0000 7fdf4ffce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf480ce4f0 0x7fdf48152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:08.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4ffce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf48159310 0x7fdf481e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:08.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4ffce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf481f21a0 con 0x7fdf48159310 2024-09-13T17:59:08.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4ffce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf48155190 con 0x7fdf480ce4f0 2024-09-13T17:59:08.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4ffce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf48152ea0 con 0x7fdf480b2b40 2024-09-13T17:59:08.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4e56b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf48159310 0x7fdf481e28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:08.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4e56b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf48159310 0x7fdf481e28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52470/0 (socket says 172.21.15.111:52470) 2024-09-13T17:59:08.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4e56b640 1 -- 172.21.15.111:0/841166298 learned_addr learned my addr 172.21.15.111:0/841166298 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:08.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4dd6a640 1 --2- 172.21.15.111:0/841166298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf480b2b40 0x7fdf481f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:08.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4e56b640 1 -- 172.21.15.111:0/841166298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf480b2b40 msgr2=0x7fdf481f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4d569640 1 --2- 172.21.15.111:0/841166298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf480ce4f0 0x7fdf48152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:08.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4e56b640 1 --2- 172.21.15.111:0/841166298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf480b2b40 0x7fdf481f1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4e56b640 1 -- 172.21.15.111:0/841166298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf480ce4f0 msgr2=0x7fdf48152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4e56b640 1 --2- 172.21.15.111:0/841166298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf480ce4f0 0x7fdf48152530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4e56b640 1 -- 172.21.15.111:0/841166298 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4000bd40 con 0x7fdf48159310 2024-09-13T17:59:08.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4d569640 1 --2- 172.21.15.111:0/841166298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf480ce4f0 0x7fdf48152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:59:08.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4e56b640 1 --2- 172.21.15.111:0/841166298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf48159310 0x7fdf481e28a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdf40037040 tx=0x7fdf40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:08.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf3e7fc640 1 -- 172.21.15.111:0/841166298 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40034680 con 0x7fdf48159310 2024-09-13T17:59:08.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf3e7fc640 1 -- 172.21.15.111:0/841166298 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf40009e90 con 0x7fdf48159310 2024-09-13T17:59:08.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf3e7fc640 1 -- 172.21.15.111:0/841166298 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40034a60 con 0x7fdf48159310 2024-09-13T17:59:08.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4ffce640 1 -- 172.21.15.111:0/841166298 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf48153180 con 0x7fdf48159310 2024-09-13T17:59:08.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.560+0000 7fdf4ffce640 1 -- 172.21.15.111:0/841166298 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf48153740 con 0x7fdf48159310 2024-09-13T17:59:08.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.561+0000 7fdf4ffce640 1 -- 172.21.15.111:0/841166298 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4806dfa0 con 0x7fdf48159310 2024-09-13T17:59:08.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.562+0000 7fdf3e7fc640 1 -- 172.21.15.111:0/841166298 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdf4003b070 con 0x7fdf48159310 2024-09-13T17:59:08.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.562+0000 7fdf3e7fc640 1 --2- 172.21.15.111:0/841166298 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdf2c06ac00 0x7fdf2c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:08.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.562+0000 7fdf4dd6a640 1 --2- 172.21.15.111:0/841166298 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdf2c06ac00 0x7fdf2c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:08.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.562+0000 7fdf3e7fc640 1 -- 172.21.15.111:0/841166298 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fdf400aa8e0 con 0x7fdf48159310 2024-09-13T17:59:08.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.563+0000 7fdf4dd6a640 1 --2- 172.21.15.111:0/841166298 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdf2c06ac00 0x7fdf2c06d0c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400b7c0 tx=0x7fdf44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:08.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.567+0000 7fdf3e7fc640 1 -- 172.21.15.111:0/841166298 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fdf400793a0 con 0x7fdf48159310 2024-09-13T17:59:08.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.729+0000 7fdf4ffce640 1 -- 172.21.15.111:0/841166298 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf4806e4b0 con 0x7fdf48159310 2024-09-13T17:59:08.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.729+0000 7fdf3e7fc640 1 -- 172.21.15.111:0/841166298 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdf400390a0 con 0x7fdf48159310 2024-09-13T17:59:08.732 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:59:08.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.731+0000 7fdf4ffce640 1 -- 172.21.15.111:0/841166298 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdf2c06ac00 msgr2=0x7fdf2c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.731+0000 7fdf4ffce640 1 --2- 172.21.15.111:0/841166298 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdf2c06ac00 0x7fdf2c06d0c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400b7c0 tx=0x7fdf44008040 comp rx=0 tx=0).stop 2024-09-13T17:59:08.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.732+0000 7fdf4ffce640 1 -- 172.21.15.111:0/841166298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf48159310 msgr2=0x7fdf481e28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.732+0000 7fdf4ffce640 1 --2- 172.21.15.111:0/841166298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf48159310 0x7fdf481e28a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdf40037040 tx=0x7fdf40004a40 comp rx=0 tx=0).stop 2024-09-13T17:59:08.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.732+0000 7fdf4ffce640 1 -- 172.21.15.111:0/841166298 shutdown_connections 2024-09-13T17:59:08.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.732+0000 7fdf4ffce640 1 --2- 172.21.15.111:0/841166298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf48159310 0x7fdf481e28a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.732+0000 7fdf4ffce640 1 --2- 172.21.15.111:0/841166298 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fdf2c06ac00 0x7fdf2c06d0c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.732+0000 7fdf4ffce640 1 --2- 172.21.15.111:0/841166298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf480ce4f0 0x7fdf48152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.732+0000 7fdf4ffce640 1 --2- 172.21.15.111:0/841166298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf480b2b40 0x7fdf481f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.732+0000 7fdf4ffce640 1 -- 172.21.15.111:0/841166298 >> 172.21.15.111:0/841166298 conn(0x7fdf4806a0b0 msgr2=0x7fdf480b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:08.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.732+0000 7fdf4ffce640 1 -- 172.21.15.111:0/841166298 shutdown_connections 2024-09-13T17:59:08.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.732+0000 7fdf4ffce640 1 -- 172.21.15.111:0/841166298 wait complete. 2024-09-13T17:59:08.744 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:59:08.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.842+0000 7f9e9d365640 1 Processor -- start 2024-09-13T17:59:08.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.842+0000 7f9e9d365640 1 -- start start 2024-09-13T17:59:08.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.843+0000 7f9e9d365640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9e980cdf10 0x7f9e98069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:08.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.843+0000 7f9e9d365640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9e980ce2f0 0x7f9e9806ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:08.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.843+0000 7f9e9d365640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e9815c9a0 con 0x7f9e9806b600 2024-09-13T17:59:08.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.843+0000 7f9e9d365640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e9806b170 con 0x7f9e980ce2f0 2024-09-13T17:59:08.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.843+0000 7f9e9d365640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e9806a560 con 0x7f9e980cdf10 2024-09-13T17:59:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.843+0000 7f9e977fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9e980ce2f0 0x7f9e9806ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.843+0000 7f9e977fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9e980ce2f0 0x7f9e9806ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52486/0 (socket says 172.21.15.111:52486) 2024-09-13T17:59:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.843+0000 7f9e977fe640 1 -- 172.21.15.111:0/2369741745 learned_addr learned my addr 172.21.15.111:0/2369741745 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.843+0000 7f9e967fc640 1 --1- 172.21.15.111:0/2369741745 >> v1:172.21.15.111:6789/0 conn(0x7f9e9806b600 0x7f9e98065c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33546/0 (socket says 172.21.15.111:33546) 2024-09-13T17:59:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.844+0000 7f9e977fe640 1 -- 172.21.15.111:0/2369741745 >> v2:172.21.15.188:3300/0 conn(0x7f9e980cdf10 msgr2=0x7f9e98069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.844+0000 7f9e977fe640 1 --2- 172.21.15.111:0/2369741745 >> v2:172.21.15.188:3300/0 conn(0x7f9e980cdf10 0x7f9e98069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.844+0000 7f9e977fe640 1 -- 172.21.15.111:0/2369741745 >> v1:172.21.15.111:6789/0 conn(0x7f9e9806b600 legacy=0x7f9e98065c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.844+0000 7f9e95ffb640 1 -- 172.21.15.111:0/2369741745 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3105120956 0 0) 0x7f9e9815c9a0 con 0x7f9e9806b600 2024-09-13T17:59:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.844+0000 7f9e977fe640 1 -- 172.21.15.111:0/2369741745 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9806a840 con 0x7f9e980ce2f0 2024-09-13T17:59:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.844+0000 7f9e977fe640 1 --2- 172.21.15.111:0/2369741745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e980ce2f0 0x7f9e9806ca30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009e50 tx=0x7f9e8802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0779f7bb9330e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.844+0000 7f9e95ffb640 1 -- 172.21.15.111:0/2369741745 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e88031600 con 0x7f9e980ce2f0 2024-09-13T17:59:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.844+0000 7f9e95ffb640 1 -- 172.21.15.111:0/2369741745 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8802d050 con 0x7f9e980ce2f0 2024-09-13T17:59:08.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.844+0000 7f9e95ffb640 1 -- 172.21.15.111:0/2369741745 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e88031980 con 0x7f9e980ce2f0 2024-09-13T17:59:08.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.845+0000 7f9e9d365640 1 -- 172.21.15.111:0/2369741745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e980ce2f0 msgr2=0x7f9e9806ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.845+0000 7f9e9d365640 1 --2- 172.21.15.111:0/2369741745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e980ce2f0 0x7f9e9806ca30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009e50 tx=0x7f9e8802a760 comp rx=0 tx=0).stop 2024-09-13T17:59:08.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.845+0000 7f9e9d365640 1 -- 172.21.15.111:0/2369741745 shutdown_connections 2024-09-13T17:59:08.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.845+0000 7f9e9d365640 1 --2- 172.21.15.111:0/2369741745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e980ce2f0 0x7f9e9806ca30 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.845+0000 7f9e9d365640 1 --2- 172.21.15.111:0/2369741745 >> v2:172.21.15.188:3300/0 conn(0x7f9e980cdf10 0x7f9e98069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.845+0000 7f9e9d365640 1 -- 172.21.15.111:0/2369741745 >> 172.21.15.111:0/2369741745 conn(0x7f9e980cb5b0 msgr2=0x7f9e980cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:08.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.845+0000 7f9e9d365640 1 -- 172.21.15.111:0/2369741745 shutdown_connections 2024-09-13T17:59:08.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.845+0000 7f9e9d365640 1 -- 172.21.15.111:0/2369741745 wait complete. 2024-09-13T17:59:08.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.846+0000 7f9e9d365640 1 Processor -- start 2024-09-13T17:59:08.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.846+0000 7f9e9d365640 1 -- start start 2024-09-13T17:59:08.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.847+0000 7f9e9d365640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e9806b600 0x7f9e980c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:08.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.847+0000 7f9e9d365640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e980cdf10 0x7f9e98157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:08.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.847+0000 7f9e9d365640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e980ce2f0 0x7f9e980b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:08.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.847+0000 7f9e9d365640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e980c6140 con 0x7f9e980cdf10 2024-09-13T17:59:08.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.847+0000 7f9e9d365640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e981e5e00 con 0x7f9e9806b600 2024-09-13T17:59:08.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.847+0000 7f9e9d365640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e981e60a0 con 0x7f9e980ce2f0 2024-09-13T17:59:08.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.847+0000 7f9e967fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e980cdf10 0x7f9e98157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:08.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.847+0000 7f9e967fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e980cdf10 0x7f9e98157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52498/0 (socket says 172.21.15.111:52498) 2024-09-13T17:59:08.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.847+0000 7f9e967fc640 1 -- 172.21.15.111:0/1433101207 learned_addr learned my addr 172.21.15.111:0/1433101207 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:08.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.847+0000 7f9e967fc640 1 -- 172.21.15.111:0/1433101207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e980ce2f0 msgr2=0x7f9e980b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.847+0000 7f9e96ffd640 1 --2- 172.21.15.111:0/1433101207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e9806b600 0x7f9e980c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:08.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.847+0000 7f9e967fc640 1 --2- 172.21.15.111:0/1433101207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e980ce2f0 0x7f9e980b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.847+0000 7f9e967fc640 1 -- 172.21.15.111:0/1433101207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e9806b600 msgr2=0x7f9e980c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:08.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.847+0000 7f9e967fc640 1 --2- 172.21.15.111:0/1433101207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e9806b600 0x7f9e980c5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:08.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.847+0000 7f9e967fc640 1 -- 172.21.15.111:0/1433101207 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e88009b30 con 0x7f9e980cdf10 2024-09-13T17:59:08.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.847+0000 7f9e96ffd640 1 --2- 172.21.15.111:0/1433101207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e9806b600 0x7f9e980c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:59:08.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.847+0000 7f9e967fc640 1 --2- 172.21.15.111:0/1433101207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e980cdf10 0x7f9e98157000 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000c4d0 tx=0x7f9e8000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:08.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.848+0000 7f9e8f7fe640 1 -- 172.21.15.111:0/1433101207 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e800074e0 con 0x7f9e980cdf10 2024-09-13T17:59:08.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.848+0000 7f9e8f7fe640 1 -- 172.21.15.111:0/1433101207 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e80007660 con 0x7f9e980cdf10 2024-09-13T17:59:08.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.848+0000 7f9e9d365640 1 -- 172.21.15.111:0/1433101207 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e981e6380 con 0x7f9e980cdf10 2024-09-13T17:59:08.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.848+0000 7f9e8f7fe640 1 -- 172.21.15.111:0/1433101207 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e80004a20 con 0x7f9e980cdf10 2024-09-13T17:59:08.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.848+0000 7f9e9d365640 1 -- 172.21.15.111:0/1433101207 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e980b4960 con 0x7f9e980cdf10 2024-09-13T17:59:08.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.849+0000 7f9e9d365640 1 -- 172.21.15.111:0/1433101207 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e981e68a0 con 0x7f9e980cdf10 2024-09-13T17:59:08.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.849+0000 7f9e8f7fe640 1 -- 172.21.15.111:0/1433101207 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9e80004b80 con 0x7f9e980cdf10 2024-09-13T17:59:08.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.850+0000 7f9e8f7fe640 1 --2- 172.21.15.111:0/1433101207 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9e7806ac00 0x7f9e7806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:08.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.850+0000 7f9e8f7fe640 1 -- 172.21.15.111:0/1433101207 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f9e80088a10 con 0x7f9e980cdf10 2024-09-13T17:59:08.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.850+0000 7f9e96ffd640 1 --2- 172.21.15.111:0/1433101207 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9e7806ac00 0x7f9e7806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:08.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.850+0000 7f9e96ffd640 1 --2- 172.21.15.111:0/1433101207 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9e7806ac00 0x7f9e7806d0c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9e90006fd0 tx=0x7f9e90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:08.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:08.854+0000 7f9e8f7fe640 1 -- 172.21.15.111:0/1433101207 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f9e981e68a0 con 0x7f9e980cdf10 2024-09-13T17:59:09.016 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.014+0000 7f9e9d365640 1 -- 172.21.15.111:0/1433101207 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e981e68a0 con 0x7f9e980cdf10 2024-09-13T17:59:09.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.015+0000 7f9e8f7fe640 1 -- 172.21.15.111:0/1433101207 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9e981e68a0 con 0x7f9e980cdf10 2024-09-13T17:59:09.017 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:59:09.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.017+0000 7f9e9d365640 1 -- 172.21.15.111:0/1433101207 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9e7806ac00 msgr2=0x7f9e7806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:09.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.017+0000 7f9e9d365640 1 --2- 172.21.15.111:0/1433101207 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9e7806ac00 0x7f9e7806d0c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9e90006fd0 tx=0x7f9e90008040 comp rx=0 tx=0).stop 2024-09-13T17:59:09.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.017+0000 7f9e9d365640 1 -- 172.21.15.111:0/1433101207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e980cdf10 msgr2=0x7f9e98157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:09.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.017+0000 7f9e9d365640 1 --2- 172.21.15.111:0/1433101207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e980cdf10 0x7f9e98157000 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000c4d0 tx=0x7f9e8000c990 comp rx=0 tx=0).stop 2024-09-13T17:59:09.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.017+0000 7f9e9d365640 1 -- 172.21.15.111:0/1433101207 shutdown_connections 2024-09-13T17:59:09.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.017+0000 7f9e9d365640 1 --2- 172.21.15.111:0/1433101207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e980ce2f0 0x7f9e980b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.017+0000 7f9e9d365640 1 --2- 172.21.15.111:0/1433101207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e980cdf10 0x7f9e98157000 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.017+0000 7f9e9d365640 1 --2- 172.21.15.111:0/1433101207 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9e7806ac00 0x7f9e7806d0c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.017+0000 7f9e9d365640 1 --2- 172.21.15.111:0/1433101207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e9806b600 0x7f9e980c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.017+0000 7f9e9d365640 1 -- 172.21.15.111:0/1433101207 >> 172.21.15.111:0/1433101207 conn(0x7f9e980cb5b0 msgr2=0x7f9e9809e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:09.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.017+0000 7f9e9d365640 1 -- 172.21.15.111:0/1433101207 shutdown_connections 2024-09-13T17:59:09.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.017+0000 7f9e9d365640 1 -- 172.21.15.111:0/1433101207 wait complete. 2024-09-13T17:59:09.030 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:59:09.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.128+0000 7fa9499c7640 1 Processor -- start 2024-09-13T17:59:09.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.128+0000 7fa9499c7640 1 -- start start 2024-09-13T17:59:09.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.128+0000 7fa9499c7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa944071d30 0x7fa944070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:09.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.129+0000 7fa9499c7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa9440c8920 0x7fa9440c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:09.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.129+0000 7fa9499c7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9440abdf0 con 0x7fa944072110 2024-09-13T17:59:09.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.129+0000 7fa9499c7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9440c92f0 con 0x7fa9440c8920 2024-09-13T17:59:09.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.129+0000 7fa9499c7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9440b2280 con 0x7fa944071d30 2024-09-13T17:59:09.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.129+0000 7fa9427fc640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa944072110 0x7fa9440b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33548/0 (socket says 172.21.15.111:33548) 2024-09-13T17:59:09.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.129+0000 7fa9437fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa9440c8920 0x7fa9440c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:09.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.129+0000 7fa9427fc640 1 -- 172.21.15.111:0/4074853463 learned_addr learned my addr 172.21.15.111:0/4074853463 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:09.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.129+0000 7fa942ffd640 1 --2- 172.21.15.111:0/4074853463 >> v2:172.21.15.188:3300/0 conn(0x7fa944071d30 0x7fa944070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:09.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.129+0000 7fa9437fe640 1 -- 172.21.15.111:0/4074853463 >> v2:172.21.15.188:3300/0 conn(0x7fa944071d30 msgr2=0x7fa944070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:09.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.129+0000 7fa9437fe640 1 --2- 172.21.15.111:0/4074853463 >> v2:172.21.15.188:3300/0 conn(0x7fa944071d30 0x7fa944070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.129+0000 7fa9437fe640 1 -- 172.21.15.111:0/4074853463 >> v1:172.21.15.111:6789/0 conn(0x7fa944072110 legacy=0x7fa9440b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:09.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.129+0000 7fa9437fe640 1 -- 172.21.15.111:0/4074853463 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9440ab1e0 con 0x7fa9440c8920 2024-09-13T17:59:09.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.129+0000 7fa941ffb640 1 -- 172.21.15.111:0/4074853463 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114243389 0 0) 0x7fa9440abdf0 con 0x7fa944072110 2024-09-13T17:59:09.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.130+0000 7fa9437fe640 1 --2- 172.21.15.111:0/4074853463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9440c8920 0x7fa9440c8db0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa9440cee60 tx=0x7fa93c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372251c17fc26249 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:09.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.130+0000 7fa941ffb640 1 -- 172.21.15.111:0/4074853463 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c02aea0 con 0x7fa9440c8920 2024-09-13T17:59:09.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.130+0000 7fa941ffb640 1 -- 172.21.15.111:0/4074853463 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c02d050 con 0x7fa9440c8920 2024-09-13T17:59:09.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.130+0000 7fa941ffb640 1 -- 172.21.15.111:0/4074853463 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c007960 con 0x7fa9440c8920 2024-09-13T17:59:09.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.130+0000 7fa9499c7640 1 -- 172.21.15.111:0/4074853463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9440c8920 msgr2=0x7fa9440c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:09.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.130+0000 7fa9499c7640 1 --2- 172.21.15.111:0/4074853463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9440c8920 0x7fa9440c8db0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa9440cee60 tx=0x7fa93c02a680 comp rx=0 tx=0).stop 2024-09-13T17:59:09.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.130+0000 7fa9499c7640 1 -- 172.21.15.111:0/4074853463 shutdown_connections 2024-09-13T17:59:09.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.130+0000 7fa9499c7640 1 --2- 172.21.15.111:0/4074853463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9440c8920 0x7fa9440c8db0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.130+0000 7fa9499c7640 1 --2- 172.21.15.111:0/4074853463 >> v2:172.21.15.188:3300/0 conn(0x7fa944071d30 0x7fa944070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.130+0000 7fa9499c7640 1 -- 172.21.15.111:0/4074853463 >> 172.21.15.111:0/4074853463 conn(0x7fa944069ed0 msgr2=0x7fa94406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:09.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.130+0000 7fa9499c7640 1 -- 172.21.15.111:0/4074853463 shutdown_connections 2024-09-13T17:59:09.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.130+0000 7fa9499c7640 1 -- 172.21.15.111:0/4074853463 wait complete. 2024-09-13T17:59:09.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.132+0000 7fa9499c7640 1 Processor -- start 2024-09-13T17:59:09.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.132+0000 7fa9499c7640 1 -- start start 2024-09-13T17:59:09.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.132+0000 7fa9499c7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa944071d30 0x7fa94415bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:09.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.132+0000 7fa9499c7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa944072110 0x7fa9440c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:09.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.132+0000 7fa9499c7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9440c8920 0x7fa9440cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:09.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.132+0000 7fa9499c7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9440c73e0 con 0x7fa9440c8920 2024-09-13T17:59:09.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.132+0000 7fa9499c7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94415c2c0 con 0x7fa944071d30 2024-09-13T17:59:09.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.132+0000 7fa9499c7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9440c67d0 con 0x7fa944072110 2024-09-13T17:59:09.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.133+0000 7fa9437fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9440c8920 0x7fa9440cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:09.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.133+0000 7fa9437fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9440c8920 0x7fa9440cad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52524/0 (socket says 172.21.15.111:52524) 2024-09-13T17:59:09.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.133+0000 7fa9437fe640 1 -- 172.21.15.111:0/3260792113 learned_addr learned my addr 172.21.15.111:0/3260792113 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:09.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.133+0000 7fa942ffd640 1 --2- 172.21.15.111:0/3260792113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa944071d30 0x7fa94415bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:09.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.133+0000 7fa9427fc640 1 --2- 172.21.15.111:0/3260792113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa944072110 0x7fa9440c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:09.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.133+0000 7fa9437fe640 1 -- 172.21.15.111:0/3260792113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa944072110 msgr2=0x7fa9440c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:09.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.133+0000 7fa9437fe640 1 --2- 172.21.15.111:0/3260792113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa944072110 0x7fa9440c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.133+0000 7fa9437fe640 1 -- 172.21.15.111:0/3260792113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa944071d30 msgr2=0x7fa94415bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:09.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.133+0000 7fa9437fe640 1 --2- 172.21.15.111:0/3260792113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa944071d30 0x7fa94415bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.133+0000 7fa9437fe640 1 -- 172.21.15.111:0/3260792113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c009a70 con 0x7fa9440c8920 2024-09-13T17:59:09.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.133+0000 7fa9427fc640 1 --2- 172.21.15.111:0/3260792113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa944072110 0x7fa9440c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:09.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.133+0000 7fa9437fe640 1 --2- 172.21.15.111:0/3260792113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9440c8920 0x7fa9440cad50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa93c02ac70 tx=0x7fa93c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:09.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.133+0000 7fa93bfff640 1 -- 172.21.15.111:0/3260792113 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c03b820 con 0x7fa9440c8920 2024-09-13T17:59:09.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.133+0000 7fa93bfff640 1 -- 172.21.15.111:0/3260792113 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c03b980 con 0x7fa9440c8920 2024-09-13T17:59:09.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.133+0000 7fa9499c7640 1 -- 172.21.15.111:0/3260792113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9440c6ab0 con 0x7fa9440c8920 2024-09-13T17:59:09.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.133+0000 7fa93bfff640 1 -- 172.21.15.111:0/3260792113 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c03bcf0 con 0x7fa9440c8920 2024-09-13T17:59:09.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.133+0000 7fa9499c7640 1 -- 172.21.15.111:0/3260792113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9440c7070 con 0x7fa9440c8920 2024-09-13T17:59:09.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.134+0000 7fa9499c7640 1 -- 172.21.15.111:0/3260792113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa944070190 con 0x7fa9440c8920 2024-09-13T17:59:09.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.135+0000 7fa93bfff640 1 -- 172.21.15.111:0/3260792113 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa93c031400 con 0x7fa9440c8920 2024-09-13T17:59:09.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.135+0000 7fa93bfff640 1 --2- 172.21.15.111:0/3260792113 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa92406ac00 0x7fa92406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:09.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.135+0000 7fa93bfff640 1 -- 172.21.15.111:0/3260792113 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fa93c0aac80 con 0x7fa9440c8920 2024-09-13T17:59:09.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.135+0000 7fa942ffd640 1 --2- 172.21.15.111:0/3260792113 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa92406ac00 0x7fa92406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:09.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.135+0000 7fa942ffd640 1 --2- 172.21.15.111:0/3260792113 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa92406ac00 0x7fa92406d0c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa934006fd0 tx=0x7fa934008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:09.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.139+0000 7fa93bfff640 1 -- 172.21.15.111:0/3260792113 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fa93c075cb0 con 0x7fa9440c8920 2024-09-13T17:59:09.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.314+0000 7fa9499c7640 1 -- 172.21.15.111:0/3260792113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa94415cb20 con 0x7fa9440c8920 2024-09-13T17:59:09.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.314+0000 7fa93bfff640 1 -- 172.21.15.111:0/3260792113 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v42) v1 ==== 80+0+122 (secure 0 0 0) 0x7fa93c0792d0 con 0x7fa9440c8920 2024-09-13T17:59:09.316 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6837/982359636 2024-09-14T17:59:01.542287+0000 2024-09-13T17:59:09.317 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6836/982359636 2024-09-14T17:59:01.542287+0000 2024-09-13T17:59:09.317 INFO:teuthology.orchestra.run.smithi111.stderr:listed 2 entries 2024-09-13T17:59:09.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.316+0000 7fa9499c7640 1 -- 172.21.15.111:0/3260792113 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa92406ac00 msgr2=0x7fa92406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:09.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.316+0000 7fa9499c7640 1 --2- 172.21.15.111:0/3260792113 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa92406ac00 0x7fa92406d0c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa934006fd0 tx=0x7fa934008040 comp rx=0 tx=0).stop 2024-09-13T17:59:09.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.317+0000 7fa9499c7640 1 -- 172.21.15.111:0/3260792113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9440c8920 msgr2=0x7fa9440cad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:09.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.317+0000 7fa9499c7640 1 --2- 172.21.15.111:0/3260792113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9440c8920 0x7fa9440cad50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa93c02ac70 tx=0x7fa93c004a40 comp rx=0 tx=0).stop 2024-09-13T17:59:09.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.317+0000 7fa9499c7640 1 -- 172.21.15.111:0/3260792113 shutdown_connections 2024-09-13T17:59:09.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.317+0000 7fa9499c7640 1 --2- 172.21.15.111:0/3260792113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9440c8920 0x7fa9440cad50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.317+0000 7fa9499c7640 1 --2- 172.21.15.111:0/3260792113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa944072110 0x7fa9440c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.317+0000 7fa9499c7640 1 --2- 172.21.15.111:0/3260792113 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa92406ac00 0x7fa92406d0c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.317+0000 7fa9499c7640 1 --2- 172.21.15.111:0/3260792113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa944071d30 0x7fa94415bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.317+0000 7fa9499c7640 1 -- 172.21.15.111:0/3260792113 >> 172.21.15.111:0/3260792113 conn(0x7fa944069ed0 msgr2=0x7fa9440ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:09.319 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.317+0000 7fa9499c7640 1 -- 172.21.15.111:0/3260792113 shutdown_connections 2024-09-13T17:59:09.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.317+0000 7fa9499c7640 1 -- 172.21.15.111:0/3260792113 wait complete. 2024-09-13T17:59:09.329 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi111.front.sepia.ceph.com... 2024-09-13T17:59:09.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:59:09.330 DEBUG:teuthology.orchestra.run.smithi111:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T17:59:09.353 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[53181]: fuse finished with error 0 and tester_r 0 2024-09-13T17:59:09.354 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T17:59:09.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:59:09.355 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T17:59:09.509 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:59:09.509 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:59:09.529 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T17:59:09.529 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-09-13T17:59:09.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:59:09.529 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-13T17:59:09.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:59:09.586 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-09-13T17:59:09.587 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:59:09.587 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:59:09.603 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056025,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14842 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T17:59:09.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T17:59:09.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:59:09.603 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-13T17:59:09.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:59:09.658 INFO:teuthology.orchestra.run.smithi111.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-13T17:59:09.659 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:59:09.659 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:59:09.677 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T17:59:09.678 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-09-13T17:59:09.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:59:09.678 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-13T17:59:09.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:59:09.738 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-13T17:59:09.739 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:59:09.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.835+0000 7fc641726640 1 Processor -- start 2024-09-13T17:59:09.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.835+0000 7fc641726640 1 -- start start 2024-09-13T17:59:09.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.835+0000 7fc641726640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc63c071d30 0x7fc63c070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.835+0000 7fc641726640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc63c072110 0x7fc63c0b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.836+0000 7fc641726640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc63c0abdf0 con 0x7fc63c0c8920 2024-09-13T17:59:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.836+0000 7fc641726640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc63c0ab1e0 con 0x7fc63c071d30 2024-09-13T17:59:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.836+0000 7fc641726640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c0ab4c0 con 0x7fc63c072110 2024-09-13T17:59:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.836+0000 7fc63affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc63c071d30 0x7fc63c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.836+0000 7fc63affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc63c071d30 0x7fc63c070190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52526/0 (socket says 172.21.15.111:52526) 2024-09-13T17:59:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.836+0000 7fc63affd640 1 -- 172.21.15.111:0/3909552556 learned_addr learned my addr 172.21.15.111:0/3909552556 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.836+0000 7fc633fff640 1 --2- 172.21.15.111:0/3909552556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc63c072110 0x7fc63c0b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.836+0000 7fc63affd640 1 -- 172.21.15.111:0/3909552556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc63c072110 msgr2=0x7fc63c0b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.836+0000 7fc63affd640 1 --2- 172.21.15.111:0/3909552556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc63c072110 0x7fc63c0b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.836+0000 7fc63affd640 1 -- 172.21.15.111:0/3909552556 >> v1:172.21.15.188:6789/0 conn(0x7fc63c0c8920 legacy=0x7fc63c0ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.836+0000 7fc63affd640 1 -- 172.21.15.111:0/3909552556 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c0ab7a0 con 0x7fc63c071d30 2024-09-13T17:59:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.836+0000 7fc633fff640 1 --2- 172.21.15.111:0/3909552556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc63c072110 0x7fc63c0b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:59:09.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.837+0000 7fc63affd640 1 --2- 172.21.15.111:0/3909552556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc63c071d30 0x7fc63c070190 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00bb40 tx=0x7fc62c02c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dde3bb93ace44f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:09.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.837+0000 7fc63a7fc640 1 -- 172.21.15.111:0/3909552556 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c0327c0 con 0x7fc63c071d30 2024-09-13T17:59:09.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.837+0000 7fc63a7fc640 1 -- 172.21.15.111:0/3909552556 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc62c032920 con 0x7fc63c071d30 2024-09-13T17:59:09.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.837+0000 7fc63a7fc640 1 -- 172.21.15.111:0/3909552556 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c032cb0 con 0x7fc63c071d30 2024-09-13T17:59:09.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.837+0000 7fc641726640 1 -- 172.21.15.111:0/3909552556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc63c071d30 msgr2=0x7fc63c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:09.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.837+0000 7fc641726640 1 --2- 172.21.15.111:0/3909552556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc63c071d30 0x7fc63c070190 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00bb40 tx=0x7fc62c02c900 comp rx=0 tx=0).stop 2024-09-13T17:59:09.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.837+0000 7fc641726640 1 -- 172.21.15.111:0/3909552556 shutdown_connections 2024-09-13T17:59:09.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.837+0000 7fc641726640 1 --2- 172.21.15.111:0/3909552556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc63c072110 0x7fc63c0b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.837+0000 7fc641726640 1 --2- 172.21.15.111:0/3909552556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc63c071d30 0x7fc63c070190 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.837+0000 7fc641726640 1 -- 172.21.15.111:0/3909552556 >> 172.21.15.111:0/3909552556 conn(0x7fc63c069ed0 msgr2=0x7fc63c06a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:09.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.837+0000 7fc641726640 1 -- 172.21.15.111:0/3909552556 shutdown_connections 2024-09-13T17:59:09.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.837+0000 7fc641726640 1 -- 172.21.15.111:0/3909552556 wait complete. 2024-09-13T17:59:09.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.838+0000 7fc641726640 1 Processor -- start 2024-09-13T17:59:09.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.839+0000 7fc641726640 1 -- start start 2024-09-13T17:59:09.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.839+0000 7fc641726640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc63c071d30 0x7fc63c0b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:09.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.839+0000 7fc641726640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc63c072110 0x7fc63c0c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:09.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.839+0000 7fc641726640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc63c0c8920 0x7fc63c14ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:09.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.839+0000 7fc641726640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c0c5440 con 0x7fc63c0c8920 2024-09-13T17:59:09.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.839+0000 7fc641726640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c06e110 con 0x7fc63c071d30 2024-09-13T17:59:09.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.839+0000 7fc641726640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c0c56b0 con 0x7fc63c072110 2024-09-13T17:59:09.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.839+0000 7fc63b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc63c0c8920 0x7fc63c14ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:09.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.839+0000 7fc63b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc63c0c8920 0x7fc63c14ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52534/0 (socket says 172.21.15.111:52534) 2024-09-13T17:59:09.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.839+0000 7fc63b7fe640 1 -- 172.21.15.111:0/2536529846 learned_addr learned my addr 172.21.15.111:0/2536529846 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:09.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.839+0000 7fc63affd640 1 --2- 172.21.15.111:0/2536529846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc63c071d30 0x7fc63c0b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:09.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.839+0000 7fc633fff640 1 --2- 172.21.15.111:0/2536529846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc63c072110 0x7fc63c0c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:09.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.840+0000 7fc63b7fe640 1 -- 172.21.15.111:0/2536529846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc63c072110 msgr2=0x7fc63c0c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:09.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.840+0000 7fc63b7fe640 1 --2- 172.21.15.111:0/2536529846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc63c072110 0x7fc63c0c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.840+0000 7fc63b7fe640 1 -- 172.21.15.111:0/2536529846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc63c071d30 msgr2=0x7fc63c0b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:09.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.840+0000 7fc63b7fe640 1 --2- 172.21.15.111:0/2536529846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc63c071d30 0x7fc63c0b00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:09.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.840+0000 7fc63b7fe640 1 -- 172.21.15.111:0/2536529846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c00b770 con 0x7fc63c0c8920 2024-09-13T17:59:09.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.840+0000 7fc63b7fe640 1 --2- 172.21.15.111:0/2536529846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc63c0c8920 0x7fc63c14ab20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc63400c780 tx=0x7fc63400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:09.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.840+0000 7fc6337fe640 1 -- 172.21.15.111:0/2536529846 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc634004b60 con 0x7fc63c0c8920 2024-09-13T17:59:09.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.840+0000 7fc6337fe640 1 -- 172.21.15.111:0/2536529846 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63400ce90 con 0x7fc63c0c8920 2024-09-13T17:59:09.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.840+0000 7fc641726640 1 -- 172.21.15.111:0/2536529846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc63c0c5990 con 0x7fc63c0c8920 2024-09-13T17:59:09.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.840+0000 7fc6337fe640 1 -- 172.21.15.111:0/2536529846 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63400a850 con 0x7fc63c0c8920 2024-09-13T17:59:09.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.840+0000 7fc641726640 1 -- 172.21.15.111:0/2536529846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc63c06dd50 con 0x7fc63c0c8920 2024-09-13T17:59:09.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.841+0000 7fc641726640 1 -- 172.21.15.111:0/2536529846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc63c0c5b80 con 0x7fc63c0c8920 2024-09-13T17:59:09.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.841+0000 7fc6337fe640 1 -- 172.21.15.111:0/2536529846 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc63400a9b0 con 0x7fc63c0c8920 2024-09-13T17:59:09.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.842+0000 7fc6337fe640 1 --2- 172.21.15.111:0/2536529846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc61c06ac30 0x7fc61c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:09.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.842+0000 7fc6337fe640 1 -- 172.21.15.111:0/2536529846 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fc634089500 con 0x7fc63c0c8920 2024-09-13T17:59:09.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.842+0000 7fc63affd640 1 --2- 172.21.15.111:0/2536529846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc61c06ac30 0x7fc61c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:09.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.842+0000 7fc63affd640 1 --2- 172.21.15.111:0/2536529846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc61c06ac30 0x7fc61c06d0f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc62c037000 tx=0x7fc62c032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:09.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:09.846+0000 7fc6337fe640 1 -- 172.21.15.111:0/2536529846 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc6340545e0 con 0x7fc63c0c8920 2024-09-13T17:59:10.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.042+0000 7fc641726640 1 -- 172.21.15.111:0/2536529846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc63c070190 con 0x7fc63c0c8920 2024-09-13T17:59:10.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.043+0000 7fc6337fe640 1 -- 172.21.15.111:0/2536529846 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4742 (secure 0 0 0) 0x7fc634057c00 con 0x7fc63c0c8920 2024-09-13T17:59:10.045 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:10.045 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4217138481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4217138481},{"type":"v1","addr":"172.21.15.188:6834","nonce":4217138481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3099948287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3099948287},{"type":"v1","addr":"172.21.15.188:6835","nonce":3099948287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/904506495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":904506495},{"type":"v1","addr":"172.21.15.111:6837","nonce":904506495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:58:43.534567+0000","modified":"2024-09-13T17:59:05.554735+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5487":{"gid":5487,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6835/214034224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":214034224},{"type":"v1","addr":"172.21.15.111:6835","nonce":214034224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:59:10.045 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 21 2024-09-13T17:59:10.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.045+0000 7fc641726640 1 -- 172.21.15.111:0/2536529846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc61c06ac30 msgr2=0x7fc61c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.045+0000 7fc641726640 1 --2- 172.21.15.111:0/2536529846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc61c06ac30 0x7fc61c06d0f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc62c037000 tx=0x7fc62c032000 comp rx=0 tx=0).stop 2024-09-13T17:59:10.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.045+0000 7fc641726640 1 -- 172.21.15.111:0/2536529846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc63c0c8920 msgr2=0x7fc63c14ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.045+0000 7fc641726640 1 --2- 172.21.15.111:0/2536529846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc63c0c8920 0x7fc63c14ab20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc63400c780 tx=0x7fc63400cc40 comp rx=0 tx=0).stop 2024-09-13T17:59:10.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.045+0000 7fc641726640 1 -- 172.21.15.111:0/2536529846 shutdown_connections 2024-09-13T17:59:10.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.045+0000 7fc641726640 1 --2- 172.21.15.111:0/2536529846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc63c0c8920 0x7fc63c14ab20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.045+0000 7fc641726640 1 --2- 172.21.15.111:0/2536529846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc63c072110 0x7fc63c0c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.045+0000 7fc641726640 1 --2- 172.21.15.111:0/2536529846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc61c06ac30 0x7fc61c06d0f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.045+0000 7fc641726640 1 --2- 172.21.15.111:0/2536529846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc63c071d30 0x7fc63c0b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.045+0000 7fc641726640 1 -- 172.21.15.111:0/2536529846 >> 172.21.15.111:0/2536529846 conn(0x7fc63c069ed0 msgr2=0x7fc63c0ac280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:10.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.045+0000 7fc641726640 1 -- 172.21.15.111:0/2536529846 shutdown_connections 2024-09-13T17:59:10.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.045+0000 7fc641726640 1 -- 172.21.15.111:0/2536529846 wait complete. 2024-09-13T17:59:10.058 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:59:10.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.154+0000 7f7116176640 1 Processor -- start 2024-09-13T17:59:10.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.154+0000 7f7116176640 1 -- start start 2024-09-13T17:59:10.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.155+0000 7f7116176640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7110071d30 0x7f7110070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.155+0000 7f7116176640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71100abdf0 con 0x7f71100c8920 2024-09-13T17:59:10.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.155+0000 7f7116176640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71100b2280 con 0x7f7110072110 2024-09-13T17:59:10.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.155+0000 7f7116176640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71100ab1e0 con 0x7f7110071d30 2024-09-13T17:59:10.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.155+0000 7f710f7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7110071d30 0x7f7110070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.155+0000 7f710ffff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f71100c8920 0x7f71100ac280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33562/0 (socket says 172.21.15.111:33562) 2024-09-13T17:59:10.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.155+0000 7f710ffff640 1 -- 172.21.15.111:0/1517084393 learned_addr learned my addr 172.21.15.111:0/1517084393 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:10.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.155+0000 7f710f7fe640 1 -- 172.21.15.111:0/1517084393 >> v1:172.21.15.188:6789/0 conn(0x7f7110072110 legacy=0x7f71100b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:10.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.155+0000 7f710f7fe640 1 -- 172.21.15.111:0/1517084393 >> v1:172.21.15.111:6789/0 conn(0x7f71100c8920 legacy=0x7f71100ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.155+0000 7f710f7fe640 1 -- 172.21.15.111:0/1517084393 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71100ab4c0 con 0x7f7110071d30 2024-09-13T17:59:10.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.155+0000 7f710e7fc640 1 -- 172.21.15.111:0/1517084393 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406260732 0 0) 0x7f71100abdf0 con 0x7f71100c8920 2024-09-13T17:59:10.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.156+0000 7f710f7fe640 1 --2- 172.21.15.111:0/1517084393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7110071d30 0x7f7110070190 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7100009d90 tx=0x7f710002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eddb9b8ba2171ef1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:10.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.156+0000 7f710e7fc640 1 -- 172.21.15.111:0/1517084393 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100007740 con 0x7f7110071d30 2024-09-13T17:59:10.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.156+0000 7f710e7fc640 1 -- 172.21.15.111:0/1517084393 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710002d050 con 0x7f7110071d30 2024-09-13T17:59:10.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.156+0000 7f710e7fc640 1 -- 172.21.15.111:0/1517084393 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100007ac0 con 0x7f7110071d30 2024-09-13T17:59:10.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.157+0000 7f7116176640 1 -- 172.21.15.111:0/1517084393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7110071d30 msgr2=0x7f7110070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.157+0000 7f7116176640 1 --2- 172.21.15.111:0/1517084393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7110071d30 0x7f7110070190 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7100009d90 tx=0x7f710002a6a0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.157+0000 7f7116176640 1 -- 172.21.15.111:0/1517084393 shutdown_connections 2024-09-13T17:59:10.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.157+0000 7f7116176640 1 --2- 172.21.15.111:0/1517084393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7110071d30 0x7f7110070190 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.157+0000 7f7116176640 1 -- 172.21.15.111:0/1517084393 >> 172.21.15.111:0/1517084393 conn(0x7f7110069ed0 msgr2=0x7f711006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:10.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.157+0000 7f7116176640 1 -- 172.21.15.111:0/1517084393 shutdown_connections 2024-09-13T17:59:10.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.157+0000 7f7116176640 1 -- 172.21.15.111:0/1517084393 wait complete. 2024-09-13T17:59:10.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.158+0000 7f7116176640 1 Processor -- start 2024-09-13T17:59:10.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.158+0000 7f7116176640 1 -- start start 2024-09-13T17:59:10.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.159+0000 7f7116176640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7110071d30 0x7f71101e60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.159+0000 7f7116176640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7110072110 0x7f71100cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.159+0000 7f7116176640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71100c8920 0x7f71101f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.159+0000 7f7116176640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71101f6ce0 con 0x7f71100c8920 2024-09-13T17:59:10.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.159+0000 7f7116176640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71101f2b60 con 0x7f7110071d30 2024-09-13T17:59:10.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.159+0000 7f7116176640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71101f6a00 con 0x7f7110072110 2024-09-13T17:59:10.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.159+0000 7f710ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71100c8920 0x7f71101f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.159+0000 7f710ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71100c8920 0x7f71101f5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52548/0 (socket says 172.21.15.111:52548) 2024-09-13T17:59:10.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.159+0000 7f710ffff640 1 -- 172.21.15.111:0/77716325 learned_addr learned my addr 172.21.15.111:0/77716325 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:10.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.159+0000 7f710ffff640 1 -- 172.21.15.111:0/77716325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7110072110 msgr2=0x7f71100cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.159+0000 7f710ffff640 1 --2- 172.21.15.111:0/77716325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7110072110 0x7f71100cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.159+0000 7f710ffff640 1 -- 172.21.15.111:0/77716325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7110071d30 msgr2=0x7f71101e60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.159+0000 7f710ffff640 1 --2- 172.21.15.111:0/77716325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7110071d30 0x7f71101e60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.159+0000 7f710ffff640 1 -- 172.21.15.111:0/77716325 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7100009a70 con 0x7f71100c8920 2024-09-13T17:59:10.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.159+0000 7f710ffff640 1 --2- 172.21.15.111:0/77716325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71100c8920 0x7f71101f5db0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f710800c450 tx=0x7f710800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:10.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.159+0000 7f70f7fff640 1 -- 172.21.15.111:0/77716325 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71080074e0 con 0x7f71100c8920 2024-09-13T17:59:10.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.159+0000 7f70f7fff640 1 -- 172.21.15.111:0/77716325 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7108007660 con 0x7f71100c8920 2024-09-13T17:59:10.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.160+0000 7f7116176640 1 -- 172.21.15.111:0/77716325 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71100cca40 con 0x7f71100c8920 2024-09-13T17:59:10.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.160+0000 7f70f7fff640 1 -- 172.21.15.111:0/77716325 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108004a20 con 0x7f71100c8920 2024-09-13T17:59:10.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.160+0000 7f7116176640 1 -- 172.21.15.111:0/77716325 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71100cd000 con 0x7f71100c8920 2024-09-13T17:59:10.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.161+0000 7f7116176640 1 -- 172.21.15.111:0/77716325 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71100cc8a0 con 0x7f71100c8920 2024-09-13T17:59:10.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.161+0000 7f70f7fff640 1 -- 172.21.15.111:0/77716325 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7108004b80 con 0x7f71100c8920 2024-09-13T17:59:10.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.161+0000 7f70f7fff640 1 --2- 172.21.15.111:0/77716325 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f70fc06ac00 0x7f70fc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.161+0000 7f70f7fff640 1 -- 172.21.15.111:0/77716325 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f7108088640 con 0x7f71100c8920 2024-09-13T17:59:10.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.161+0000 7f710f7fe640 1 --2- 172.21.15.111:0/77716325 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f70fc06ac00 0x7f70fc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.162+0000 7f710f7fe640 1 --2- 172.21.15.111:0/77716325 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f70fc06ac00 0x7f70fc06d0c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7100034000 tx=0x7f710002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:10.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.165+0000 7f70f7fff640 1 -- 172.21.15.111:0/77716325 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f71100cc8a0 con 0x7f71100c8920 2024-09-13T17:59:10.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.329+0000 7f7116176640 1 -- 172.21.15.111:0/77716325 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71100cc8a0 con 0x7f71100c8920 2024-09-13T17:59:10.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.330+0000 7f70f7fff640 1 -- 172.21.15.111:0/77716325 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+17012 (secure 0 0 0) 0x7f71100cc8a0 con 0x7f71100c8920 2024-09-13T17:59:10.334 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:10.334 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":42,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:59:08.617279+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:58:41.698998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T17:58:42.499870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1c","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6837/982359636":"2024-09-14T17:59:01.542287+0000","172.21.15.111:6836/982359636":"2024-09-14T17:59:01.542287+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:59:10.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.332+0000 7f7116176640 1 -- 172.21.15.111:0/77716325 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f70fc06ac00 msgr2=0x7f70fc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.332+0000 7f7116176640 1 --2- 172.21.15.111:0/77716325 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f70fc06ac00 0x7f70fc06d0c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7100034000 tx=0x7f710002e000 comp rx=0 tx=0).stop 2024-09-13T17:59:10.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.332+0000 7f7116176640 1 -- 172.21.15.111:0/77716325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71100c8920 msgr2=0x7f71101f5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.332+0000 7f7116176640 1 --2- 172.21.15.111:0/77716325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71100c8920 0x7f71101f5db0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f710800c450 tx=0x7f710800c910 comp rx=0 tx=0).stop 2024-09-13T17:59:10.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.332+0000 7f7116176640 1 -- 172.21.15.111:0/77716325 shutdown_connections 2024-09-13T17:59:10.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.332+0000 7f7116176640 1 --2- 172.21.15.111:0/77716325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71100c8920 0x7f71101f5db0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.332+0000 7f7116176640 1 --2- 172.21.15.111:0/77716325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7110072110 0x7f71100cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.332+0000 7f7116176640 1 --2- 172.21.15.111:0/77716325 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f70fc06ac00 0x7f70fc06d0c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.332+0000 7f7116176640 1 --2- 172.21.15.111:0/77716325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7110071d30 0x7f71101e60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.332+0000 7f7116176640 1 -- 172.21.15.111:0/77716325 >> 172.21.15.111:0/77716325 conn(0x7f7110069ed0 msgr2=0x7f71100aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:10.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.332+0000 7f7116176640 1 -- 172.21.15.111:0/77716325 shutdown_connections 2024-09-13T17:59:10.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.332+0000 7f7116176640 1 -- 172.21.15.111:0/77716325 wait complete. 2024-09-13T17:59:10.345 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:57:14.294908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:58:41.698998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:58:42.499870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:59:10.345 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:59:10.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.444+0000 7f07b32b7640 1 Processor -- start 2024-09-13T17:59:10.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.444+0000 7f07b32b7640 1 -- start start 2024-09-13T17:59:10.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.445+0000 7f07b32b7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f07ac0cec00 0x7f07ac0ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.445+0000 7f07b32b7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07ac0ccbe0 con 0x7f07ac0b2b40 2024-09-13T17:59:10.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.445+0000 7f07b32b7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07ac0b1810 con 0x7f07ac159310 2024-09-13T17:59:10.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.445+0000 7f07b32b7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07ac0ad940 con 0x7f07ac0cec00 2024-09-13T17:59:10.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.445+0000 7f07b1854640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f07ac0cec00 0x7f07ac0ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.445+0000 7f07b0852640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f07ac0b2b40 0x7f07ac0ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33576/0 (socket says 172.21.15.111:33576) 2024-09-13T17:59:10.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.445+0000 7f07b0852640 1 -- 172.21.15.111:0/3645959816 learned_addr learned my addr 172.21.15.111:0/3645959816 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:10.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.446+0000 7f07b1854640 1 -- 172.21.15.111:0/3645959816 >> v1:172.21.15.188:6789/0 conn(0x7f07ac159310 legacy=0x7f07ac0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:10.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.446+0000 7f07b1854640 1 -- 172.21.15.111:0/3645959816 >> v1:172.21.15.111:6789/0 conn(0x7f07ac0b2b40 legacy=0x7f07ac0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.446+0000 7f07b1854640 1 -- 172.21.15.111:0/3645959816 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ac06d9e0 con 0x7f07ac0cec00 2024-09-13T17:59:10.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.446+0000 7f07b1854640 1 --2- 172.21.15.111:0/3645959816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07ac0cec00 0x7f07ac0ad400 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f07a40097a0 tx=0x7f07a402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8654d1bc40119 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:10.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.446+0000 7f07a3fff640 1 -- 172.21.15.111:0/3645959816 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a402ae50 con 0x7f07ac0cec00 2024-09-13T17:59:10.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.446+0000 7f07a3fff640 1 -- 172.21.15.111:0/3645959816 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a4032440 con 0x7f07ac0cec00 2024-09-13T17:59:10.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.446+0000 7f07a3fff640 1 -- 172.21.15.111:0/3645959816 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a40327d0 con 0x7f07ac0cec00 2024-09-13T17:59:10.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.447+0000 7f07b32b7640 1 -- 172.21.15.111:0/3645959816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07ac0cec00 msgr2=0x7f07ac0ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.447+0000 7f07b32b7640 1 --2- 172.21.15.111:0/3645959816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07ac0cec00 0x7f07ac0ad400 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f07a40097a0 tx=0x7f07a402a140 comp rx=0 tx=0).stop 2024-09-13T17:59:10.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.447+0000 7f07b32b7640 1 -- 172.21.15.111:0/3645959816 shutdown_connections 2024-09-13T17:59:10.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.447+0000 7f07b32b7640 1 --2- 172.21.15.111:0/3645959816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07ac0cec00 0x7f07ac0ad400 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.447+0000 7f07b32b7640 1 -- 172.21.15.111:0/3645959816 >> 172.21.15.111:0/3645959816 conn(0x7f07ac06a0b0 msgr2=0x7f07ac06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:10.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.447+0000 7f07b32b7640 1 -- 172.21.15.111:0/3645959816 shutdown_connections 2024-09-13T17:59:10.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.447+0000 7f07b32b7640 1 -- 172.21.15.111:0/3645959816 wait complete. 2024-09-13T17:59:10.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.448+0000 7f07b32b7640 1 Processor -- start 2024-09-13T17:59:10.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.448+0000 7f07b32b7640 1 -- start start 2024-09-13T17:59:10.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.449+0000 7f07b32b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07ac0b2b40 0x7f07ac1f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.449+0000 7f07b32b7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07ac0cec00 0x7f07ac156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.449+0000 7f07b32b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07ac159310 0x7f07ac15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.449+0000 7f07b32b7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac09b640 con 0x7f07ac0cec00 2024-09-13T17:59:10.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.449+0000 7f07b32b7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac1f4520 con 0x7f07ac159310 2024-09-13T17:59:10.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.449+0000 7f07b32b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac157430 con 0x7f07ac0b2b40 2024-09-13T17:59:10.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.449+0000 7f07b0852640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07ac0cec00 0x7f07ac156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.449+0000 7f07b0852640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07ac0cec00 0x7f07ac156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52558/0 (socket says 172.21.15.111:52558) 2024-09-13T17:59:10.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.449+0000 7f07b0852640 1 -- 172.21.15.111:0/835669450 learned_addr learned my addr 172.21.15.111:0/835669450 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:10.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.449+0000 7f07b0852640 1 -- 172.21.15.111:0/835669450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07ac0b2b40 msgr2=0x7f07ac1f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.449+0000 7f07b0852640 1 --2- 172.21.15.111:0/835669450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07ac0b2b40 0x7f07ac1f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.449+0000 7f07b0852640 1 -- 172.21.15.111:0/835669450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07ac159310 msgr2=0x7f07ac15b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.449+0000 7f07b0852640 1 --2- 172.21.15.111:0/835669450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07ac159310 0x7f07ac15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.449+0000 7f07b0852640 1 -- 172.21.15.111:0/835669450 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a4009400 con 0x7f07ac0cec00 2024-09-13T17:59:10.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.449+0000 7f07b0852640 1 --2- 172.21.15.111:0/835669450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07ac0cec00 0x7f07ac156870 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f079c00c4a0 tx=0x7f079c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:10.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.450+0000 7f07a1ffb640 1 -- 172.21.15.111:0/835669450 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c005ce0 con 0x7f07ac0cec00 2024-09-13T17:59:10.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.450+0000 7f07a1ffb640 1 -- 172.21.15.111:0/835669450 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c005e40 con 0x7f07ac0cec00 2024-09-13T17:59:10.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.450+0000 7f07b32b7640 1 -- 172.21.15.111:0/835669450 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ac0c89f0 con 0x7f07ac0cec00 2024-09-13T17:59:10.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.450+0000 7f07a1ffb640 1 -- 172.21.15.111:0/835669450 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c007270 con 0x7f07ac0cec00 2024-09-13T17:59:10.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.450+0000 7f07b32b7640 1 -- 172.21.15.111:0/835669450 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ac0c8c90 con 0x7f07ac0cec00 2024-09-13T17:59:10.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.451+0000 7f07b32b7640 1 -- 172.21.15.111:0/835669450 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07ac0ad400 con 0x7f07ac0cec00 2024-09-13T17:59:10.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.451+0000 7f07a1ffb640 1 -- 172.21.15.111:0/835669450 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f079c0073d0 con 0x7f07ac0cec00 2024-09-13T17:59:10.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.452+0000 7f07a1ffb640 1 --2- 172.21.15.111:0/835669450 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f079006ac30 0x7f079006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.452+0000 7f07a1ffb640 1 -- 172.21.15.111:0/835669450 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f079c0888d0 con 0x7f07ac0cec00 2024-09-13T17:59:10.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.452+0000 7f07b1053640 1 --2- 172.21.15.111:0/835669450 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f079006ac30 0x7f079006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.452+0000 7f07b1053640 1 --2- 172.21.15.111:0/835669450 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f079006ac30 0x7f079006d0f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f07a800abb0 tx=0x7f07a8009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:10.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.456+0000 7f07a1ffb640 1 -- 172.21.15.111:0/835669450 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f079c0539c0 con 0x7f07ac0cec00 2024-09-13T17:59:10.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.617+0000 7f07b32b7640 1 -- 172.21.15.111:0/835669450 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f07ac06e4b0 con 0x7f07ac0cec00 2024-09-13T17:59:10.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.617+0000 7f07a1ffb640 1 -- 172.21.15.111:0/835669450 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f079c017020 con 0x7f07ac0cec00 2024-09-13T17:59:10.620 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-13T17:59:10.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.620+0000 7f07b32b7640 1 -- 172.21.15.111:0/835669450 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f079006ac30 msgr2=0x7f079006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.620+0000 7f07b32b7640 1 --2- 172.21.15.111:0/835669450 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f079006ac30 0x7f079006d0f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f07a800abb0 tx=0x7f07a8009250 comp rx=0 tx=0).stop 2024-09-13T17:59:10.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.620+0000 7f07b32b7640 1 -- 172.21.15.111:0/835669450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07ac0cec00 msgr2=0x7f07ac156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.620+0000 7f07b32b7640 1 --2- 172.21.15.111:0/835669450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07ac0cec00 0x7f07ac156870 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f079c00c4a0 tx=0x7f079c00c960 comp rx=0 tx=0).stop 2024-09-13T17:59:10.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.620+0000 7f07b32b7640 1 -- 172.21.15.111:0/835669450 shutdown_connections 2024-09-13T17:59:10.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.620+0000 7f07b32b7640 1 --2- 172.21.15.111:0/835669450 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f079006ac30 0x7f079006d0f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.620+0000 7f07b32b7640 1 --2- 172.21.15.111:0/835669450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07ac159310 0x7f07ac15b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.620+0000 7f07b32b7640 1 --2- 172.21.15.111:0/835669450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07ac0cec00 0x7f07ac156870 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.620+0000 7f07b32b7640 1 --2- 172.21.15.111:0/835669450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07ac0b2b40 0x7f07ac1f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.620+0000 7f07b32b7640 1 -- 172.21.15.111:0/835669450 >> 172.21.15.111:0/835669450 conn(0x7f07ac06a0b0 msgr2=0x7f07ac0afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:10.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.620+0000 7f07b32b7640 1 -- 172.21.15.111:0/835669450 shutdown_connections 2024-09-13T17:59:10.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.620+0000 7f07b32b7640 1 -- 172.21.15.111:0/835669450 wait complete. 2024-09-13T17:59:10.633 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:59:10.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.730+0000 7f6b36412640 1 Processor -- start 2024-09-13T17:59:10.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.730+0000 7f6b36412640 1 -- start start 2024-09-13T17:59:10.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.731+0000 7f6b36412640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6b30158b00 0x7f6b3006c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.731+0000 7f6b36412640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b300b2d10 con 0x7f6b3006c6d0 2024-09-13T17:59:10.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.731+0000 7f6b36412640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b300b2ff0 con 0x7f6b300b2170 2024-09-13T17:59:10.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.731+0000 7f6b36412640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3006cb60 con 0x7f6b30158b00 2024-09-13T17:59:10.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.731+0000 7f6b27fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6b3006c6d0 0x7f6b30070b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33580/0 (socket says 172.21.15.111:33580) 2024-09-13T17:59:10.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.731+0000 7f6b27fff640 1 -- 172.21.15.111:0/1997874064 learned_addr learned my addr 172.21.15.111:0/1997874064 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:10.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2ffff640 1 --2- 172.21.15.111:0/1997874064 >> v2:172.21.15.188:3300/0 conn(0x7f6b30158b00 0x7f6b3006c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2f7fe640 1 -- 172.21.15.111:0/1997874064 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075254584 0 0) 0x7f6b300b2d10 con 0x7f6b3006c6d0 2024-09-13T17:59:10.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2f7fe640 1 -- 172.21.15.111:0/1997874064 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b1c003520 con 0x7f6b3006c6d0 2024-09-13T17:59:10.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2f7fe640 1 -- 172.21.15.111:0/1997874064 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019413149 0 0) 0x7f6b300b2ff0 con 0x7f6b300b2170 2024-09-13T17:59:10.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2f7fe640 1 -- 172.21.15.111:0/1997874064 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b300b2d10 con 0x7f6b300b2170 2024-09-13T17:59:10.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2f7fe640 1 -- 172.21.15.111:0/1997874064 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2578751140 0 0) 0x7f6b1c003520 con 0x7f6b3006c6d0 2024-09-13T17:59:10.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2f7fe640 1 -- 172.21.15.111:0/1997874064 >> v2:172.21.15.188:3300/0 conn(0x7f6b30158b00 msgr2=0x7f6b3006c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:10.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2f7fe640 1 --2- 172.21.15.111:0/1997874064 >> v2:172.21.15.188:3300/0 conn(0x7f6b30158b00 0x7f6b3006c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2f7fe640 1 -- 172.21.15.111:0/1997874064 >> v1:172.21.15.188:6789/0 conn(0x7f6b300b2170 legacy=0x7f6b30071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2f7fe640 1 -- 172.21.15.111:0/1997874064 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b30070310 con 0x7f6b3006c6d0 2024-09-13T17:59:10.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2f7fe640 1 -- 172.21.15.111:0/1997874064 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f6b18001670 con 0x7f6b3006c6d0 2024-09-13T17:59:10.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2f7fe640 1 -- 172.21.15.111:0/1997874064 >> v1:172.21.15.111:6789/0 conn(0x7f6b3006c6d0 legacy=0x7f6b30070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2f7fe640 1 --2- 172.21.15.111:0/1997874064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b1c003b50 0x7f6b1c003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2f7fe640 1 --2- 172.21.15.111:0/1997874064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1c0047c0 0x7f6b1c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2f7fe640 1 --2- 172.21.15.111:0/1997874064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b1c005180 0x7f6b1c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2f7fe640 1 -- 172.21.15.111:0/1997874064 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3006cb60 con 0x7f6b1c003b50 2024-09-13T17:59:10.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2f7fe640 1 -- 172.21.15.111:0/1997874064 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c00c050 con 0x7f6b1c0047c0 2024-09-13T17:59:10.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2f7fe640 1 -- 172.21.15.111:0/1997874064 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c00c210 con 0x7f6b1c005180 2024-09-13T17:59:10.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2ffff640 1 --2- 172.21.15.111:0/1997874064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b1c003b50 0x7f6b1c003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b27fff640 1 --2- 172.21.15.111:0/1997874064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1c0047c0 0x7f6b1c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b349af640 1 --2- 172.21.15.111:0/1997874064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b1c005180 0x7f6b1c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2ffff640 1 -- 172.21.15.111:0/1997874064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b1c005180 msgr2=0x7f6b1c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:10.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2ffff640 1 --2- 172.21.15.111:0/1997874064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b1c005180 0x7f6b1c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2ffff640 1 -- 172.21.15.111:0/1997874064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1c0047c0 msgr2=0x7f6b1c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:10.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2ffff640 1 --2- 172.21.15.111:0/1997874064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1c0047c0 0x7f6b1c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b2ffff640 1 -- 172.21.15.111:0/1997874064 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c003520 con 0x7f6b1c003b50 2024-09-13T17:59:10.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b349af640 1 --2- 172.21.15.111:0/1997874064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b1c005180 0x7f6b1c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:10.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.732+0000 7f6b349af640 1 -- 172.21.15.111:0/1997874064 reap_dead start 2024-09-13T17:59:10.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.733+0000 7f6b2ffff640 1 --2- 172.21.15.111:0/1997874064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b1c003b50 0x7f6b1c003f50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009ac0 tx=0x7f6b1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3625836459152364 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:10.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.733+0000 7f6b2f7fe640 1 -- 172.21.15.111:0/1997874064 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20031400 con 0x7f6b1c003b50 2024-09-13T17:59:10.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.733+0000 7f6b2f7fe640 1 -- 172.21.15.111:0/1997874064 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2002d050 con 0x7f6b1c003b50 2024-09-13T17:59:10.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.733+0000 7f6b2f7fe640 1 -- 172.21.15.111:0/1997874064 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20031780 con 0x7f6b1c003b50 2024-09-13T17:59:10.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.734+0000 7f6b36412640 1 -- 172.21.15.111:0/1997874064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b1c003b50 msgr2=0x7f6b1c003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.734+0000 7f6b36412640 1 --2- 172.21.15.111:0/1997874064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b1c003b50 0x7f6b1c003f50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009ac0 tx=0x7f6b1c0057f0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.734+0000 7f6b36412640 1 -- 172.21.15.111:0/1997874064 shutdown_connections 2024-09-13T17:59:10.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.734+0000 7f6b36412640 1 --2- 172.21.15.111:0/1997874064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b1c003b50 0x7f6b1c003f50 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.734+0000 7f6b36412640 1 -- 172.21.15.111:0/1997874064 >> 172.21.15.111:0/1997874064 conn(0x7f6b3006a0b0 msgr2=0x7f6b3006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:10.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.734+0000 7f6b36412640 1 -- 172.21.15.111:0/1997874064 shutdown_connections 2024-09-13T17:59:10.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.734+0000 7f6b36412640 1 -- 172.21.15.111:0/1997874064 wait complete. 2024-09-13T17:59:10.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.735+0000 7f6b36412640 1 Processor -- start 2024-09-13T17:59:10.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.735+0000 7f6b36412640 1 -- start start 2024-09-13T17:59:10.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b36412640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b1c003b50 0x7f6b30097890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b36412640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b30070a80 0x7f6b30070f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b36412640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b301564f0 0x7f6b30156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b36412640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b30071440 con 0x7f6b1c003b50 2024-09-13T17:59:10.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b36412640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b30156eb0 con 0x7f6b301564f0 2024-09-13T17:59:10.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b36412640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b300acc50 con 0x7f6b30070a80 2024-09-13T17:59:10.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b2ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b1c003b50 0x7f6b30097890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b2ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b1c003b50 0x7f6b30097890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52580/0 (socket says 172.21.15.111:52580) 2024-09-13T17:59:10.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b2ffff640 1 -- 172.21.15.111:0/71666973 learned_addr learned my addr 172.21.15.111:0/71666973 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:10.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b349af640 1 --2- 172.21.15.111:0/71666973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b301564f0 0x7f6b30156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b27fff640 1 --2- 172.21.15.111:0/71666973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b30070a80 0x7f6b30070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b2ffff640 1 -- 172.21.15.111:0/71666973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b30070a80 msgr2=0x7f6b30070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b2ffff640 1 --2- 172.21.15.111:0/71666973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b30070a80 0x7f6b30070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b2ffff640 1 -- 172.21.15.111:0/71666973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b301564f0 msgr2=0x7f6b30156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b2ffff640 1 --2- 172.21.15.111:0/71666973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b301564f0 0x7f6b30156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b2ffff640 1 -- 172.21.15.111:0/71666973 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b301573b0 con 0x7f6b1c003b50 2024-09-13T17:59:10.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b349af640 1 --2- 172.21.15.111:0/71666973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b301564f0 0x7f6b30156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:10.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b2ffff640 1 --2- 172.21.15.111:0/71666973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b1c003b50 0x7f6b30097890 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6b20034040 tx=0x7f6b20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:10.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b2d7fa640 1 -- 172.21.15.111:0/71666973 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20003e90 con 0x7f6b1c003b50 2024-09-13T17:59:10.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b2d7fa640 1 -- 172.21.15.111:0/71666973 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2003b400 con 0x7f6b1c003b50 2024-09-13T17:59:10.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b36412640 1 -- 172.21.15.111:0/71666973 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b301570d0 con 0x7f6b1c003b50 2024-09-13T17:59:10.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b2d7fa640 1 -- 172.21.15.111:0/71666973 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2003b790 con 0x7f6b1c003b50 2024-09-13T17:59:10.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.736+0000 7f6b36412640 1 -- 172.21.15.111:0/71666973 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b30098660 con 0x7f6b1c003b50 2024-09-13T17:59:10.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.737+0000 7f6b36412640 1 -- 172.21.15.111:0/71666973 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b300b2ff0 con 0x7f6b1c003b50 2024-09-13T17:59:10.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.738+0000 7f6b2d7fa640 1 -- 172.21.15.111:0/71666973 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6b20005990 con 0x7f6b1c003b50 2024-09-13T17:59:10.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.738+0000 7f6b2d7fa640 1 --2- 172.21.15.111:0/71666973 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b1006ac30 0x7f6b1006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.738+0000 7f6b2d7fa640 1 -- 172.21.15.111:0/71666973 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f6b200a9760 con 0x7f6b1c003b50 2024-09-13T17:59:10.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.738+0000 7f6b27fff640 1 --2- 172.21.15.111:0/71666973 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b1006ac30 0x7f6b1006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.739+0000 7f6b27fff640 1 --2- 172.21.15.111:0/71666973 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b1006ac30 0x7f6b1006d0f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6b18000c00 tx=0x7f6b18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:10.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.742+0000 7f6b2d7fa640 1 -- 172.21.15.111:0/71666973 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f6b20074820 con 0x7f6b1c003b50 2024-09-13T17:59:10.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.903+0000 7f6b36412640 1 -- 172.21.15.111:0/71666973 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6b300cb9f0 con 0x7f6b1c003b50 2024-09-13T17:59:10.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.904+0000 7f6b2d7fa640 1 -- 172.21.15.111:0/71666973 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6b20077e40 con 0x7f6b1c003b50 2024-09-13T17:59:10.906 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-13T17:59:10.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.906+0000 7f6b36412640 1 -- 172.21.15.111:0/71666973 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b1006ac30 msgr2=0x7f6b1006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.906+0000 7f6b36412640 1 --2- 172.21.15.111:0/71666973 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b1006ac30 0x7f6b1006d0f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6b18000c00 tx=0x7f6b18009040 comp rx=0 tx=0).stop 2024-09-13T17:59:10.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.906+0000 7f6b36412640 1 -- 172.21.15.111:0/71666973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b1c003b50 msgr2=0x7f6b30097890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.906+0000 7f6b36412640 1 --2- 172.21.15.111:0/71666973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b1c003b50 0x7f6b30097890 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6b20034040 tx=0x7f6b20004730 comp rx=0 tx=0).stop 2024-09-13T17:59:10.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.906+0000 7f6b36412640 1 -- 172.21.15.111:0/71666973 shutdown_connections 2024-09-13T17:59:10.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.906+0000 7f6b36412640 1 --2- 172.21.15.111:0/71666973 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6b1006ac30 0x7f6b1006d0f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.906+0000 7f6b36412640 1 --2- 172.21.15.111:0/71666973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b301564f0 0x7f6b30156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.906+0000 7f6b36412640 1 --2- 172.21.15.111:0/71666973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b30070a80 0x7f6b30070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.906+0000 7f6b36412640 1 --2- 172.21.15.111:0/71666973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b1c003b50 0x7f6b30097890 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.906+0000 7f6b36412640 1 -- 172.21.15.111:0/71666973 >> 172.21.15.111:0/71666973 conn(0x7f6b3006a0b0 msgr2=0x7f6b30071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:10.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.906+0000 7f6b36412640 1 -- 172.21.15.111:0/71666973 shutdown_connections 2024-09-13T17:59:10.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:10.906+0000 7f6b36412640 1 -- 172.21.15.111:0/71666973 wait complete. 2024-09-13T17:59:10.919 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:59:11.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.016+0000 7f3866565640 1 Processor -- start 2024-09-13T17:59:11.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.016+0000 7f3866565640 1 -- start start 2024-09-13T17:59:11.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.017+0000 7f3866565640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3860158b00 0x7f386006c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.017+0000 7f3866565640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f386006c6d0 0x7f3860070b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.017+0000 7f3866565640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38600b2d10 con 0x7f3860071050 2024-09-13T17:59:11.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.017+0000 7f3866565640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386006cb60 con 0x7f3860158b00 2024-09-13T17:59:11.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.017+0000 7f3866565640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38600b2ff0 con 0x7f386006c6d0 2024-09-13T17:59:11.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.017+0000 7f385ffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3860158b00 0x7f386006c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.017+0000 7f3864b02640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3860071050 0x7f38600714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33588/0 (socket says 172.21.15.111:33588) 2024-09-13T17:59:11.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.017+0000 7f385ffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3860158b00 0x7f386006c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52594/0 (socket says 172.21.15.111:52594) 2024-09-13T17:59:11.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.017+0000 7f3864b02640 1 -- 172.21.15.111:0/2137493155 learned_addr learned my addr 172.21.15.111:0/2137493155 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:11.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.017+0000 7f385f7fe640 1 --2- 172.21.15.111:0/2137493155 >> v2:172.21.15.188:3300/0 conn(0x7f386006c6d0 0x7f3860070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.017+0000 7f385ffff640 1 -- 172.21.15.111:0/2137493155 >> v2:172.21.15.188:3300/0 conn(0x7f386006c6d0 msgr2=0x7f3860070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:11.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.017+0000 7f385ffff640 1 --2- 172.21.15.111:0/2137493155 >> v2:172.21.15.188:3300/0 conn(0x7f386006c6d0 0x7f3860070b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.017+0000 7f385ffff640 1 -- 172.21.15.111:0/2137493155 >> v1:172.21.15.111:6789/0 conn(0x7f3860071050 legacy=0x7f38600714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.017+0000 7f385effd640 1 -- 172.21.15.111:0/2137493155 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3061520807 0 0) 0x7f38600b2d10 con 0x7f3860071050 2024-09-13T17:59:11.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.017+0000 7f385ffff640 1 -- 172.21.15.111:0/2137493155 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3860070310 con 0x7f3860158b00 2024-09-13T17:59:11.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.018+0000 7f385ffff640 1 --2- 172.21.15.111:0/2137493155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3860158b00 0x7f386006c190 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3860070140 tx=0x7f385002b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba60426821d17813 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:11.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.018+0000 7f385effd640 1 -- 172.21.15.111:0/2137493155 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850030710 con 0x7f3860158b00 2024-09-13T17:59:11.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.018+0000 7f385effd640 1 -- 172.21.15.111:0/2137493155 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385002e050 con 0x7f3860158b00 2024-09-13T17:59:11.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.018+0000 7f385effd640 1 -- 172.21.15.111:0/2137493155 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850030a90 con 0x7f3860158b00 2024-09-13T17:59:11.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.019+0000 7f3866565640 1 -- 172.21.15.111:0/2137493155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3860158b00 msgr2=0x7f386006c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.019+0000 7f3866565640 1 --2- 172.21.15.111:0/2137493155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3860158b00 0x7f386006c190 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3860070140 tx=0x7f385002b560 comp rx=0 tx=0).stop 2024-09-13T17:59:11.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.019+0000 7f3866565640 1 -- 172.21.15.111:0/2137493155 shutdown_connections 2024-09-13T17:59:11.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.019+0000 7f3866565640 1 --2- 172.21.15.111:0/2137493155 >> v2:172.21.15.188:3300/0 conn(0x7f386006c6d0 0x7f3860070b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.019+0000 7f3866565640 1 --2- 172.21.15.111:0/2137493155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3860158b00 0x7f386006c190 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.019+0000 7f3866565640 1 -- 172.21.15.111:0/2137493155 >> 172.21.15.111:0/2137493155 conn(0x7f386006a0b0 msgr2=0x7f386006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:11.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.019+0000 7f3866565640 1 -- 172.21.15.111:0/2137493155 shutdown_connections 2024-09-13T17:59:11.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.019+0000 7f3866565640 1 -- 172.21.15.111:0/2137493155 wait complete. 2024-09-13T17:59:11.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.020+0000 7f3866565640 1 Processor -- start 2024-09-13T17:59:11.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.020+0000 7f3866565640 1 -- start start 2024-09-13T17:59:11.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f3866565640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f386006c6d0 0x7f3860157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f3866565640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3860071050 0x7f38601e3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f3866565640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3860158b00 0x7f38601e3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f3866565640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386009b0a0 con 0x7f3860071050 2024-09-13T17:59:11.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f3866565640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38600b1cf0 con 0x7f3860158b00 2024-09-13T17:59:11.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f3866565640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3860099170 con 0x7f386006c6d0 2024-09-13T17:59:11.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f385f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3860071050 0x7f38601e3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f385f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3860071050 0x7f38601e3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52604/0 (socket says 172.21.15.111:52604) 2024-09-13T17:59:11.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f385f7fe640 1 -- 172.21.15.111:0/1823426762 learned_addr learned my addr 172.21.15.111:0/1823426762 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:11.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f385ffff640 1 --2- 172.21.15.111:0/1823426762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f386006c6d0 0x7f3860157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f3864b02640 1 --2- 172.21.15.111:0/1823426762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3860158b00 0x7f38601e3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f385f7fe640 1 -- 172.21.15.111:0/1823426762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f386006c6d0 msgr2=0x7f3860157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f385f7fe640 1 --2- 172.21.15.111:0/1823426762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f386006c6d0 0x7f3860157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f385f7fe640 1 -- 172.21.15.111:0/1823426762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3860158b00 msgr2=0x7f38601e3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f385f7fe640 1 --2- 172.21.15.111:0/1823426762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3860158b00 0x7f38601e3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f385f7fe640 1 -- 172.21.15.111:0/1823426762 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385000a5a0 con 0x7f3860071050 2024-09-13T17:59:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f385ffff640 1 --2- 172.21.15.111:0/1823426762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f386006c6d0 0x7f3860157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f385f7fe640 1 --2- 172.21.15.111:0/1823426762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3860071050 0x7f38601e3820 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f385400c820 tx=0x7f385400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f385cff9640 1 -- 172.21.15.111:0/1823426762 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854004220 con 0x7f3860071050 2024-09-13T17:59:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f385cff9640 1 -- 172.21.15.111:0/1823426762 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38540043a0 con 0x7f3860071050 2024-09-13T17:59:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f3866565640 1 -- 172.21.15.111:0/1823426762 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386009b280 con 0x7f3860071050 2024-09-13T17:59:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f385cff9640 1 -- 172.21.15.111:0/1823426762 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854007a10 con 0x7f3860071050 2024-09-13T17:59:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.021+0000 7f3866565640 1 -- 172.21.15.111:0/1823426762 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386009b840 con 0x7f3860071050 2024-09-13T17:59:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.022+0000 7f3866565640 1 -- 172.21.15.111:0/1823426762 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3860070b10 con 0x7f3860071050 2024-09-13T17:59:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.023+0000 7f385cff9640 1 -- 172.21.15.111:0/1823426762 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3854007b70 con 0x7f3860071050 2024-09-13T17:59:11.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.023+0000 7f385cff9640 1 --2- 172.21.15.111:0/1823426762 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f384c06ac00 0x7f384c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.024+0000 7f385ffff640 1 --2- 172.21.15.111:0/1823426762 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f384c06ac00 0x7f384c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.024+0000 7f385cff9640 1 -- 172.21.15.111:0/1823426762 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f3854089270 con 0x7f3860071050 2024-09-13T17:59:11.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.024+0000 7f385ffff640 1 --2- 172.21.15.111:0/1823426762 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f384c06ac00 0x7f384c06d0c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3850035000 tx=0x7f3850008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:11.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.028+0000 7f385cff9640 1 -- 172.21.15.111:0/1823426762 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f3854054330 con 0x7f3860071050 2024-09-13T17:59:11.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.188+0000 7f3866565640 1 -- 172.21.15.111:0/1823426762 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f386006c1c0 con 0x7f3860071050 2024-09-13T17:59:11.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.189+0000 7f385cff9640 1 -- 172.21.15.111:0/1823426762 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3854057950 con 0x7f3860071050 2024-09-13T17:59:11.191 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-13T17:59:11.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.191+0000 7f3866565640 1 -- 172.21.15.111:0/1823426762 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f384c06ac00 msgr2=0x7f384c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.191+0000 7f3866565640 1 --2- 172.21.15.111:0/1823426762 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f384c06ac00 0x7f384c06d0c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3850035000 tx=0x7f3850008040 comp rx=0 tx=0).stop 2024-09-13T17:59:11.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.191+0000 7f3866565640 1 -- 172.21.15.111:0/1823426762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3860071050 msgr2=0x7f38601e3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.191+0000 7f3866565640 1 --2- 172.21.15.111:0/1823426762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3860071050 0x7f38601e3820 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f385400c820 tx=0x7f385400cce0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.191+0000 7f3866565640 1 -- 172.21.15.111:0/1823426762 shutdown_connections 2024-09-13T17:59:11.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.191+0000 7f3866565640 1 --2- 172.21.15.111:0/1823426762 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f384c06ac00 0x7f384c06d0c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.191+0000 7f3866565640 1 --2- 172.21.15.111:0/1823426762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3860158b00 0x7f38601e3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.191+0000 7f3866565640 1 --2- 172.21.15.111:0/1823426762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3860071050 0x7f38601e3820 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.191+0000 7f3866565640 1 --2- 172.21.15.111:0/1823426762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f386006c6d0 0x7f3860157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.191+0000 7f3866565640 1 -- 172.21.15.111:0/1823426762 >> 172.21.15.111:0/1823426762 conn(0x7f386006a0b0 msgr2=0x7f38600ac930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:11.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.191+0000 7f3866565640 1 -- 172.21.15.111:0/1823426762 shutdown_connections 2024-09-13T17:59:11.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.191+0000 7f3866565640 1 -- 172.21.15.111:0/1823426762 wait complete. 2024-09-13T17:59:11.204 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:59:11.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.299+0000 7fc6a33e2640 1 Processor -- start 2024-09-13T17:59:11.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.299+0000 7fc6a33e2640 1 -- start start 2024-09-13T17:59:11.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.300+0000 7fc6a33e2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc69c06c4c0 0x7fc69c06c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.300+0000 7fc6a33e2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc69c0b2590 0x7fc69c06d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.300+0000 7fc6a33e2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc69c0cbea0 0x7fc69c0cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.300+0000 7fc6a33e2640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69c06deb0 con 0x7fc69c0b2590 2024-09-13T17:59:11.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.300+0000 7fc6a33e2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69c0cc890 con 0x7fc69c0cbea0 2024-09-13T17:59:11.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.300+0000 7fc6a33e2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c06e0c0 con 0x7fc69c06c4c0 2024-09-13T17:59:11.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.300+0000 7fc6a097d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc69c0b2590 0x7fc69c06d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.300+0000 7fc6a097d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc69c0b2590 0x7fc69c06d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52620/0 (socket says 172.21.15.111:52620) 2024-09-13T17:59:11.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.300+0000 7fc6a097d640 1 -- 172.21.15.111:0/4139663837 learned_addr learned my addr 172.21.15.111:0/4139663837 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:11.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.300+0000 7fc6a097d640 1 -- 172.21.15.111:0/4139663837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc69c06c4c0 msgr2=0x7fc69c06c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:11.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.300+0000 7fc6a117e640 1 --2- 172.21.15.111:0/4139663837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc69c06c4c0 0x7fc69c06c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.300+0000 7fc6a197f640 1 --2- 172.21.15.111:0/4139663837 >> v2:172.21.15.188:3300/0 conn(0x7fc69c0cbea0 0x7fc69c0cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.300+0000 7fc6a097d640 1 --2- 172.21.15.111:0/4139663837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc69c06c4c0 0x7fc69c06c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.300+0000 7fc6a097d640 1 -- 172.21.15.111:0/4139663837 >> v2:172.21.15.188:3300/0 conn(0x7fc69c0cbea0 msgr2=0x7fc69c0cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:11.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.300+0000 7fc6a097d640 1 --2- 172.21.15.111:0/4139663837 >> v2:172.21.15.188:3300/0 conn(0x7fc69c0cbea0 0x7fc69c0cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.300+0000 7fc6a097d640 1 -- 172.21.15.111:0/4139663837 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c06e3a0 con 0x7fc69c0b2590 2024-09-13T17:59:11.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.301+0000 7fc6a097d640 1 --2- 172.21.15.111:0/4139663837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc69c0b2590 0x7fc69c06d3a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc69000aa10 tx=0x7fc69002b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2c99bfad95e7d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:11.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.301+0000 7fc69bfff640 1 -- 172.21.15.111:0/4139663837 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc690030710 con 0x7fc69c0b2590 2024-09-13T17:59:11.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.301+0000 7fc69bfff640 1 -- 172.21.15.111:0/4139663837 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69002e050 con 0x7fc69c0b2590 2024-09-13T17:59:11.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.301+0000 7fc69bfff640 1 -- 172.21.15.111:0/4139663837 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc690030a90 con 0x7fc69c0b2590 2024-09-13T17:59:11.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.302+0000 7fc6a33e2640 1 -- 172.21.15.111:0/4139663837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc69c0b2590 msgr2=0x7fc69c06d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.302+0000 7fc6a33e2640 1 --2- 172.21.15.111:0/4139663837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc69c0b2590 0x7fc69c06d3a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc69000aa10 tx=0x7fc69002b5d0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.302+0000 7fc6a33e2640 1 -- 172.21.15.111:0/4139663837 shutdown_connections 2024-09-13T17:59:11.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.302+0000 7fc6a33e2640 1 --2- 172.21.15.111:0/4139663837 >> v2:172.21.15.188:3300/0 conn(0x7fc69c0cbea0 0x7fc69c0cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.302+0000 7fc6a33e2640 1 --2- 172.21.15.111:0/4139663837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc69c0b2590 0x7fc69c06d3a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.302+0000 7fc6a33e2640 1 --2- 172.21.15.111:0/4139663837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc69c06c4c0 0x7fc69c06c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.302+0000 7fc6a33e2640 1 -- 172.21.15.111:0/4139663837 >> 172.21.15.111:0/4139663837 conn(0x7fc69c069ed0 msgr2=0x7fc69c06a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:11.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.302+0000 7fc6a33e2640 1 -- 172.21.15.111:0/4139663837 shutdown_connections 2024-09-13T17:59:11.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.302+0000 7fc6a33e2640 1 -- 172.21.15.111:0/4139663837 wait complete. 2024-09-13T17:59:11.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.303+0000 7fc6a33e2640 1 Processor -- start 2024-09-13T17:59:11.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.303+0000 7fc6a33e2640 1 -- start start 2024-09-13T17:59:11.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a33e2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc69c06c4c0 0x7fc69c0c0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a33e2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc69c0b2590 0x7fc69c0c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a33e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc69c0cbea0 0x7fc69c1f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a33e2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0c53a0 con 0x7fc69c06c4c0 2024-09-13T17:59:11.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a33e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0c5960 con 0x7fc69c0cbea0 2024-09-13T17:59:11.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a33e2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c1f3a70 con 0x7fc69c0b2590 2024-09-13T17:59:11.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a117e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc69c06c4c0 0x7fc69c0c0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a117e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc69c06c4c0 0x7fc69c0c0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52630/0 (socket says 172.21.15.111:52630) 2024-09-13T17:59:11.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a117e640 1 -- 172.21.15.111:0/527910238 learned_addr learned my addr 172.21.15.111:0/527910238 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:11.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a197f640 1 --2- 172.21.15.111:0/527910238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc69c0cbea0 0x7fc69c1f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a097d640 1 --2- 172.21.15.111:0/527910238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc69c0b2590 0x7fc69c0c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a117e640 1 -- 172.21.15.111:0/527910238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc69c0b2590 msgr2=0x7fc69c0c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a117e640 1 --2- 172.21.15.111:0/527910238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc69c0b2590 0x7fc69c0c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a117e640 1 -- 172.21.15.111:0/527910238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc69c0cbea0 msgr2=0x7fc69c1f2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a117e640 1 --2- 172.21.15.111:0/527910238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc69c0cbea0 0x7fc69c1f2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a117e640 1 -- 172.21.15.111:0/527910238 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69000a670 con 0x7fc69c06c4c0 2024-09-13T17:59:11.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a197f640 1 --2- 172.21.15.111:0/527910238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc69c0cbea0 0x7fc69c1f2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:11.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a097d640 1 --2- 172.21.15.111:0/527910238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc69c0b2590 0x7fc69c0c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:11.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a117e640 1 --2- 172.21.15.111:0/527910238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc69c06c4c0 0x7fc69c0c0be0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00ce10 tx=0x7fc68c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:11.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc699ffb640 1 -- 172.21.15.111:0/527910238 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c003b90 con 0x7fc69c06c4c0 2024-09-13T17:59:11.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc699ffb640 1 -- 172.21.15.111:0/527910238 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c003cf0 con 0x7fc69c06c4c0 2024-09-13T17:59:11.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc6a33e2640 1 -- 172.21.15.111:0/527910238 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69c1f3d50 con 0x7fc69c06c4c0 2024-09-13T17:59:11.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.304+0000 7fc699ffb640 1 -- 172.21.15.111:0/527910238 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c003ea0 con 0x7fc69c06c4c0 2024-09-13T17:59:11.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.305+0000 7fc6a33e2640 1 -- 172.21.15.111:0/527910238 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69c0c1120 con 0x7fc69c06c4c0 2024-09-13T17:59:11.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.306+0000 7fc6a33e2640 1 -- 172.21.15.111:0/527910238 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69c06d3a0 con 0x7fc69c06c4c0 2024-09-13T17:59:11.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.306+0000 7fc699ffb640 1 -- 172.21.15.111:0/527910238 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc68c004240 con 0x7fc69c06c4c0 2024-09-13T17:59:11.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.306+0000 7fc699ffb640 1 --2- 172.21.15.111:0/527910238 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc68806ac00 0x7fc68806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.306+0000 7fc699ffb640 1 -- 172.21.15.111:0/527910238 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fc68c012070 con 0x7fc69c06c4c0 2024-09-13T17:59:11.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.306+0000 7fc6a097d640 1 --2- 172.21.15.111:0/527910238 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc68806ac00 0x7fc68806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.307+0000 7fc6a097d640 1 --2- 172.21.15.111:0/527910238 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc68806ac00 0x7fc68806d0c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc690035000 tx=0x7fc690008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:11.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.310+0000 7fc699ffb640 1 -- 172.21.15.111:0/527910238 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc68c054f60 con 0x7fc69c06c4c0 2024-09-13T17:59:11.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.503+0000 7fc6a33e2640 1 -- 172.21.15.111:0/527910238 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc69c0c6330 con 0x7fc69c06c4c0 2024-09-13T17:59:11.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.504+0000 7fc699ffb640 1 -- 172.21.15.111:0/527910238 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4742 (secure 0 0 0) 0x7fc68c058580 con 0x7fc69c06c4c0 2024-09-13T17:59:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:11.506 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4217138481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4217138481},{"type":"v1","addr":"172.21.15.188:6834","nonce":4217138481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3099948287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3099948287},{"type":"v1","addr":"172.21.15.188:6835","nonce":3099948287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/904506495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":904506495},{"type":"v1","addr":"172.21.15.111:6837","nonce":904506495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:58:43.534567+0000","modified":"2024-09-13T17:59:05.554735+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5487":{"gid":5487,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6835/214034224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":214034224},{"type":"v1","addr":"172.21.15.111:6835","nonce":214034224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:59:11.506 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 21 2024-09-13T17:59:11.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.505+0000 7fc6a33e2640 1 -- 172.21.15.111:0/527910238 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc68806ac00 msgr2=0x7fc68806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.506+0000 7fc6a33e2640 1 --2- 172.21.15.111:0/527910238 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc68806ac00 0x7fc68806d0c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc690035000 tx=0x7fc690008040 comp rx=0 tx=0).stop 2024-09-13T17:59:11.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.506+0000 7fc6a33e2640 1 -- 172.21.15.111:0/527910238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc69c06c4c0 msgr2=0x7fc69c0c0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.506+0000 7fc6a33e2640 1 --2- 172.21.15.111:0/527910238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc69c06c4c0 0x7fc69c0c0be0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00ce10 tx=0x7fc68c00a660 comp rx=0 tx=0).stop 2024-09-13T17:59:11.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.506+0000 7fc6a33e2640 1 -- 172.21.15.111:0/527910238 shutdown_connections 2024-09-13T17:59:11.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.506+0000 7fc6a33e2640 1 --2- 172.21.15.111:0/527910238 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc68806ac00 0x7fc68806d0c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.506+0000 7fc6a33e2640 1 --2- 172.21.15.111:0/527910238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc69c0cbea0 0x7fc69c1f2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.506+0000 7fc6a33e2640 1 --2- 172.21.15.111:0/527910238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc69c0b2590 0x7fc69c0c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.506+0000 7fc6a33e2640 1 --2- 172.21.15.111:0/527910238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc69c06c4c0 0x7fc69c0c0be0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.506+0000 7fc6a33e2640 1 -- 172.21.15.111:0/527910238 >> 172.21.15.111:0/527910238 conn(0x7fc69c069ed0 msgr2=0x7fc69c06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:11.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.506+0000 7fc6a33e2640 1 -- 172.21.15.111:0/527910238 shutdown_connections 2024-09-13T17:59:11.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.506+0000 7fc6a33e2640 1 -- 172.21.15.111:0/527910238 wait complete. 2024-09-13T17:59:11.518 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:59:11.616 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.614+0000 7ff36c280640 1 Processor -- start 2024-09-13T17:59:11.616 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.614+0000 7ff36c280640 1 -- start start 2024-09-13T17:59:11.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.615+0000 7ff36c280640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff364071d30 0x7ff364070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.615+0000 7ff36c280640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff364072110 0x7ff3640b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.615+0000 7ff36c280640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3640abdf0 con 0x7ff3640c8920 2024-09-13T17:59:11.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.615+0000 7ff36c280640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3640ab1e0 con 0x7ff364072110 2024-09-13T17:59:11.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.615+0000 7ff36c280640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3640ab4c0 con 0x7ff364071d30 2024-09-13T17:59:11.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.615+0000 7ff36981b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff364072110 0x7ff3640b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.615+0000 7ff36981b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff364072110 0x7ff3640b1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52634/0 (socket says 172.21.15.111:52634) 2024-09-13T17:59:11.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.615+0000 7ff36981b640 1 -- 172.21.15.111:0/3050053688 learned_addr learned my addr 172.21.15.111:0/3050053688 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:11.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.615+0000 7ff36a01c640 1 --2- 172.21.15.111:0/3050053688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff364071d30 0x7ff364070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.615+0000 7ff36981b640 1 -- 172.21.15.111:0/3050053688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff364071d30 msgr2=0x7ff364070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:11.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.615+0000 7ff36981b640 1 --2- 172.21.15.111:0/3050053688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff364071d30 0x7ff364070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.615+0000 7ff36981b640 1 -- 172.21.15.111:0/3050053688 >> v1:172.21.15.188:6789/0 conn(0x7ff3640c8920 legacy=0x7ff3640ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:11.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.615+0000 7ff36981b640 1 -- 172.21.15.111:0/3050053688 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3640ab7a0 con 0x7ff364072110 2024-09-13T17:59:11.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.615+0000 7ff36a01c640 1 --2- 172.21.15.111:0/3050053688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff364071d30 0x7ff364070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:59:11.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.616+0000 7ff36981b640 1 --2- 172.21.15.111:0/3050053688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff364072110 0x7ff3640b1b70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff3540098a0 tx=0x7ff35402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8246814dbcca5bd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:11.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.616+0000 7ff36901a640 1 -- 172.21.15.111:0/3050053688 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff354032440 con 0x7ff364072110 2024-09-13T17:59:11.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.616+0000 7ff36901a640 1 -- 172.21.15.111:0/3050053688 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3540325a0 con 0x7ff364072110 2024-09-13T17:59:11.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.616+0000 7ff36901a640 1 -- 172.21.15.111:0/3050053688 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff354032930 con 0x7ff364072110 2024-09-13T17:59:11.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.617+0000 7ff36c280640 1 -- 172.21.15.111:0/3050053688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff364072110 msgr2=0x7ff3640b1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.617+0000 7ff36c280640 1 --2- 172.21.15.111:0/3050053688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff364072110 0x7ff3640b1b70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff3540098a0 tx=0x7ff35402a250 comp rx=0 tx=0).stop 2024-09-13T17:59:11.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.617+0000 7ff36c280640 1 -- 172.21.15.111:0/3050053688 shutdown_connections 2024-09-13T17:59:11.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.617+0000 7ff36c280640 1 --2- 172.21.15.111:0/3050053688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff364072110 0x7ff3640b1b70 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.617+0000 7ff36c280640 1 --2- 172.21.15.111:0/3050053688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff364071d30 0x7ff364070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.617+0000 7ff36c280640 1 -- 172.21.15.111:0/3050053688 >> 172.21.15.111:0/3050053688 conn(0x7ff364069ed0 msgr2=0x7ff36406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:11.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.617+0000 7ff36c280640 1 -- 172.21.15.111:0/3050053688 shutdown_connections 2024-09-13T17:59:11.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.617+0000 7ff36c280640 1 -- 172.21.15.111:0/3050053688 wait complete. 2024-09-13T17:59:11.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.618+0000 7ff36c280640 1 Processor -- start 2024-09-13T17:59:11.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.618+0000 7ff36c280640 1 -- start start 2024-09-13T17:59:11.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.619+0000 7ff36c280640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff364071d30 0x7ff36406dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.619+0000 7ff36c280640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff364072110 0x7ff3640cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.619+0000 7ff36c280640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3640c8920 0x7ff364152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.619+0000 7ff36c280640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3640c7e60 con 0x7ff364072110 2024-09-13T17:59:11.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.619+0000 7ff36c280640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3640c83e0 con 0x7ff364071d30 2024-09-13T17:59:11.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.619+0000 7ff36c280640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff36406e1e0 con 0x7ff3640c8920 2024-09-13T17:59:11.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.619+0000 7ff36981b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff364072110 0x7ff3640cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.619+0000 7ff36981b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff364072110 0x7ff3640cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52644/0 (socket says 172.21.15.111:52644) 2024-09-13T17:59:11.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.619+0000 7ff36981b640 1 -- 172.21.15.111:0/253306192 learned_addr learned my addr 172.21.15.111:0/253306192 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:11.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.619+0000 7ff36a01c640 1 --2- 172.21.15.111:0/253306192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff364071d30 0x7ff36406dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.619+0000 7ff36a81d640 1 --2- 172.21.15.111:0/253306192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3640c8920 0x7ff364152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.619+0000 7ff36981b640 1 -- 172.21.15.111:0/253306192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3640c8920 msgr2=0x7ff364152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.619+0000 7ff36981b640 1 --2- 172.21.15.111:0/253306192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3640c8920 0x7ff364152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.619+0000 7ff36981b640 1 -- 172.21.15.111:0/253306192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff364071d30 msgr2=0x7ff36406dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.619+0000 7ff36981b640 1 --2- 172.21.15.111:0/253306192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff364071d30 0x7ff36406dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.619+0000 7ff36981b640 1 -- 172.21.15.111:0/253306192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3540094d0 con 0x7ff364072110 2024-09-13T17:59:11.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.619+0000 7ff36981b640 1 --2- 172.21.15.111:0/253306192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff364072110 0x7ff3640cd670 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff354000c00 tx=0x7ff3540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:11.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.620+0000 7ff35affd640 1 -- 172.21.15.111:0/253306192 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff354032440 con 0x7ff364072110 2024-09-13T17:59:11.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.620+0000 7ff35affd640 1 -- 172.21.15.111:0/253306192 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff354030ba0 con 0x7ff364072110 2024-09-13T17:59:11.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.620+0000 7ff36c280640 1 -- 172.21.15.111:0/253306192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3640ce050 con 0x7ff364072110 2024-09-13T17:59:11.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.620+0000 7ff35affd640 1 -- 172.21.15.111:0/253306192 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff354004030 con 0x7ff364072110 2024-09-13T17:59:11.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.620+0000 7ff36c280640 1 -- 172.21.15.111:0/253306192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3640c8070 con 0x7ff364072110 2024-09-13T17:59:11.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.621+0000 7ff36c280640 1 -- 172.21.15.111:0/253306192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff364070190 con 0x7ff364072110 2024-09-13T17:59:11.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.621+0000 7ff35affd640 1 -- 172.21.15.111:0/253306192 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff354039040 con 0x7ff364072110 2024-09-13T17:59:11.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.621+0000 7ff35affd640 1 --2- 172.21.15.111:0/253306192 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff34806ac00 0x7ff34806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.621+0000 7ff36a01c640 1 --2- 172.21.15.111:0/253306192 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff34806ac00 0x7ff34806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.621+0000 7ff35affd640 1 -- 172.21.15.111:0/253306192 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7ff3540a9f70 con 0x7ff364072110 2024-09-13T17:59:11.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.622+0000 7ff36a01c640 1 --2- 172.21.15.111:0/253306192 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff34806ac00 0x7ff34806d0c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff364155a30 tx=0x7ff360008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:11.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.625+0000 7ff35affd640 1 -- 172.21.15.111:0/253306192 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff3540dd9c0 con 0x7ff364072110 2024-09-13T17:59:11.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.788+0000 7ff36c280640 1 -- 172.21.15.111:0/253306192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff36415cb20 con 0x7ff364072110 2024-09-13T17:59:11.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.789+0000 7ff35affd640 1 -- 172.21.15.111:0/253306192 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+17012 (secure 0 0 0) 0x7ff354075050 con 0x7ff364072110 2024-09-13T17:59:11.791 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:11.791 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":42,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:59:08.617279+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:58:41.698998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T17:58:42.499870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1c","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6837/982359636":"2024-09-14T17:59:01.542287+0000","172.21.15.111:6836/982359636":"2024-09-14T17:59:01.542287+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:59:11.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.791+0000 7ff36c280640 1 -- 172.21.15.111:0/253306192 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff34806ac00 msgr2=0x7ff34806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.791+0000 7ff36c280640 1 --2- 172.21.15.111:0/253306192 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff34806ac00 0x7ff34806d0c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff364155a30 tx=0x7ff360008040 comp rx=0 tx=0).stop 2024-09-13T17:59:11.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.791+0000 7ff36c280640 1 -- 172.21.15.111:0/253306192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff364072110 msgr2=0x7ff3640cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.791+0000 7ff36c280640 1 --2- 172.21.15.111:0/253306192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff364072110 0x7ff3640cd670 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff354000c00 tx=0x7ff3540046c0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.791+0000 7ff36c280640 1 -- 172.21.15.111:0/253306192 shutdown_connections 2024-09-13T17:59:11.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.791+0000 7ff36c280640 1 --2- 172.21.15.111:0/253306192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3640c8920 0x7ff364152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.791+0000 7ff36c280640 1 --2- 172.21.15.111:0/253306192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff364072110 0x7ff3640cd670 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.791+0000 7ff36c280640 1 --2- 172.21.15.111:0/253306192 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff34806ac00 0x7ff34806d0c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.791+0000 7ff36c280640 1 --2- 172.21.15.111:0/253306192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff364071d30 0x7ff36406dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.791+0000 7ff36c280640 1 -- 172.21.15.111:0/253306192 >> 172.21.15.111:0/253306192 conn(0x7ff364069ed0 msgr2=0x7ff364155a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:11.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.791+0000 7ff36c280640 1 -- 172.21.15.111:0/253306192 shutdown_connections 2024-09-13T17:59:11.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.791+0000 7ff36c280640 1 -- 172.21.15.111:0/253306192 wait complete. 2024-09-13T17:59:11.804 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T17:59:11.804 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:59:11.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.900+0000 7fc946745640 1 Processor -- start 2024-09-13T17:59:11.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.900+0000 7fc946745640 1 -- start start 2024-09-13T17:59:11.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.901+0000 7fc946745640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc94015a2e0 0x7fc9400c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.901+0000 7fc946745640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc9400cb3f0 0x7fc94006d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.901+0000 7fc946745640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9400c8920 con 0x7fc9400c9f80 2024-09-13T17:59:11.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.901+0000 7fc946745640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc94015a6c0 con 0x7fc9400cb3f0 2024-09-13T17:59:11.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.901+0000 7fc946745640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9400ca500 con 0x7fc94015a2e0 2024-09-13T17:59:11.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.901+0000 7fc944ce2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc9400cb3f0 0x7fc94006d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.901+0000 7fc93f7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc9400c9f80 0x7fc9400cace0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33590/0 (socket says 172.21.15.111:33590) 2024-09-13T17:59:11.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.901+0000 7fc93f7fe640 1 -- 172.21.15.111:0/2959100151 learned_addr learned my addr 172.21.15.111:0/2959100151 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:11.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.901+0000 7fc93ffff640 1 --2- 172.21.15.111:0/2959100151 >> v2:172.21.15.188:3300/0 conn(0x7fc94015a2e0 0x7fc9400c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.901+0000 7fc93effd640 1 -- 172.21.15.111:0/2959100151 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651705868 0 0) 0x7fc9400c8920 con 0x7fc9400c9f80 2024-09-13T17:59:11.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.901+0000 7fc93effd640 1 -- 172.21.15.111:0/2959100151 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc928003520 con 0x7fc9400c9f80 2024-09-13T17:59:11.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.901+0000 7fc944ce2640 1 -- 172.21.15.111:0/2959100151 >> v2:172.21.15.188:3300/0 conn(0x7fc94015a2e0 msgr2=0x7fc9400c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:11.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.901+0000 7fc944ce2640 1 --2- 172.21.15.111:0/2959100151 >> v2:172.21.15.188:3300/0 conn(0x7fc94015a2e0 0x7fc9400c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.901+0000 7fc944ce2640 1 -- 172.21.15.111:0/2959100151 >> v1:172.21.15.111:6789/0 conn(0x7fc9400c9f80 legacy=0x7fc9400cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.901+0000 7fc944ce2640 1 -- 172.21.15.111:0/2959100151 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9400c8b70 con 0x7fc9400cb3f0 2024-09-13T17:59:11.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.902+0000 7fc944ce2640 1 --2- 172.21.15.111:0/2959100151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9400cb3f0 0x7fc94006d030 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc9400cf340 tx=0x7fc93802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d210b4624525f3f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:11.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.902+0000 7fc93effd640 1 -- 172.21.15.111:0/2959100151 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93802aea0 con 0x7fc9400cb3f0 2024-09-13T17:59:11.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.902+0000 7fc93effd640 1 -- 172.21.15.111:0/2959100151 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93802d050 con 0x7fc9400cb3f0 2024-09-13T17:59:11.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.902+0000 7fc93effd640 1 -- 172.21.15.111:0/2959100151 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938007960 con 0x7fc9400cb3f0 2024-09-13T17:59:11.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.903+0000 7fc946745640 1 -- 172.21.15.111:0/2959100151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9400cb3f0 msgr2=0x7fc94006d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.903+0000 7fc946745640 1 --2- 172.21.15.111:0/2959100151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9400cb3f0 0x7fc94006d030 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc9400cf340 tx=0x7fc93802a680 comp rx=0 tx=0).stop 2024-09-13T17:59:11.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.903+0000 7fc946745640 1 -- 172.21.15.111:0/2959100151 shutdown_connections 2024-09-13T17:59:11.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.903+0000 7fc946745640 1 --2- 172.21.15.111:0/2959100151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9400cb3f0 0x7fc94006d030 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.903+0000 7fc946745640 1 --2- 172.21.15.111:0/2959100151 >> v2:172.21.15.188:3300/0 conn(0x7fc94015a2e0 0x7fc9400c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.903+0000 7fc946745640 1 -- 172.21.15.111:0/2959100151 >> 172.21.15.111:0/2959100151 conn(0x7fc940069ed0 msgr2=0x7fc94006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:11.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.903+0000 7fc946745640 1 -- 172.21.15.111:0/2959100151 shutdown_connections 2024-09-13T17:59:11.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.903+0000 7fc946745640 1 -- 172.21.15.111:0/2959100151 wait complete. 2024-09-13T17:59:11.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.904+0000 7fc946745640 1 Processor -- start 2024-09-13T17:59:11.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.904+0000 7fc946745640 1 -- start start 2024-09-13T17:59:11.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc946745640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9400c9f80 0x7fc940155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc946745640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9400cb3f0 0x7fc9400c11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc946745640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc94015a2e0 0x7fc940156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc946745640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9400cc860 con 0x7fc9400cb3f0 2024-09-13T17:59:11.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc946745640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9401558c0 con 0x7fc94015a2e0 2024-09-13T17:59:11.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc946745640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9400cd380 con 0x7fc9400c9f80 2024-09-13T17:59:11.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc93f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9400cb3f0 0x7fc9400c11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc93f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9400cb3f0 0x7fc9400c11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52664/0 (socket says 172.21.15.111:52664) 2024-09-13T17:59:11.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc93f7fe640 1 -- 172.21.15.111:0/1534353919 learned_addr learned my addr 172.21.15.111:0/1534353919 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:11.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc93ffff640 1 --2- 172.21.15.111:0/1534353919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9400c9f80 0x7fc940155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc944ce2640 1 --2- 172.21.15.111:0/1534353919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc94015a2e0 0x7fc940156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc93f7fe640 1 -- 172.21.15.111:0/1534353919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9400c9f80 msgr2=0x7fc940155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc93f7fe640 1 --2- 172.21.15.111:0/1534353919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9400c9f80 0x7fc940155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc93f7fe640 1 -- 172.21.15.111:0/1534353919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc94015a2e0 msgr2=0x7fc940156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:11.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc93f7fe640 1 --2- 172.21.15.111:0/1534353919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc94015a2e0 0x7fc940156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:11.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc93f7fe640 1 -- 172.21.15.111:0/1534353919 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc938009a70 con 0x7fc9400cb3f0 2024-09-13T17:59:11.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc93ffff640 1 --2- 172.21.15.111:0/1534353919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9400c9f80 0x7fc940155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:11.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc944ce2640 1 --2- 172.21.15.111:0/1534353919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc94015a2e0 0x7fc940156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:11.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc93f7fe640 1 --2- 172.21.15.111:0/1534353919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9400cb3f0 0x7fc9400c11d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc93400c410 tx=0x7fc93400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:11.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc93cff9640 1 -- 172.21.15.111:0/1534353919 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934005ce0 con 0x7fc9400cb3f0 2024-09-13T17:59:11.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc93cff9640 1 -- 172.21.15.111:0/1534353919 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc934005e40 con 0x7fc9400cb3f0 2024-09-13T17:59:11.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc946745640 1 -- 172.21.15.111:0/1534353919 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc940155af0 con 0x7fc9400cb3f0 2024-09-13T17:59:11.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc93cff9640 1 -- 172.21.15.111:0/1534353919 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9340072f0 con 0x7fc9400cb3f0 2024-09-13T17:59:11.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.905+0000 7fc946745640 1 -- 172.21.15.111:0/1534353919 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9400ccd30 con 0x7fc9400cb3f0 2024-09-13T17:59:11.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.907+0000 7fc946745640 1 -- 172.21.15.111:0/1534353919 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9400cd4c0 con 0x7fc9400cb3f0 2024-09-13T17:59:11.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.907+0000 7fc93cff9640 1 -- 172.21.15.111:0/1534353919 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc934007510 con 0x7fc9400cb3f0 2024-09-13T17:59:11.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.907+0000 7fc93cff9640 1 --2- 172.21.15.111:0/1534353919 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc92c06ac00 0x7fc92c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:11.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.907+0000 7fc93cff9640 1 -- 172.21.15.111:0/1534353919 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fc934088af0 con 0x7fc9400cb3f0 2024-09-13T17:59:11.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.907+0000 7fc93ffff640 1 --2- 172.21.15.111:0/1534353919 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc92c06ac00 0x7fc92c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:11.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.908+0000 7fc93ffff640 1 --2- 172.21.15.111:0/1534353919 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc92c06ac00 0x7fc92c06d0c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc930009b90 tx=0x7fc930008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:11.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:11.911+0000 7fc93cff9640 1 -- 172.21.15.111:0/1534353919 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc934053c70 con 0x7fc9400cb3f0 2024-09-13T17:59:12.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.108+0000 7fc946745640 1 -- 172.21.15.111:0/1534353919 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc94006a450 con 0x7fc9400cb3f0 2024-09-13T17:59:12.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.109+0000 7fc93cff9640 1 -- 172.21.15.111:0/1534353919 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4742 (secure 0 0 0) 0x7fc934017090 con 0x7fc9400cb3f0 2024-09-13T17:59:12.111 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:12.111 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4217138481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4217138481},{"type":"v1","addr":"172.21.15.188:6834","nonce":4217138481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3099948287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3099948287},{"type":"v1","addr":"172.21.15.188:6835","nonce":3099948287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/904506495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":904506495},{"type":"v1","addr":"172.21.15.111:6837","nonce":904506495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:58:43.534567+0000","modified":"2024-09-13T17:59:05.554735+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5487":{"gid":5487,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6835/214034224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":214034224},{"type":"v1","addr":"172.21.15.111:6835","nonce":214034224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:59:12.111 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 21 2024-09-13T17:59:12.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.111+0000 7fc946745640 1 -- 172.21.15.111:0/1534353919 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc92c06ac00 msgr2=0x7fc92c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:12.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.111+0000 7fc946745640 1 --2- 172.21.15.111:0/1534353919 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc92c06ac00 0x7fc92c06d0c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc930009b90 tx=0x7fc930008040 comp rx=0 tx=0).stop 2024-09-13T17:59:12.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.111+0000 7fc946745640 1 -- 172.21.15.111:0/1534353919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9400cb3f0 msgr2=0x7fc9400c11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:12.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.111+0000 7fc946745640 1 --2- 172.21.15.111:0/1534353919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9400cb3f0 0x7fc9400c11d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc93400c410 tx=0x7fc93400c8d0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.111+0000 7fc946745640 1 -- 172.21.15.111:0/1534353919 shutdown_connections 2024-09-13T17:59:12.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.111+0000 7fc946745640 1 --2- 172.21.15.111:0/1534353919 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc92c06ac00 0x7fc92c06d0c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.111+0000 7fc946745640 1 --2- 172.21.15.111:0/1534353919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc94015a2e0 0x7fc940156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.111+0000 7fc946745640 1 --2- 172.21.15.111:0/1534353919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9400cb3f0 0x7fc9400c11d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.111+0000 7fc946745640 1 --2- 172.21.15.111:0/1534353919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9400c9f80 0x7fc940155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.111+0000 7fc946745640 1 -- 172.21.15.111:0/1534353919 >> 172.21.15.111:0/1534353919 conn(0x7fc940069ed0 msgr2=0x7fc940071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:12.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.111+0000 7fc946745640 1 -- 172.21.15.111:0/1534353919 shutdown_connections 2024-09-13T17:59:12.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.111+0000 7fc946745640 1 -- 172.21.15.111:0/1534353919 wait complete. 2024-09-13T17:59:12.124 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:59:12.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.219+0000 7f0161087640 1 Processor -- start 2024-09-13T17:59:12.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.219+0000 7f0161087640 1 -- start start 2024-09-13T17:59:12.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.220+0000 7f0161087640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f015c072110 0x7f015c070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.220+0000 7f0161087640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f015c0c8920 0x7f015c0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.220+0000 7f0161087640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015c0abdf0 con 0x7f015c071d30 2024-09-13T17:59:12.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.220+0000 7f0161087640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015c0c9310 con 0x7f015c072110 2024-09-13T17:59:12.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.220+0000 7f0161087640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015c0b2280 con 0x7f015c0c8920 2024-09-13T17:59:12.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.220+0000 7f015a59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f015c072110 0x7f015c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.220+0000 7f015a59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f015c072110 0x7f015c070190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55236/0 (socket says 172.21.15.111:55236) 2024-09-13T17:59:12.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.220+0000 7f015ad9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f015c071d30 0x7f015c0b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43032/0 (socket says 172.21.15.111:43032) 2024-09-13T17:59:12.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.220+0000 7f015a59c640 1 -- 172.21.15.111:0/2208699714 learned_addr learned my addr 172.21.15.111:0/2208699714 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:12.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.220+0000 7f015b59e640 1 --2- 172.21.15.111:0/2208699714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f015c0c8920 0x7f015c0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.220+0000 7f015a59c640 1 -- 172.21.15.111:0/2208699714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f015c0c8920 msgr2=0x7f015c0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:12.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.220+0000 7f015a59c640 1 --2- 172.21.15.111:0/2208699714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f015c0c8920 0x7f015c0c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.220+0000 7f015a59c640 1 -- 172.21.15.111:0/2208699714 >> v1:172.21.15.111:6789/0 conn(0x7f015c071d30 legacy=0x7f015c0b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:12.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.220+0000 7f015a59c640 1 -- 172.21.15.111:0/2208699714 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c0ab1e0 con 0x7f015c072110 2024-09-13T17:59:12.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.220+0000 7f0159d9b640 1 -- 172.21.15.111:0/2208699714 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386293766 0 0) 0x7f015c0abdf0 con 0x7f015c071d30 2024-09-13T17:59:12.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.220+0000 7f015b59e640 1 --2- 172.21.15.111:0/2208699714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f015c0c8920 0x7f015c0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:59:12.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.221+0000 7f015a59c640 1 --2- 172.21.15.111:0/2208699714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f015c072110 0x7f015c070190 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0150009870 tx=0x7f015002c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5fd7b7951b051d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:12.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.221+0000 7f0159d9b640 1 -- 172.21.15.111:0/2208699714 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0150031710 con 0x7f015c072110 2024-09-13T17:59:12.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.221+0000 7f0159d9b640 1 -- 172.21.15.111:0/2208699714 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015002f050 con 0x7f015c072110 2024-09-13T17:59:12.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.221+0000 7f0159d9b640 1 -- 172.21.15.111:0/2208699714 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0150031a90 con 0x7f015c072110 2024-09-13T17:59:12.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.222+0000 7f0161087640 1 -- 172.21.15.111:0/2208699714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f015c072110 msgr2=0x7f015c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:12.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.222+0000 7f0161087640 1 --2- 172.21.15.111:0/2208699714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f015c072110 0x7f015c070190 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0150009870 tx=0x7f015002c480 comp rx=0 tx=0).stop 2024-09-13T17:59:12.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.222+0000 7f0161087640 1 -- 172.21.15.111:0/2208699714 shutdown_connections 2024-09-13T17:59:12.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.222+0000 7f0161087640 1 --2- 172.21.15.111:0/2208699714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f015c0c8920 0x7f015c0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.222+0000 7f0161087640 1 --2- 172.21.15.111:0/2208699714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f015c072110 0x7f015c070190 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.222+0000 7f0161087640 1 -- 172.21.15.111:0/2208699714 >> 172.21.15.111:0/2208699714 conn(0x7f015c069ed0 msgr2=0x7f015c06a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:12.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.222+0000 7f0161087640 1 -- 172.21.15.111:0/2208699714 shutdown_connections 2024-09-13T17:59:12.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.222+0000 7f0161087640 1 -- 172.21.15.111:0/2208699714 wait complete. 2024-09-13T17:59:12.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.223+0000 7f0161087640 1 Processor -- start 2024-09-13T17:59:12.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.223+0000 7f0161087640 1 -- start start 2024-09-13T17:59:12.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.224+0000 7f0161087640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f015c071d30 0x7f015c15c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.224+0000 7f0161087640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f015c072110 0x7f015c0992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.224+0000 7f0161087640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f015c0c8920 0x7f015c0c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.224+0000 7f0161087640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c0c63f0 con 0x7f015c072110 2024-09-13T17:59:12.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.224+0000 7f0161087640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c15c910 con 0x7f015c071d30 2024-09-13T17:59:12.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.224+0000 7f0161087640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015c0c6640 con 0x7f015c0c8920 2024-09-13T17:59:12.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.224+0000 7f015a59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f015c072110 0x7f015c0992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.224+0000 7f015a59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f015c072110 0x7f015c0992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55250/0 (socket says 172.21.15.111:55250) 2024-09-13T17:59:12.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.224+0000 7f015a59c640 1 -- 172.21.15.111:0/1015748008 learned_addr learned my addr 172.21.15.111:0/1015748008 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:12.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.224+0000 7f015b59e640 1 --2- 172.21.15.111:0/1015748008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f015c0c8920 0x7f015c0c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.224+0000 7f015ad9d640 1 --2- 172.21.15.111:0/1015748008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f015c071d30 0x7f015c15c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.224+0000 7f015a59c640 1 -- 172.21.15.111:0/1015748008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f015c0c8920 msgr2=0x7f015c0c2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:12.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.224+0000 7f015a59c640 1 --2- 172.21.15.111:0/1015748008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f015c0c8920 0x7f015c0c2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.224+0000 7f015a59c640 1 -- 172.21.15.111:0/1015748008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f015c071d30 msgr2=0x7f015c15c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:12.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.224+0000 7f015a59c640 1 --2- 172.21.15.111:0/1015748008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f015c071d30 0x7f015c15c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.224+0000 7f015a59c640 1 -- 172.21.15.111:0/1015748008 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01500094d0 con 0x7f015c072110 2024-09-13T17:59:12.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.224+0000 7f015ad9d640 1 --2- 172.21.15.111:0/1015748008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f015c071d30 0x7f015c15c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:12.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.225+0000 7f015a59c640 1 --2- 172.21.15.111:0/1015748008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f015c072110 0x7f015c0992b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0150000c00 tx=0x7f01500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:12.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.225+0000 7f01437fe640 1 -- 172.21.15.111:0/1015748008 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0150033990 con 0x7f015c072110 2024-09-13T17:59:12.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.225+0000 7f01437fe640 1 -- 172.21.15.111:0/1015748008 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0150033b10 con 0x7f015c072110 2024-09-13T17:59:12.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.225+0000 7f0161087640 1 -- 172.21.15.111:0/1015748008 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015c0c6920 con 0x7f015c072110 2024-09-13T17:59:12.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.225+0000 7f01437fe640 1 -- 172.21.15.111:0/1015748008 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015003c400 con 0x7f015c072110 2024-09-13T17:59:12.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.225+0000 7f0161087640 1 -- 172.21.15.111:0/1015748008 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c0c6ee0 con 0x7f015c072110 2024-09-13T17:59:12.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.226+0000 7f0161087640 1 -- 172.21.15.111:0/1015748008 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015c0c6780 con 0x7f015c072110 2024-09-13T17:59:12.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.226+0000 7f01437fe640 1 -- 172.21.15.111:0/1015748008 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f015003a040 con 0x7f015c072110 2024-09-13T17:59:12.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.227+0000 7f01437fe640 1 --2- 172.21.15.111:0/1015748008 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f014806ac00 0x7f014806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.227+0000 7f01437fe640 1 -- 172.21.15.111:0/1015748008 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f01500abd60 con 0x7f015c072110 2024-09-13T17:59:12.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.227+0000 7f015ad9d640 1 --2- 172.21.15.111:0/1015748008 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f014806ac00 0x7f014806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.227+0000 7f015ad9d640 1 --2- 172.21.15.111:0/1015748008 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f014806ac00 0x7f014806d0c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f014c000f60 tx=0x7f014c007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:12.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.231+0000 7f01437fe640 1 -- 172.21.15.111:0/1015748008 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f015c0c6780 con 0x7f015c072110 2024-09-13T17:59:12.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.425+0000 7f0161087640 1 -- 172.21.15.111:0/1015748008 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f015c0c6780 con 0x7f015c072110 2024-09-13T17:59:12.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.425+0000 7f01437fe640 1 -- 172.21.15.111:0/1015748008 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4742 (secure 0 0 0) 0x7f015c0c6780 con 0x7f015c072110 2024-09-13T17:59:12.428 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:12.428 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4217138481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4217138481},{"type":"v1","addr":"172.21.15.188:6834","nonce":4217138481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3099948287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3099948287},{"type":"v1","addr":"172.21.15.188:6835","nonce":3099948287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/904506495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":904506495},{"type":"v1","addr":"172.21.15.111:6837","nonce":904506495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:58:43.534567+0000","modified":"2024-09-13T17:59:05.554735+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5487":{"gid":5487,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6835/214034224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":214034224},{"type":"v1","addr":"172.21.15.111:6835","nonce":214034224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:59:12.428 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 21 2024-09-13T17:59:12.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.427+0000 7f0161087640 1 -- 172.21.15.111:0/1015748008 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f014806ac00 msgr2=0x7f014806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:12.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.427+0000 7f0161087640 1 --2- 172.21.15.111:0/1015748008 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f014806ac00 0x7f014806d0c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f014c000f60 tx=0x7f014c007c90 comp rx=0 tx=0).stop 2024-09-13T17:59:12.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.427+0000 7f0161087640 1 -- 172.21.15.111:0/1015748008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f015c072110 msgr2=0x7f015c0992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:12.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.427+0000 7f0161087640 1 --2- 172.21.15.111:0/1015748008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f015c072110 0x7f015c0992b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0150000c00 tx=0x7f01500046c0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.427+0000 7f0161087640 1 -- 172.21.15.111:0/1015748008 shutdown_connections 2024-09-13T17:59:12.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.427+0000 7f0161087640 1 --2- 172.21.15.111:0/1015748008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f015c0c8920 0x7f015c0c2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.427+0000 7f0161087640 1 --2- 172.21.15.111:0/1015748008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f015c072110 0x7f015c0992b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.427+0000 7f0161087640 1 --2- 172.21.15.111:0/1015748008 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f014806ac00 0x7f014806d0c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.427+0000 7f0161087640 1 --2- 172.21.15.111:0/1015748008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f015c071d30 0x7f015c15c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.427+0000 7f0161087640 1 -- 172.21.15.111:0/1015748008 >> 172.21.15.111:0/1015748008 conn(0x7f015c069ed0 msgr2=0x7f015c0b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:12.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.428+0000 7f0161087640 1 -- 172.21.15.111:0/1015748008 shutdown_connections 2024-09-13T17:59:12.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.428+0000 7f0161087640 1 -- 172.21.15.111:0/1015748008 wait complete. 2024-09-13T17:59:12.441 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:59:12.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.535+0000 7f364e24f640 1 Processor -- start 2024-09-13T17:59:12.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.536+0000 7f364e24f640 1 -- start start 2024-09-13T17:59:12.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.536+0000 7f364e24f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3648072110 0x7f3648070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.536+0000 7f364e24f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36480c8920 0x7f36480c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.536+0000 7f364e24f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36480abdf0 con 0x7f3648071d30 2024-09-13T17:59:12.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.536+0000 7f364e24f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36480c9310 con 0x7f3648072110 2024-09-13T17:59:12.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.536+0000 7f364e24f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36480b2280 con 0x7f36480c8920 2024-09-13T17:59:12.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.537+0000 7f3646ffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3648072110 0x7f3648070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.537+0000 7f3647fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36480c8920 0x7f36480c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.537+0000 7f3647fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36480c8920 0x7f36480c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.111:43028/0 (socket says 172.21.15.111:43028) 2024-09-13T17:59:12.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.537+0000 7f3647fff640 1 -- 172.21.15.111:0/4178495177 learned_addr learned my addr 172.21.15.111:0/4178495177 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:12.540 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.537+0000 7f36467fc640 1 -- 172.21.15.111:0/4178495177 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352754504 0 0) 0x7f36480abdf0 con 0x7f3648071d30 2024-09-13T17:59:12.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.537+0000 7f3647fff640 1 -- 172.21.15.111:0/4178495177 >> v2:172.21.15.188:3300/0 conn(0x7f3648072110 msgr2=0x7f3648070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:12.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.537+0000 7f3647fff640 1 --2- 172.21.15.111:0/4178495177 >> v2:172.21.15.188:3300/0 conn(0x7f3648072110 0x7f3648070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.537+0000 7f3647fff640 1 -- 172.21.15.111:0/4178495177 >> v1:172.21.15.188:6789/0 conn(0x7f3648071d30 legacy=0x7f36480b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:12.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.537+0000 7f3647fff640 1 -- 172.21.15.111:0/4178495177 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36480ab1e0 con 0x7f36480c8920 2024-09-13T17:59:12.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.538+0000 7f3647fff640 1 --2- 172.21.15.111:0/4178495177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36480c8920 0x7f36480c8dd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3640009e50 tx=0x7f364002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5ca30246318143f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:12.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.538+0000 7f36467fc640 1 -- 172.21.15.111:0/4178495177 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640031600 con 0x7f36480c8920 2024-09-13T17:59:12.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.538+0000 7f36467fc640 1 -- 172.21.15.111:0/4178495177 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364002d050 con 0x7f36480c8920 2024-09-13T17:59:12.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.538+0000 7f36467fc640 1 -- 172.21.15.111:0/4178495177 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640031980 con 0x7f36480c8920 2024-09-13T17:59:12.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.539+0000 7f364e24f640 1 -- 172.21.15.111:0/4178495177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36480c8920 msgr2=0x7f36480c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:12.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.539+0000 7f364e24f640 1 --2- 172.21.15.111:0/4178495177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36480c8920 0x7f36480c8dd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3640009e50 tx=0x7f364002a710 comp rx=0 tx=0).stop 2024-09-13T17:59:12.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.539+0000 7f364e24f640 1 -- 172.21.15.111:0/4178495177 shutdown_connections 2024-09-13T17:59:12.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.539+0000 7f364e24f640 1 --2- 172.21.15.111:0/4178495177 >> v2:172.21.15.188:3300/0 conn(0x7f3648072110 0x7f3648070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.539+0000 7f364e24f640 1 --2- 172.21.15.111:0/4178495177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36480c8920 0x7f36480c8dd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.539+0000 7f364e24f640 1 -- 172.21.15.111:0/4178495177 >> 172.21.15.111:0/4178495177 conn(0x7f3648069ed0 msgr2=0x7f364806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:12.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.539+0000 7f364e24f640 1 -- 172.21.15.111:0/4178495177 shutdown_connections 2024-09-13T17:59:12.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.539+0000 7f364e24f640 1 -- 172.21.15.111:0/4178495177 wait complete. 2024-09-13T17:59:12.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.540+0000 7f364e24f640 1 Processor -- start 2024-09-13T17:59:12.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.540+0000 7f364e24f640 1 -- start start 2024-09-13T17:59:12.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f364e24f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3648071d30 0x7f364815a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f364e24f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3648072110 0x7f36480c0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f364e24f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36480c8920 0x7f364815ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f364e24f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36480c4a50 con 0x7f36480c8920 2024-09-13T17:59:12.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f364e24f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36480c4d30 con 0x7f3648072110 2024-09-13T17:59:12.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f364e24f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36480c5010 con 0x7f3648071d30 2024-09-13T17:59:12.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f3647fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36480c8920 0x7f364815ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f3647fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36480c8920 0x7f364815ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55258/0 (socket says 172.21.15.111:55258) 2024-09-13T17:59:12.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f3647fff640 1 -- 172.21.15.111:0/372368439 learned_addr learned my addr 172.21.15.111:0/372368439 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:12.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f3646ffd640 1 --2- 172.21.15.111:0/372368439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3648072110 0x7f36480c0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f36477fe640 1 --2- 172.21.15.111:0/372368439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3648071d30 0x7f364815a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f3647fff640 1 -- 172.21.15.111:0/372368439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3648071d30 msgr2=0x7f364815a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:12.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f3647fff640 1 --2- 172.21.15.111:0/372368439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3648071d30 0x7f364815a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f3647fff640 1 -- 172.21.15.111:0/372368439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3648072110 msgr2=0x7f36480c0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:12.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f3647fff640 1 --2- 172.21.15.111:0/372368439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3648072110 0x7f36480c0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f3647fff640 1 -- 172.21.15.111:0/372368439 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3640009b30 con 0x7f36480c8920 2024-09-13T17:59:12.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f3647fff640 1 --2- 172.21.15.111:0/372368439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36480c8920 0x7f364815ae20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3640009e50 tx=0x7f3640004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:12.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f362ffff640 1 -- 172.21.15.111:0/372368439 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640003b90 con 0x7f36480c8920 2024-09-13T17:59:12.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f362ffff640 1 -- 172.21.15.111:0/372368439 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3640003cf0 con 0x7f36480c8920 2024-09-13T17:59:12.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f362ffff640 1 -- 172.21.15.111:0/372368439 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640003ea0 con 0x7f36480c8920 2024-09-13T17:59:12.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.541+0000 7f364e24f640 1 -- 172.21.15.111:0/372368439 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36480ccb50 con 0x7f36480c8920 2024-09-13T17:59:12.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.542+0000 7f364e24f640 1 -- 172.21.15.111:0/372368439 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36480c1c40 con 0x7f36480c8920 2024-09-13T17:59:12.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.543+0000 7f364e24f640 1 -- 172.21.15.111:0/372368439 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36480adc50 con 0x7f36480c8920 2024-09-13T17:59:12.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.543+0000 7f362ffff640 1 -- 172.21.15.111:0/372368439 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3640004230 con 0x7f36480c8920 2024-09-13T17:59:12.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.543+0000 7f362ffff640 1 --2- 172.21.15.111:0/372368439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f363406ac00 0x7f363406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.543+0000 7f362ffff640 1 -- 172.21.15.111:0/372368439 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f36400a9fe0 con 0x7f36480c8920 2024-09-13T17:59:12.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.543+0000 7f36477fe640 1 --2- 172.21.15.111:0/372368439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f363406ac00 0x7f363406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.544+0000 7f36477fe640 1 --2- 172.21.15.111:0/372368439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f363406ac00 0x7f363406d0c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3638000f60 tx=0x7f3638009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:12.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.547+0000 7f362ffff640 1 -- 172.21.15.111:0/372368439 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f36400750c0 con 0x7f36480c8920 2024-09-13T17:59:12.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.712+0000 7f364e24f640 1 -- 172.21.15.111:0/372368439 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3648070190 con 0x7f36480c8920 2024-09-13T17:59:12.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.713+0000 7f362ffff640 1 -- 172.21.15.111:0/372368439 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+17012 (secure 0 0 0) 0x7f3640049090 con 0x7f36480c8920 2024-09-13T17:59:12.715 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:12.715 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":42,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:59:08.617279+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:58:41.698998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T17:58:42.499870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1c","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6837/982359636":"2024-09-14T17:59:01.542287+0000","172.21.15.111:6836/982359636":"2024-09-14T17:59:01.542287+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:59:12.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.714+0000 7f364e24f640 1 -- 172.21.15.111:0/372368439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f363406ac00 msgr2=0x7f363406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:12.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.714+0000 7f364e24f640 1 --2- 172.21.15.111:0/372368439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f363406ac00 0x7f363406d0c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3638000f60 tx=0x7f3638009040 comp rx=0 tx=0).stop 2024-09-13T17:59:12.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.714+0000 7f364e24f640 1 -- 172.21.15.111:0/372368439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36480c8920 msgr2=0x7f364815ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:12.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.714+0000 7f364e24f640 1 --2- 172.21.15.111:0/372368439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36480c8920 0x7f364815ae20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3640009e50 tx=0x7f3640004a40 comp rx=0 tx=0).stop 2024-09-13T17:59:12.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.714+0000 7f364e24f640 1 -- 172.21.15.111:0/372368439 shutdown_connections 2024-09-13T17:59:12.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.714+0000 7f364e24f640 1 --2- 172.21.15.111:0/372368439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36480c8920 0x7f364815ae20 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.714+0000 7f364e24f640 1 --2- 172.21.15.111:0/372368439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f363406ac00 0x7f363406d0c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.714+0000 7f364e24f640 1 --2- 172.21.15.111:0/372368439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3648072110 0x7f36480c0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.715+0000 7f364e24f640 1 --2- 172.21.15.111:0/372368439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3648071d30 0x7f364815a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.715+0000 7f364e24f640 1 -- 172.21.15.111:0/372368439 >> 172.21.15.111:0/372368439 conn(0x7f3648069ed0 msgr2=0x7f3648070730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:12.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.715+0000 7f364e24f640 1 -- 172.21.15.111:0/372368439 shutdown_connections 2024-09-13T17:59:12.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.715+0000 7f364e24f640 1 -- 172.21.15.111:0/372368439 wait complete. 2024-09-13T17:59:12.727 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T17:59:12.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.824+0000 7f804ed6c640 1 Processor -- start 2024-09-13T17:59:12.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.824+0000 7f804ed6c640 1 -- start start 2024-09-13T17:59:12.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.825+0000 7f804ed6c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f804815a2e0 0x7f80480c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.825+0000 7f804ed6c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f80480c9f80 0x7f80480cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.825+0000 7f804ed6c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80480cb670 con 0x7f80480cb220 2024-09-13T17:59:12.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.825+0000 7f804ed6c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80480c8920 con 0x7f80480c9f80 2024-09-13T17:59:12.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.825+0000 7f804ed6c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804815a6c0 con 0x7f804815a2e0 2024-09-13T17:59:12.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.825+0000 7f8047fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f80480c9f80 0x7f80480cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.825+0000 7f8047fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f80480c9f80 0x7f80480cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55268/0 (socket says 172.21.15.111:55268) 2024-09-13T17:59:12.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.825+0000 7f8047fff640 1 -- 172.21.15.111:0/736321608 learned_addr learned my addr 172.21.15.111:0/736321608 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:12.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.825+0000 7f804cb08640 1 --2- 172.21.15.111:0/736321608 >> v2:172.21.15.188:3300/0 conn(0x7f804815a2e0 0x7f80480c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.825+0000 7f8047fff640 1 -- 172.21.15.111:0/736321608 >> v2:172.21.15.188:3300/0 conn(0x7f804815a2e0 msgr2=0x7f80480c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:12.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.825+0000 7f8047fff640 1 --2- 172.21.15.111:0/736321608 >> v2:172.21.15.188:3300/0 conn(0x7f804815a2e0 0x7f80480c9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.825+0000 7f8047fff640 1 -- 172.21.15.111:0/736321608 >> v1:172.21.15.188:6789/0 conn(0x7f80480cb220 legacy=0x7f80480ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:12.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.825+0000 7f8047fff640 1 -- 172.21.15.111:0/736321608 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80480ca500 con 0x7f80480c9f80 2024-09-13T17:59:12.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.826+0000 7f8047fff640 1 --2- 172.21.15.111:0/736321608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80480c9f80 0x7f80480cace0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f80480cf340 tx=0x7f803402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0b6d7c8a56741 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:12.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.826+0000 7f80477fe640 1 -- 172.21.15.111:0/736321608 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034032440 con 0x7f80480c9f80 2024-09-13T17:59:12.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.826+0000 7f80477fe640 1 -- 172.21.15.111:0/736321608 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80340325a0 con 0x7f80480c9f80 2024-09-13T17:59:12.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.826+0000 7f80477fe640 1 -- 172.21.15.111:0/736321608 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034032930 con 0x7f80480c9f80 2024-09-13T17:59:12.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.826+0000 7f804ed6c640 1 -- 172.21.15.111:0/736321608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80480c9f80 msgr2=0x7f80480cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:12.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.826+0000 7f804ed6c640 1 --2- 172.21.15.111:0/736321608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80480c9f80 0x7f80480cace0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f80480cf340 tx=0x7f803402a230 comp rx=0 tx=0).stop 2024-09-13T17:59:12.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.826+0000 7f804ed6c640 1 -- 172.21.15.111:0/736321608 shutdown_connections 2024-09-13T17:59:12.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.826+0000 7f804ed6c640 1 --2- 172.21.15.111:0/736321608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80480c9f80 0x7f80480cace0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.826+0000 7f804ed6c640 1 --2- 172.21.15.111:0/736321608 >> v2:172.21.15.188:3300/0 conn(0x7f804815a2e0 0x7f80480c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.826+0000 7f804ed6c640 1 -- 172.21.15.111:0/736321608 >> 172.21.15.111:0/736321608 conn(0x7f8048069ed0 msgr2=0x7f804806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:12.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.826+0000 7f804ed6c640 1 -- 172.21.15.111:0/736321608 shutdown_connections 2024-09-13T17:59:12.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.827+0000 7f804ed6c640 1 -- 172.21.15.111:0/736321608 wait complete. 2024-09-13T17:59:12.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.828+0000 7f804ed6c640 1 Processor -- start 2024-09-13T17:59:12.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.828+0000 7f804ed6c640 1 -- start start 2024-09-13T17:59:12.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.828+0000 7f804ed6c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80480c9f80 0x7f804815b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f804ed6c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80480cb220 0x7f80480977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f804ed6c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f804815a2e0 0x7f804806cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f804ed6c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804815b110 con 0x7f80480c9f80 2024-09-13T17:59:12.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f804ed6c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804815a800 con 0x7f80480cb220 2024-09-13T17:59:12.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f804ed6c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80480abfe0 con 0x7f804815a2e0 2024-09-13T17:59:12.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f804cb08640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80480c9f80 0x7f804815b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f804cb08640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80480c9f80 0x7f804815b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55278/0 (socket says 172.21.15.111:55278) 2024-09-13T17:59:12.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f804cb08640 1 -- 172.21.15.111:0/3404167439 learned_addr learned my addr 172.21.15.111:0/3404167439 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:12.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f804d309640 1 --2- 172.21.15.111:0/3404167439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f804815a2e0 0x7f804806cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f8047fff640 1 --2- 172.21.15.111:0/3404167439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80480cb220 0x7f80480977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f804cb08640 1 -- 172.21.15.111:0/3404167439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f804815a2e0 msgr2=0x7f804806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:12.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f804cb08640 1 --2- 172.21.15.111:0/3404167439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f804815a2e0 0x7f804806cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f804cb08640 1 -- 172.21.15.111:0/3404167439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80480cb220 msgr2=0x7f80480977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:12.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f804cb08640 1 --2- 172.21.15.111:0/3404167439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80480cb220 0x7f80480977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:12.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f804cb08640 1 -- 172.21.15.111:0/3404167439 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80340094d0 con 0x7f80480c9f80 2024-09-13T17:59:12.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f804cb08640 1 --2- 172.21.15.111:0/3404167439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80480c9f80 0x7f804815b530 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f803c00c7f0 tx=0x7f803c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:12.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f80457fa640 1 -- 172.21.15.111:0/3404167439 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c004220 con 0x7f80480c9f80 2024-09-13T17:59:12.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f80457fa640 1 -- 172.21.15.111:0/3404167439 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c004380 con 0x7f80480c9f80 2024-09-13T17:59:12.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f80457fa640 1 -- 172.21.15.111:0/3404167439 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c007840 con 0x7f80480c9f80 2024-09-13T17:59:12.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f804ed6c640 1 -- 172.21.15.111:0/3404167439 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804815aa50 con 0x7f80480c9f80 2024-09-13T17:59:12.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.829+0000 7f804ed6c640 1 -- 172.21.15.111:0/3404167439 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8048097d30 con 0x7f80480c9f80 2024-09-13T17:59:12.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.830+0000 7f804ed6c640 1 -- 172.21.15.111:0/3404167439 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80480c9630 con 0x7f80480c9f80 2024-09-13T17:59:12.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.831+0000 7f80457fa640 1 -- 172.21.15.111:0/3404167439 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f803c007bb0 con 0x7f80480c9f80 2024-09-13T17:59:12.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.831+0000 7f80457fa640 1 --2- 172.21.15.111:0/3404167439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f802c06ac00 0x7f802c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:12.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.831+0000 7f80457fa640 1 -- 172.21.15.111:0/3404167439 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f803c089360 con 0x7f80480c9f80 2024-09-13T17:59:12.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.831+0000 7f8047fff640 1 --2- 172.21.15.111:0/3404167439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f802c06ac00 0x7f802c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:12.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.832+0000 7f8047fff640 1 --2- 172.21.15.111:0/3404167439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f802c06ac00 0x7f802c06d0c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8034034000 tx=0x7f803402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:12.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:12.836+0000 7f80457fa640 1 -- 172.21.15.111:0/3404167439 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f803c054420 con 0x7f80480c9f80 2024-09-13T17:59:13.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.033+0000 7f804ed6c640 1 -- 172.21.15.111:0/3404167439 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f80480aeff0 con 0x7f80480c9f80 2024-09-13T17:59:13.035 INFO:tasks.ceph.mon.a.smithi111.stderr:2024-09-13T17:59:13.034+0000 7f3bedaae640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T17:59:13.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.040+0000 7f80457fa640 1 -- 172.21.15.111:0/3404167439 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) v1 ==== 161+0+0 (secure 0 0 0) 0x7f803c007e60 con 0x7f80480c9f80 2024-09-13T17:59:13.042 INFO:teuthology.orchestra.run.smithi111.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T17:59:13.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.043+0000 7f804ed6c640 1 -- 172.21.15.111:0/3404167439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f802c06ac00 msgr2=0x7f802c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:13.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.043+0000 7f804ed6c640 1 --2- 172.21.15.111:0/3404167439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f802c06ac00 0x7f802c06d0c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8034034000 tx=0x7f803402e000 comp rx=0 tx=0).stop 2024-09-13T17:59:13.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.043+0000 7f804ed6c640 1 -- 172.21.15.111:0/3404167439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80480c9f80 msgr2=0x7f804815b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:13.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.043+0000 7f804ed6c640 1 --2- 172.21.15.111:0/3404167439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80480c9f80 0x7f804815b530 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f803c00c7f0 tx=0x7f803c00ccb0 comp rx=0 tx=0).stop 2024-09-13T17:59:13.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.043+0000 7f804ed6c640 1 -- 172.21.15.111:0/3404167439 shutdown_connections 2024-09-13T17:59:13.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.043+0000 7f804ed6c640 1 --2- 172.21.15.111:0/3404167439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f804815a2e0 0x7f804806cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:13.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.043+0000 7f804ed6c640 1 --2- 172.21.15.111:0/3404167439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f802c06ac00 0x7f802c06d0c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:13.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.043+0000 7f804ed6c640 1 --2- 172.21.15.111:0/3404167439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80480cb220 0x7f80480977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:13.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.043+0000 7f804ed6c640 1 --2- 172.21.15.111:0/3404167439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80480c9f80 0x7f804815b530 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:13.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.043+0000 7f804ed6c640 1 -- 172.21.15.111:0/3404167439 >> 172.21.15.111:0/3404167439 conn(0x7f8048069ed0 msgr2=0x7f8048071900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:13.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.043+0000 7f804ed6c640 1 -- 172.21.15.111:0/3404167439 shutdown_connections 2024-09-13T17:59:13.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.043+0000 7f804ed6c640 1 -- 172.21.15.111:0/3404167439 wait complete. 2024-09-13T17:59:13.047 INFO:tasks.ceph.mds.c.smithi111.stderr: -330> 2024-09-13T17:59:07.873+0000 7f26fbac7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:59:13.057 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T17:59:13.072 INFO:tasks.ceph.mds.c.smithi111.stdout:starting mds.c at 2024-09-13T17:59:13.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.155+0000 7f9165073640 1 Processor -- start 2024-09-13T17:59:13.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.155+0000 7f9165073640 1 -- start start 2024-09-13T17:59:13.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.155+0000 7f9165073640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9160159310 0x7f91600ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:13.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.155+0000 7f9165073640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91600ccbe0 con 0x7f91600ce4f0 2024-09-13T17:59:13.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.155+0000 7f9165073640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91600b1810 con 0x7f91600b2b40 2024-09-13T17:59:13.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.155+0000 7f9165073640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91600ad940 con 0x7f9160159310 2024-09-13T17:59:13.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.156+0000 7f915f59e640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f91600ce4f0 0x7f91600ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43034/0 (socket says 172.21.15.111:43034) 2024-09-13T17:59:13.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.155+0000 7f915ed9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9160159310 0x7f91600ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:13.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.156+0000 7f915f59e640 1 -- 172.21.15.111:0/3885806408 learned_addr learned my addr 172.21.15.111:0/3885806408 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:13.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.156+0000 7f915ed9d640 1 -- 172.21.15.111:0/3885806408 >> v1:172.21.15.188:6789/0 conn(0x7f91600b2b40 legacy=0x7f91600ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:13.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.156+0000 7f915ed9d640 1 -- 172.21.15.111:0/3885806408 >> v1:172.21.15.111:6789/0 conn(0x7f91600ce4f0 legacy=0x7f91600ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:13.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.156+0000 7f915ed9d640 1 -- 172.21.15.111:0/3885806408 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916006d9e0 con 0x7f9160159310 2024-09-13T17:59:13.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.156+0000 7f915e59c640 1 -- 172.21.15.111:0/3885806408 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853770266 0 0) 0x7f91600ccbe0 con 0x7f91600ce4f0 2024-09-13T17:59:13.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.157+0000 7f915ed9d640 1 --2- 172.21.15.111:0/3885806408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9160159310 0x7f91600ad400 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f91500097a0 tx=0x7f915002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3546ff36bb4ed5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:13.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.157+0000 7f915e59c640 1 -- 172.21.15.111:0/3885806408 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150032440 con 0x7f9160159310 2024-09-13T17:59:13.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.157+0000 7f915e59c640 1 -- 172.21.15.111:0/3885806408 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91500325a0 con 0x7f9160159310 2024-09-13T17:59:13.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.157+0000 7f915e59c640 1 -- 172.21.15.111:0/3885806408 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150032930 con 0x7f9160159310 2024-09-13T17:59:13.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.157+0000 7f9165073640 1 -- 172.21.15.111:0/3885806408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9160159310 msgr2=0x7f91600ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:13.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.157+0000 7f9165073640 1 --2- 172.21.15.111:0/3885806408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9160159310 0x7f91600ad400 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f91500097a0 tx=0x7f915002a1a0 comp rx=0 tx=0).stop 2024-09-13T17:59:13.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.157+0000 7f9165073640 1 -- 172.21.15.111:0/3885806408 shutdown_connections 2024-09-13T17:59:13.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.157+0000 7f9165073640 1 --2- 172.21.15.111:0/3885806408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9160159310 0x7f91600ad400 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:13.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.157+0000 7f9165073640 1 -- 172.21.15.111:0/3885806408 >> 172.21.15.111:0/3885806408 conn(0x7f916006a0b0 msgr2=0x7f916006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:13.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.157+0000 7f9165073640 1 -- 172.21.15.111:0/3885806408 shutdown_connections 2024-09-13T17:59:13.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.157+0000 7f9165073640 1 -- 172.21.15.111:0/3885806408 wait complete. 2024-09-13T17:59:13.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.159+0000 7f9165073640 1 Processor -- start 2024-09-13T17:59:13.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.159+0000 7f9165073640 1 -- start start 2024-09-13T17:59:13.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.159+0000 7f9165073640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91600b2b40 0x7f91601e3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:13.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.159+0000 7f9165073640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91600ce4f0 0x7f9160157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:13.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.159+0000 7f9165073640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9160159310 0x7f91601df6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:13.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f9165073640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91601dfbf0 con 0x7f91600b2b40 2024-09-13T17:59:13.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f9165073640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91601dfe90 con 0x7f91600ce4f0 2024-09-13T17:59:13.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f9165073640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91601e0170 con 0x7f9160159310 2024-09-13T17:59:13.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f915ed9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91600b2b40 0x7f91601e3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:13.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f915ed9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91600b2b40 0x7f91601e3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55302/0 (socket says 172.21.15.111:55302) 2024-09-13T17:59:13.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f915ed9d640 1 -- 172.21.15.111:0/3468459872 learned_addr learned my addr 172.21.15.111:0/3468459872 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:13.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f915659c640 1 --2- 172.21.15.111:0/3468459872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91600ce4f0 0x7f9160157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:13.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f915f59e640 1 --2- 172.21.15.111:0/3468459872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9160159310 0x7f91601df6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:13.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f915ed9d640 1 -- 172.21.15.111:0/3468459872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9160159310 msgr2=0x7f91601df6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:13.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f915ed9d640 1 --2- 172.21.15.111:0/3468459872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9160159310 0x7f91601df6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:13.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f915ed9d640 1 -- 172.21.15.111:0/3468459872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91600ce4f0 msgr2=0x7f9160157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:13.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f915ed9d640 1 --2- 172.21.15.111:0/3468459872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91600ce4f0 0x7f9160157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:13.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f915ed9d640 1 -- 172.21.15.111:0/3468459872 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9150009400 con 0x7f91600b2b40 2024-09-13T17:59:13.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f915f59e640 1 --2- 172.21.15.111:0/3468459872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9160159310 0x7f91601df6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:13.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f915ed9d640 1 --2- 172.21.15.111:0/3468459872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91600b2b40 0x7f91601e3310 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9150000c00 tx=0x7f91500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:13.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f9157fff640 1 -- 172.21.15.111:0/3468459872 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150030b80 con 0x7f91600b2b40 2024-09-13T17:59:13.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f9157fff640 1 -- 172.21.15.111:0/3468459872 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9150030ce0 con 0x7f91600b2b40 2024-09-13T17:59:13.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f9165073640 1 -- 172.21.15.111:0/3468459872 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91601e0450 con 0x7f91600b2b40 2024-09-13T17:59:13.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f9157fff640 1 -- 172.21.15.111:0/3468459872 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915003d680 con 0x7f91600b2b40 2024-09-13T17:59:13.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.160+0000 7f9165073640 1 -- 172.21.15.111:0/3468459872 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91601e0a10 con 0x7f91600b2b40 2024-09-13T17:59:13.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.161+0000 7f9165073640 1 -- 172.21.15.111:0/3468459872 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91600ad400 con 0x7f91600b2b40 2024-09-13T17:59:13.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.162+0000 7f9157fff640 1 -- 172.21.15.111:0/3468459872 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9150039040 con 0x7f91600b2b40 2024-09-13T17:59:13.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.162+0000 7f9157fff640 1 --2- 172.21.15.111:0/3468459872 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f914006ac00 0x7f914006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:13.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.162+0000 7f9157fff640 1 -- 172.21.15.111:0/3468459872 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f91500a9ff0 con 0x7f91600b2b40 2024-09-13T17:59:13.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.162+0000 7f915659c640 1 --2- 172.21.15.111:0/3468459872 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f914006ac00 0x7f914006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:13.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.163+0000 7f915659c640 1 --2- 172.21.15.111:0/3468459872 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f914006ac00 0x7f914006d0c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f916009b6b0 tx=0x7f9148009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:13.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.166+0000 7f9157fff640 1 -- 172.21.15.111:0/3468459872 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f9150075050 con 0x7f91600b2b40 2024-09-13T17:59:13.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:13.363+0000 7f9165073640 1 -- 172.21.15.111:0/3468459872 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f916006e4b0 con 0x7f91600b2b40 2024-09-13T17:59:14.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.042+0000 7f9157fff640 1 -- 172.21.15.111:0/3468459872 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9150006e10 con 0x7f91600b2b40 2024-09-13T17:59:14.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.044+0000 7f9165073640 1 -- 172.21.15.111:0/3468459872 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f914006ac00 msgr2=0x7f914006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:14.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.044+0000 7f9165073640 1 --2- 172.21.15.111:0/3468459872 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f914006ac00 0x7f914006d0c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f916009b6b0 tx=0x7f9148009040 comp rx=0 tx=0).stop 2024-09-13T17:59:14.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.044+0000 7f9165073640 1 -- 172.21.15.111:0/3468459872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91600b2b40 msgr2=0x7f91601e3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:14.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.044+0000 7f9165073640 1 --2- 172.21.15.111:0/3468459872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91600b2b40 0x7f91601e3310 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9150000c00 tx=0x7f91500046c0 comp rx=0 tx=0).stop 2024-09-13T17:59:14.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.045+0000 7f9165073640 1 -- 172.21.15.111:0/3468459872 shutdown_connections 2024-09-13T17:59:14.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.045+0000 7f9165073640 1 --2- 172.21.15.111:0/3468459872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9160159310 0x7f91601df6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:14.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.045+0000 7f9165073640 1 --2- 172.21.15.111:0/3468459872 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f914006ac00 0x7f914006d0c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:14.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.045+0000 7f9165073640 1 --2- 172.21.15.111:0/3468459872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91600ce4f0 0x7f9160157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:14.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.045+0000 7f9165073640 1 --2- 172.21.15.111:0/3468459872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91600b2b40 0x7f91601e3310 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:14.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.045+0000 7f9165073640 1 -- 172.21.15.111:0/3468459872 >> 172.21.15.111:0/3468459872 conn(0x7f916006a0b0 msgr2=0x7f9160157db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:14.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.045+0000 7f9165073640 1 -- 172.21.15.111:0/3468459872 shutdown_connections 2024-09-13T17:59:14.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.045+0000 7f9165073640 1 -- 172.21.15.111:0/3468459872 wait complete. 2024-09-13T17:59:14.059 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T17:59:14.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.201+0000 7f995f202640 1 Processor -- start 2024-09-13T17:59:14.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.201+0000 7f995f202640 1 -- start start 2024-09-13T17:59:14.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.202+0000 7f995f202640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9958158b00 0x7f995806c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:14.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.202+0000 7f995f202640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f995806c6d0 0x7f9958070b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:14.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.202+0000 7f995f202640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99580b2d10 con 0x7f9958071050 2024-09-13T17:59:14.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.202+0000 7f995f202640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f995806cb60 con 0x7f995806c6d0 2024-09-13T17:59:14.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.202+0000 7f995f202640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99580b2ff0 con 0x7f9958158b00 2024-09-13T17:59:14.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.202+0000 7f994ffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f995806c6d0 0x7f9958070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:14.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.202+0000 7f994ffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f995806c6d0 0x7f9958070b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55312/0 (socket says 172.21.15.111:55312) 2024-09-13T17:59:14.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.202+0000 7f994ffff640 1 -- 172.21.15.111:0/4216718435 learned_addr learned my addr 172.21.15.111:0/4216718435 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:14.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.202+0000 7f995cf9e640 1 --2- 172.21.15.111:0/4216718435 >> v2:172.21.15.188:3300/0 conn(0x7f9958158b00 0x7f995806c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:14.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.202+0000 7f994ffff640 1 -- 172.21.15.111:0/4216718435 >> v2:172.21.15.188:3300/0 conn(0x7f9958158b00 msgr2=0x7f995806c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:14.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.202+0000 7f994ffff640 1 --2- 172.21.15.111:0/4216718435 >> v2:172.21.15.188:3300/0 conn(0x7f9958158b00 0x7f995806c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:14.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.202+0000 7f994ffff640 1 -- 172.21.15.111:0/4216718435 >> v1:172.21.15.188:6789/0 conn(0x7f9958071050 legacy=0x7f99580714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:14.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.203+0000 7f994ffff640 1 -- 172.21.15.111:0/4216718435 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9958070310 con 0x7f995806c6d0 2024-09-13T17:59:14.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.203+0000 7f994ffff640 1 --2- 172.21.15.111:0/4216718435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f995806c6d0 0x7f9958070b10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f995806ade0 tx=0x7f994802c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb43e19efe16f5f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:14.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.203+0000 7f994f7fe640 1 -- 172.21.15.111:0/4216718435 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948032640 con 0x7f995806c6d0 2024-09-13T17:59:14.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.203+0000 7f994f7fe640 1 -- 172.21.15.111:0/4216718435 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99480327a0 con 0x7f995806c6d0 2024-09-13T17:59:14.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.203+0000 7f994f7fe640 1 -- 172.21.15.111:0/4216718435 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948032b30 con 0x7f995806c6d0 2024-09-13T17:59:14.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.204+0000 7f995f202640 1 -- 172.21.15.111:0/4216718435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f995806c6d0 msgr2=0x7f9958070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:14.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.204+0000 7f995f202640 1 --2- 172.21.15.111:0/4216718435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f995806c6d0 0x7f9958070b10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f995806ade0 tx=0x7f994802c940 comp rx=0 tx=0).stop 2024-09-13T17:59:14.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.204+0000 7f995f202640 1 -- 172.21.15.111:0/4216718435 shutdown_connections 2024-09-13T17:59:14.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.204+0000 7f995f202640 1 --2- 172.21.15.111:0/4216718435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f995806c6d0 0x7f9958070b10 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:14.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.204+0000 7f995f202640 1 --2- 172.21.15.111:0/4216718435 >> v2:172.21.15.188:3300/0 conn(0x7f9958158b00 0x7f995806c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:14.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.204+0000 7f995f202640 1 -- 172.21.15.111:0/4216718435 >> 172.21.15.111:0/4216718435 conn(0x7f995806a0b0 msgr2=0x7f995806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:14.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.204+0000 7f995f202640 1 -- 172.21.15.111:0/4216718435 shutdown_connections 2024-09-13T17:59:14.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.204+0000 7f995f202640 1 -- 172.21.15.111:0/4216718435 wait complete. 2024-09-13T17:59:14.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.205+0000 7f995f202640 1 Processor -- start 2024-09-13T17:59:14.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.205+0000 7f995f202640 1 -- start start 2024-09-13T17:59:14.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.206+0000 7f995f202640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f995806c6d0 0x7f99580c3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:14.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.206+0000 7f995f202640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9958071050 0x7f99580977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:14.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.206+0000 7f995f202640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9958158b00 0x7f995815b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:14.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.206+0000 7f995f202640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9958157d40 con 0x7f995806c6d0 2024-09-13T17:59:14.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.206+0000 7f995f202640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99581e3fa0 con 0x7f9958158b00 2024-09-13T17:59:14.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.206+0000 7f995f202640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99580c2650 con 0x7f9958071050 2024-09-13T17:59:14.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.206+0000 7f995cf9e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f995806c6d0 0x7f99580c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:14.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.206+0000 7f995cf9e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f995806c6d0 0x7f99580c3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55322/0 (socket says 172.21.15.111:55322) 2024-09-13T17:59:14.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.206+0000 7f995cf9e640 1 -- 172.21.15.111:0/3568022578 learned_addr learned my addr 172.21.15.111:0/3568022578 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:14.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.206+0000 7f995cf9e640 1 -- 172.21.15.111:0/3568022578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9958071050 msgr2=0x7f99580977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:14.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.206+0000 7f995cf9e640 1 --2- 172.21.15.111:0/3568022578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9958071050 0x7f99580977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:14.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.206+0000 7f995cf9e640 1 -- 172.21.15.111:0/3568022578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9958158b00 msgr2=0x7f995815b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:14.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.206+0000 7f995cf9e640 1 --2- 172.21.15.111:0/3568022578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9958158b00 0x7f995815b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:14.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.206+0000 7f995cf9e640 1 -- 172.21.15.111:0/3568022578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994800b770 con 0x7f995806c6d0 2024-09-13T17:59:14.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.206+0000 7f995cf9e640 1 --2- 172.21.15.111:0/3568022578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f995806c6d0 0x7f99580c3410 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f995400e870 tx=0x7f995400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:14.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.207+0000 7f994d7fa640 1 -- 172.21.15.111:0/3568022578 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99540094f0 con 0x7f995806c6d0 2024-09-13T17:59:14.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.207+0000 7f994d7fa640 1 -- 172.21.15.111:0/3568022578 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995400c940 con 0x7f995806c6d0 2024-09-13T17:59:14.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.207+0000 7f995f202640 1 -- 172.21.15.111:0/3568022578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99580c2930 con 0x7f995806c6d0 2024-09-13T17:59:14.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.207+0000 7f994d7fa640 1 -- 172.21.15.111:0/3568022578 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995400ccf0 con 0x7f995806c6d0 2024-09-13T17:59:14.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.207+0000 7f995f202640 1 -- 172.21.15.111:0/3568022578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99580c2ef0 con 0x7f995806c6d0 2024-09-13T17:59:14.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.207+0000 7f995f202640 1 -- 172.21.15.111:0/3568022578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99580cc160 con 0x7f995806c6d0 2024-09-13T17:59:14.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.208+0000 7f994d7fa640 1 -- 172.21.15.111:0/3568022578 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9954013470 con 0x7f995806c6d0 2024-09-13T17:59:14.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.208+0000 7f994d7fa640 1 --2- 172.21.15.111:0/3568022578 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f994406ac00 0x7f994406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:14.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.208+0000 7f994d7fa640 1 -- 172.21.15.111:0/3568022578 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f99540896e0 con 0x7f995806c6d0 2024-09-13T17:59:14.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.208+0000 7f994ffff640 1 --2- 172.21.15.111:0/3568022578 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f994406ac00 0x7f994406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:14.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.209+0000 7f994ffff640 1 --2- 172.21.15.111:0/3568022578 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f994406ac00 0x7f994406d0c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9948037000 tx=0x7f9948031000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:14.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.213+0000 7f994d7fa640 1 -- 172.21.15.111:0/3568022578 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f9954054620 con 0x7f995806c6d0 2024-09-13T17:59:14.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:14.375+0000 7f995f202640 1 -- 172.21.15.111:0/3568022578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9958070b10 con 0x7f995806c6d0 2024-09-13T17:59:15.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.047+0000 7f994d7fa640 1 -- 172.21.15.111:0/3568022578 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9954057c40 con 0x7f995806c6d0 2024-09-13T17:59:15.049 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' removed 2024-09-13T17:59:15.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.050+0000 7f995f202640 1 -- 172.21.15.111:0/3568022578 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f994406ac00 msgr2=0x7f994406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:15.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.050+0000 7f995f202640 1 --2- 172.21.15.111:0/3568022578 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f994406ac00 0x7f994406d0c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9948037000 tx=0x7f9948031000 comp rx=0 tx=0).stop 2024-09-13T17:59:15.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.050+0000 7f995f202640 1 -- 172.21.15.111:0/3568022578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f995806c6d0 msgr2=0x7f99580c3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:15.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.050+0000 7f995f202640 1 --2- 172.21.15.111:0/3568022578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f995806c6d0 0x7f99580c3410 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f995400e870 tx=0x7f995400ed30 comp rx=0 tx=0).stop 2024-09-13T17:59:15.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.050+0000 7f995f202640 1 -- 172.21.15.111:0/3568022578 shutdown_connections 2024-09-13T17:59:15.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.050+0000 7f995f202640 1 --2- 172.21.15.111:0/3568022578 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f994406ac00 0x7f994406d0c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:15.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.050+0000 7f995f202640 1 --2- 172.21.15.111:0/3568022578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9958158b00 0x7f995815b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:15.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.050+0000 7f995f202640 1 --2- 172.21.15.111:0/3568022578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9958071050 0x7f99580977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:15.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.050+0000 7f995f202640 1 --2- 172.21.15.111:0/3568022578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f995806c6d0 0x7f99580c3410 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:15.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.050+0000 7f995f202640 1 -- 172.21.15.111:0/3568022578 >> 172.21.15.111:0/3568022578 conn(0x7f995806a0b0 msgr2=0x7f99580ab140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:15.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.050+0000 7f995f202640 1 -- 172.21.15.111:0/3568022578 shutdown_connections 2024-09-13T17:59:15.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.050+0000 7f995f202640 1 -- 172.21.15.111:0/3568022578 wait complete. 2024-09-13T17:59:15.065 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T17:59:15.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.166+0000 7f2cc8a0d640 1 Processor -- start 2024-09-13T17:59:15.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.166+0000 7f2cc8a0d640 1 -- start start 2024-09-13T17:59:15.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.167+0000 7f2cc8a0d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cc415c500 0x7f2cc40d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:15.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.167+0000 7f2cc8a0d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2cc40cdff0 0x7f2cc40d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:15.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.167+0000 7f2cc8a0d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc40cd450 con 0x7f2cc40d2590 2024-09-13T17:59:15.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.167+0000 7f2cc8a0d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc40d29f0 con 0x7f2cc40cdff0 2024-09-13T17:59:15.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.167+0000 7f2cc8a0d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc40d0cf0 con 0x7f2cc415c500 2024-09-13T17:59:15.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.167+0000 7f2cc2d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2cc40cdff0 0x7f2cc40d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:15.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.167+0000 7f2cc2d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2cc40cdff0 0x7f2cc40d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55328/0 (socket says 172.21.15.111:55328) 2024-09-13T17:59:15.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.167+0000 7f2cc2d9d640 1 -- 172.21.15.111:0/3068011097 learned_addr learned my addr 172.21.15.111:0/3068011097 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:15.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.168+0000 7f2cc259c640 1 --2- 172.21.15.111:0/3068011097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cc415c500 0x7f2cc40d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:15.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.168+0000 7f2cc2d9d640 1 -- 172.21.15.111:0/3068011097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cc415c500 msgr2=0x7f2cc40d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:15.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.168+0000 7f2cc2d9d640 1 --2- 172.21.15.111:0/3068011097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cc415c500 0x7f2cc40d2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:15.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.168+0000 7f2cc2d9d640 1 -- 172.21.15.111:0/3068011097 >> v1:172.21.15.188:6789/0 conn(0x7f2cc40d2590 legacy=0x7f2cc40cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:15.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.168+0000 7f2cc2d9d640 1 -- 172.21.15.111:0/3068011097 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc40d0fd0 con 0x7f2cc40cdff0 2024-09-13T17:59:15.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.168+0000 7f2cc2d9d640 1 --2- 172.21.15.111:0/3068011097 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cc40cdff0 0x7f2cc40d0000 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009870 tx=0x7f2cbc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cb4a6287d7a7df server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:15.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.169+0000 7f2cc159a640 1 -- 172.21.15.111:0/3068011097 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc032440 con 0x7f2cc40cdff0 2024-09-13T17:59:15.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.169+0000 7f2cc159a640 1 -- 172.21.15.111:0/3068011097 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cbc0325a0 con 0x7f2cc40cdff0 2024-09-13T17:59:15.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.169+0000 7f2cc159a640 1 -- 172.21.15.111:0/3068011097 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc032930 con 0x7f2cc40cdff0 2024-09-13T17:59:15.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.169+0000 7f2cc8a0d640 1 -- 172.21.15.111:0/3068011097 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cc40cdff0 msgr2=0x7f2cc40d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:15.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.169+0000 7f2cc8a0d640 1 --2- 172.21.15.111:0/3068011097 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cc40cdff0 0x7f2cc40d0000 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009870 tx=0x7f2cbc02a1c0 comp rx=0 tx=0).stop 2024-09-13T17:59:15.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.169+0000 7f2cc8a0d640 1 -- 172.21.15.111:0/3068011097 shutdown_connections 2024-09-13T17:59:15.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.169+0000 7f2cc8a0d640 1 --2- 172.21.15.111:0/3068011097 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cc40cdff0 0x7f2cc40d0000 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:15.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.169+0000 7f2cc8a0d640 1 --2- 172.21.15.111:0/3068011097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cc415c500 0x7f2cc40d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:15.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.169+0000 7f2cc8a0d640 1 -- 172.21.15.111:0/3068011097 >> 172.21.15.111:0/3068011097 conn(0x7f2cc4089f60 msgr2=0x7f2cc409b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:15.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.169+0000 7f2cc8a0d640 1 -- 172.21.15.111:0/3068011097 shutdown_connections 2024-09-13T17:59:15.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.169+0000 7f2cc8a0d640 1 -- 172.21.15.111:0/3068011097 wait complete. 2024-09-13T17:59:15.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.171+0000 7f2cc8a0d640 1 Processor -- start 2024-09-13T17:59:15.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.171+0000 7f2cc8a0d640 1 -- start start 2024-09-13T17:59:15.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.171+0000 7f2cc8a0d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cc40cdff0 0x7f2cc415cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:15.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.171+0000 7f2cc8a0d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc40d2590 0x7f2cc4153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:15.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.171+0000 7f2cc8a0d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cc415c500 0x7f2cc415d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:15.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.171+0000 7f2cc8a0d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc41f3390 con 0x7f2cc40cdff0 2024-09-13T17:59:15.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.171+0000 7f2cc8a0d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc41f25d0 con 0x7f2cc40d2590 2024-09-13T17:59:15.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.171+0000 7f2cc8a0d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc41f28b0 con 0x7f2cc415c500 2024-09-13T17:59:15.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.171+0000 7f2cc259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cc40cdff0 0x7f2cc415cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:15.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.171+0000 7f2cc259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cc40cdff0 0x7f2cc415cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55334/0 (socket says 172.21.15.111:55334) 2024-09-13T17:59:15.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.171+0000 7f2cc259c640 1 -- 172.21.15.111:0/1933308716 learned_addr learned my addr 172.21.15.111:0/1933308716 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:15.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.171+0000 7f2cc1d9b640 1 --2- 172.21.15.111:0/1933308716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc40d2590 0x7f2cc4153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:15.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.171+0000 7f2cc2d9d640 1 --2- 172.21.15.111:0/1933308716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cc415c500 0x7f2cc415d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:15.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.172+0000 7f2cc259c640 1 -- 172.21.15.111:0/1933308716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cc415c500 msgr2=0x7f2cc415d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:15.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.172+0000 7f2cc259c640 1 --2- 172.21.15.111:0/1933308716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cc415c500 0x7f2cc415d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:15.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.172+0000 7f2cc259c640 1 -- 172.21.15.111:0/1933308716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc40d2590 msgr2=0x7f2cc4153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:15.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.172+0000 7f2cc259c640 1 --2- 172.21.15.111:0/1933308716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc40d2590 0x7f2cc4153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:15.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.172+0000 7f2cc259c640 1 -- 172.21.15.111:0/1933308716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc0094d0 con 0x7f2cc40cdff0 2024-09-13T17:59:15.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.172+0000 7f2cc1d9b640 1 --2- 172.21.15.111:0/1933308716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc40d2590 0x7f2cc4153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:15.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.172+0000 7f2cc2d9d640 1 --2- 172.21.15.111:0/1933308716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cc415c500 0x7f2cc415d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:15.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.172+0000 7f2cc259c640 1 --2- 172.21.15.111:0/1933308716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cc40cdff0 0x7f2cc415cc70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4007e40 tx=0x7f2cb400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:15.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.172+0000 7f2cbaffd640 1 -- 172.21.15.111:0/1933308716 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4014620 con 0x7f2cc40cdff0 2024-09-13T17:59:15.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.172+0000 7f2cbaffd640 1 -- 172.21.15.111:0/1933308716 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb400ee90 con 0x7f2cc40cdff0 2024-09-13T17:59:15.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.172+0000 7f2cbaffd640 1 -- 172.21.15.111:0/1933308716 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40149c0 con 0x7f2cc40cdff0 2024-09-13T17:59:15.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.172+0000 7f2cc8a0d640 1 -- 172.21.15.111:0/1933308716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc41f2b90 con 0x7f2cc40cdff0 2024-09-13T17:59:15.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.172+0000 7f2cc8a0d640 1 -- 172.21.15.111:0/1933308716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc415d7b0 con 0x7f2cc40cdff0 2024-09-13T17:59:15.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.173+0000 7f2cc8a0d640 1 -- 172.21.15.111:0/1933308716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc40d0970 con 0x7f2cc40cdff0 2024-09-13T17:59:15.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.173+0000 7f2cbaffd640 1 -- 172.21.15.111:0/1933308716 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2cb4014b20 con 0x7f2cc40cdff0 2024-09-13T17:59:15.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.174+0000 7f2cbaffd640 1 --2- 172.21.15.111:0/1933308716 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f2ca406ac00 0x7f2ca406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:15.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.174+0000 7f2cbaffd640 1 -- 172.21.15.111:0/1933308716 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f2cb4089c90 con 0x7f2cc40cdff0 2024-09-13T17:59:15.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.174+0000 7f2cc1d9b640 1 --2- 172.21.15.111:0/1933308716 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f2ca406ac00 0x7f2ca406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:15.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.174+0000 7f2cc1d9b640 1 --2- 172.21.15.111:0/1933308716 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f2ca406ac00 0x7f2ca406d0c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2cac007fd0 tx=0x7f2cac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:15.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.178+0000 7f2cbaffd640 1 -- 172.21.15.111:0/1933308716 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f2cb4054eb0 con 0x7f2cc40cdff0 2024-09-13T17:59:15.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:15.340+0000 7f2cc8a0d640 1 -- 172.21.15.111:0/1933308716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2cc406b160 con 0x7f2cc40cdff0 2024-09-13T17:59:16.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.049+0000 7f2cbaffd640 1 -- 172.21.15.111:0/1933308716 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2cb40584d0 con 0x7f2cc40cdff0 2024-09-13T17:59:16.051 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' removed 2024-09-13T17:59:16.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.053+0000 7f2cc8a0d640 1 -- 172.21.15.111:0/1933308716 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f2ca406ac00 msgr2=0x7f2ca406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:16.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.053+0000 7f2cc8a0d640 1 --2- 172.21.15.111:0/1933308716 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f2ca406ac00 0x7f2ca406d0c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2cac007fd0 tx=0x7f2cac009040 comp rx=0 tx=0).stop 2024-09-13T17:59:16.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.053+0000 7f2cc8a0d640 1 -- 172.21.15.111:0/1933308716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cc40cdff0 msgr2=0x7f2cc415cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:16.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.053+0000 7f2cc8a0d640 1 --2- 172.21.15.111:0/1933308716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cc40cdff0 0x7f2cc415cc70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4007e40 tx=0x7f2cb400e4b0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.053+0000 7f2cc8a0d640 1 -- 172.21.15.111:0/1933308716 shutdown_connections 2024-09-13T17:59:16.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.053+0000 7f2cc8a0d640 1 --2- 172.21.15.111:0/1933308716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cc415c500 0x7f2cc415d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.053+0000 7f2cc8a0d640 1 --2- 172.21.15.111:0/1933308716 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f2ca406ac00 0x7f2ca406d0c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.053+0000 7f2cc8a0d640 1 --2- 172.21.15.111:0/1933308716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc40d2590 0x7f2cc4153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.053+0000 7f2cc8a0d640 1 --2- 172.21.15.111:0/1933308716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2cc40cdff0 0x7f2cc415cc70 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.053+0000 7f2cc8a0d640 1 -- 172.21.15.111:0/1933308716 >> 172.21.15.111:0/1933308716 conn(0x7f2cc4089f60 msgr2=0x7f2cc406b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:16.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.053+0000 7f2cc8a0d640 1 -- 172.21.15.111:0/1933308716 shutdown_connections 2024-09-13T17:59:16.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.053+0000 7f2cc8a0d640 1 -- 172.21.15.111:0/1933308716 wait complete. 2024-09-13T17:59:16.067 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-13T17:59:16.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.166+0000 7f3082565640 1 Processor -- start 2024-09-13T17:59:16.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.166+0000 7f3082565640 1 -- start start 2024-09-13T17:59:16.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.167+0000 7f3082565640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c0b20a0 0x7f307c0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:16.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.167+0000 7f3082565640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f307c0cea30 0x7f307c0ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:16.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.167+0000 7f3082565640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307c15a790 con 0x7f307c15a3b0 2024-09-13T17:59:16.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.167+0000 7f3082565640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307c0ceec0 con 0x7f307c0cea30 2024-09-13T17:59:16.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.167+0000 7f3082565640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0b2680 con 0x7f307c0b20a0 2024-09-13T17:59:16.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.167+0000 7f3080b02640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f307c0cea30 0x7f307c0ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:16.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.167+0000 7f3080b02640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f307c0cea30 0x7f307c0ab9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55342/0 (socket says 172.21.15.111:55342) 2024-09-13T17:59:16.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.167+0000 7f3080b02640 1 -- 172.21.15.111:0/2959266690 learned_addr learned my addr 172.21.15.111:0/2959266690 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:16.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.168+0000 7f3073fff640 1 --2- 172.21.15.111:0/2959266690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c0b20a0 0x7f307c0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:16.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.168+0000 7f3080b02640 1 -- 172.21.15.111:0/2959266690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c0b20a0 msgr2=0x7f307c0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:16.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.168+0000 7f3080b02640 1 --2- 172.21.15.111:0/2959266690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c0b20a0 0x7f307c0ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.168+0000 7f3080b02640 1 -- 172.21.15.111:0/2959266690 >> v1:172.21.15.188:6789/0 conn(0x7f307c15a3b0 legacy=0x7f307c0b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:16.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.168+0000 7f3080b02640 1 -- 172.21.15.111:0/2959266690 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c0b2960 con 0x7f307c0cea30 2024-09-13T17:59:16.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.168+0000 7f3080b02640 1 --2- 172.21.15.111:0/2959266690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f307c0cea30 0x7f307c0ab9e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3074009e50 tx=0x7f307402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c9458fc8c9444d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:16.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.169+0000 7f307b7fe640 1 -- 172.21.15.111:0/2959266690 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307402aea0 con 0x7f307c0cea30 2024-09-13T17:59:16.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.169+0000 7f307b7fe640 1 -- 172.21.15.111:0/2959266690 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307402d050 con 0x7f307c0cea30 2024-09-13T17:59:16.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.169+0000 7f307b7fe640 1 -- 172.21.15.111:0/2959266690 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074031820 con 0x7f307c0cea30 2024-09-13T17:59:16.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.169+0000 7f3082565640 1 -- 172.21.15.111:0/2959266690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f307c0cea30 msgr2=0x7f307c0ab9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:16.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.169+0000 7f3082565640 1 --2- 172.21.15.111:0/2959266690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f307c0cea30 0x7f307c0ab9e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3074009e50 tx=0x7f307402a6c0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.169+0000 7f3082565640 1 -- 172.21.15.111:0/2959266690 shutdown_connections 2024-09-13T17:59:16.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.169+0000 7f3082565640 1 --2- 172.21.15.111:0/2959266690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f307c0cea30 0x7f307c0ab9e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.169+0000 7f3082565640 1 --2- 172.21.15.111:0/2959266690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c0b20a0 0x7f307c0ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.169+0000 7f3082565640 1 -- 172.21.15.111:0/2959266690 >> 172.21.15.111:0/2959266690 conn(0x7f307c06a0b0 msgr2=0x7f307c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:16.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.169+0000 7f3082565640 1 -- 172.21.15.111:0/2959266690 shutdown_connections 2024-09-13T17:59:16.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.169+0000 7f3082565640 1 -- 172.21.15.111:0/2959266690 wait complete. 2024-09-13T17:59:16.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.170+0000 7f3082565640 1 Processor -- start 2024-09-13T17:59:16.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.170+0000 7f3082565640 1 -- start start 2024-09-13T17:59:16.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.171+0000 7f3082565640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c0b20a0 0x7f307c1f0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:16.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.171+0000 7f3082565640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f307c0cea30 0x7f307c157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:16.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.171+0000 7f3082565640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f307c15a3b0 0x7f307c15c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:16.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.171+0000 7f3082565640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c158510 con 0x7f307c0cea30 2024-09-13T17:59:16.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.171+0000 7f3082565640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0aec70 con 0x7f307c15a3b0 2024-09-13T17:59:16.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.171+0000 7f3082565640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0aef50 con 0x7f307c0b20a0 2024-09-13T17:59:16.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.171+0000 7f3073fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f307c0cea30 0x7f307c157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.171+0000 7f3073fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f307c0cea30 0x7f307c157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55352/0 (socket says 172.21.15.111:55352) 2024-09-13T17:59:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.171+0000 7f3073fff640 1 -- 172.21.15.111:0/3583708597 learned_addr learned my addr 172.21.15.111:0/3583708597 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.171+0000 7f3080b02640 1 --2- 172.21.15.111:0/3583708597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f307c15a3b0 0x7f307c15c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.171+0000 7f307bfff640 1 --2- 172.21.15.111:0/3583708597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c0b20a0 0x7f307c1f0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.171+0000 7f3073fff640 1 -- 172.21.15.111:0/3583708597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c0b20a0 msgr2=0x7f307c1f0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.171+0000 7f3073fff640 1 --2- 172.21.15.111:0/3583708597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c0b20a0 0x7f307c1f0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.171+0000 7f3073fff640 1 -- 172.21.15.111:0/3583708597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f307c15a3b0 msgr2=0x7f307c15c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.171+0000 7f3073fff640 1 --2- 172.21.15.111:0/3583708597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f307c15a3b0 0x7f307c15c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.171+0000 7f3073fff640 1 -- 172.21.15.111:0/3583708597 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3074009b30 con 0x7f307c0cea30 2024-09-13T17:59:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.172+0000 7f307bfff640 1 --2- 172.21.15.111:0/3583708597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c0b20a0 0x7f307c1f0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.172+0000 7f3080b02640 1 --2- 172.21.15.111:0/3583708597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f307c15a3b0 0x7f307c15c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.172+0000 7f3073fff640 1 --2- 172.21.15.111:0/3583708597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f307c0cea30 0x7f307c157fd0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f306400c7f0 tx=0x7f306400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.172+0000 7f30797fa640 1 -- 172.21.15.111:0/3583708597 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064004220 con 0x7f307c0cea30 2024-09-13T17:59:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.172+0000 7f30797fa640 1 -- 172.21.15.111:0/3583708597 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30640043a0 con 0x7f307c0cea30 2024-09-13T17:59:16.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.172+0000 7f30797fa640 1 -- 172.21.15.111:0/3583708597 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064007860 con 0x7f307c0cea30 2024-09-13T17:59:16.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.172+0000 7f3082565640 1 -- 172.21.15.111:0/3583708597 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307c0af230 con 0x7f307c0cea30 2024-09-13T17:59:16.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.172+0000 7f3082565640 1 -- 172.21.15.111:0/3583708597 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c0b1730 con 0x7f307c0cea30 2024-09-13T17:59:16.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.173+0000 7f3082565640 1 -- 172.21.15.111:0/3583708597 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307c0aedb0 con 0x7f307c0cea30 2024-09-13T17:59:16.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.173+0000 7f30797fa640 1 -- 172.21.15.111:0/3583708597 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3064007bd0 con 0x7f307c0cea30 2024-09-13T17:59:16.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.174+0000 7f30797fa640 1 --2- 172.21.15.111:0/3583708597 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f305c06ac00 0x7f305c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:16.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.174+0000 7f30797fa640 1 -- 172.21.15.111:0/3583708597 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f30640891e0 con 0x7f307c0cea30 2024-09-13T17:59:16.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.174+0000 7f307bfff640 1 --2- 172.21.15.111:0/3583708597 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f305c06ac00 0x7f305c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:16.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.174+0000 7f307bfff640 1 --2- 172.21.15.111:0/3583708597 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f305c06ac00 0x7f305c06d0c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f306c006fd0 tx=0x7f306c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:16.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.178+0000 7f30797fa640 1 -- 172.21.15.111:0/3583708597 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f307c0aedb0 con 0x7f307c0cea30 2024-09-13T17:59:16.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.342+0000 7f3082565640 1 -- 172.21.15.111:0/3583708597 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f307c0af750 con 0x7f307c0cea30 2024-09-13T17:59:16.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.346+0000 7f30797fa640 1 -- 172.21.15.111:0/3583708597 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f307c0af750 con 0x7f307c0cea30 2024-09-13T17:59:16.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.348+0000 7f3082565640 1 -- 172.21.15.111:0/3583708597 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f305c06ac00 msgr2=0x7f305c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:16.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.348+0000 7f3082565640 1 --2- 172.21.15.111:0/3583708597 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f305c06ac00 0x7f305c06d0c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f306c006fd0 tx=0x7f306c008040 comp rx=0 tx=0).stop 2024-09-13T17:59:16.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.348+0000 7f3082565640 1 -- 172.21.15.111:0/3583708597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f307c0cea30 msgr2=0x7f307c157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:16.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.348+0000 7f3082565640 1 --2- 172.21.15.111:0/3583708597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f307c0cea30 0x7f307c157fd0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f306400c7f0 tx=0x7f306400ccb0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.348+0000 7f3082565640 1 -- 172.21.15.111:0/3583708597 shutdown_connections 2024-09-13T17:59:16.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.348+0000 7f3082565640 1 --2- 172.21.15.111:0/3583708597 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f305c06ac00 0x7f305c06d0c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.348+0000 7f3082565640 1 --2- 172.21.15.111:0/3583708597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f307c15a3b0 0x7f307c15c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.348+0000 7f3082565640 1 --2- 172.21.15.111:0/3583708597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f307c0cea30 0x7f307c157fd0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.348+0000 7f3082565640 1 --2- 172.21.15.111:0/3583708597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c0b20a0 0x7f307c1f0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.348+0000 7f3082565640 1 -- 172.21.15.111:0/3583708597 >> 172.21.15.111:0/3583708597 conn(0x7f307c06a0b0 msgr2=0x7f307c0ac4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:16.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.348+0000 7f3082565640 1 -- 172.21.15.111:0/3583708597 shutdown_connections 2024-09-13T17:59:16.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.348+0000 7f3082565640 1 -- 172.21.15.111:0/3583708597 wait complete. 2024-09-13T17:59:16.361 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create' 2024-09-13T17:59:16.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.457+0000 7ff06f40d640 1 Processor -- start 2024-09-13T17:59:16.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.457+0000 7ff06f40d640 1 -- start start 2024-09-13T17:59:16.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.458+0000 7ff06f40d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff068071d30 0x7ff068070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:16.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.458+0000 7ff06f40d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680c8920 0x7ff0680c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:16.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.458+0000 7ff06f40d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0680abdf0 con 0x7ff068072110 2024-09-13T17:59:16.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.458+0000 7ff06f40d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0680c9310 con 0x7ff068071d30 2024-09-13T17:59:16.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.458+0000 7ff06f40d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0680b2280 con 0x7ff0680c8920 2024-09-13T17:59:16.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.458+0000 7ff06c9a8640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff068072110 0x7ff0680b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43050/0 (socket says 172.21.15.111:43050) 2024-09-13T17:59:16.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.458+0000 7ff06c9a8640 1 -- 172.21.15.111:0/1543320811 learned_addr learned my addr 172.21.15.111:0/1543320811 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:16.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.458+0000 7ff06d9aa640 1 --2- 172.21.15.111:0/1543320811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680c8920 0x7ff0680c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:16.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.458+0000 7ff06d1a9640 1 --2- 172.21.15.111:0/1543320811 >> v2:172.21.15.188:3300/0 conn(0x7ff068071d30 0x7ff068070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:16.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.458+0000 7ff05ffff640 1 -- 172.21.15.111:0/1543320811 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65312057 0 0) 0x7ff0680abdf0 con 0x7ff068072110 2024-09-13T17:59:16.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.458+0000 7ff05ffff640 1 -- 172.21.15.111:0/1543320811 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff054003520 con 0x7ff068072110 2024-09-13T17:59:16.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff05ffff640 1 -- 172.21.15.111:0/1543320811 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3376310007 0 0) 0x7ff054003520 con 0x7ff068072110 2024-09-13T17:59:16.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff05ffff640 1 -- 172.21.15.111:0/1543320811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680c8920 msgr2=0x7ff0680c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:16.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff05ffff640 1 --2- 172.21.15.111:0/1543320811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680c8920 0x7ff0680c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff05ffff640 1 -- 172.21.15.111:0/1543320811 >> v2:172.21.15.188:3300/0 conn(0x7ff068071d30 msgr2=0x7ff068070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:16.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff05ffff640 1 --2- 172.21.15.111:0/1543320811 >> v2:172.21.15.188:3300/0 conn(0x7ff068071d30 0x7ff068070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff05ffff640 1 -- 172.21.15.111:0/1543320811 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0680ab1e0 con 0x7ff068072110 2024-09-13T17:59:16.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff05ffff640 1 -- 172.21.15.111:0/1543320811 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7ff058001670 con 0x7ff068072110 2024-09-13T17:59:16.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff05ffff640 1 -- 172.21.15.111:0/1543320811 >> v1:172.21.15.111:6789/0 conn(0x7ff068072110 legacy=0x7ff0680b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:16.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff05ffff640 1 --2- 172.21.15.111:0/1543320811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff054003970 0x7ff054003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:16.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff05ffff640 1 --2- 172.21.15.111:0/1543320811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680c8920 0x7ff0540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:16.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff05ffff640 1 --2- 172.21.15.111:0/1543320811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff054004d00 0x7ff054005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:16.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff05ffff640 1 -- 172.21.15.111:0/1543320811 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0680c9310 con 0x7ff054003970 2024-09-13T17:59:16.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff05ffff640 1 -- 172.21.15.111:0/1543320811 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0680b2280 con 0x7ff054004d00 2024-09-13T17:59:16.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff05ffff640 1 -- 172.21.15.111:0/1543320811 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05400db90 con 0x7ff0680c8920 2024-09-13T17:59:16.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff06d1a9640 1 --2- 172.21.15.111:0/1543320811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff054003970 0x7ff054003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:16.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff06d9aa640 1 --2- 172.21.15.111:0/1543320811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff054004d00 0x7ff054005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:16.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff06c9a8640 1 --2- 172.21.15.111:0/1543320811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680c8920 0x7ff0540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:16.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff06d1a9640 1 -- 172.21.15.111:0/1543320811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680c8920 msgr2=0x7ff0540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:16.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff06d1a9640 1 --2- 172.21.15.111:0/1543320811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680c8920 0x7ff0540047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff06d1a9640 1 -- 172.21.15.111:0/1543320811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff054004d00 msgr2=0x7ff054005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:16.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff06d1a9640 1 --2- 172.21.15.111:0/1543320811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff054004d00 0x7ff054005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff06d1a9640 1 -- 172.21.15.111:0/1543320811 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff054003520 con 0x7ff054003970 2024-09-13T17:59:16.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.459+0000 7ff06d9aa640 1 --2- 172.21.15.111:0/1543320811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff054004d00 0x7ff054005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:16.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.460+0000 7ff06d1a9640 1 --2- 172.21.15.111:0/1543320811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff054003970 0x7ff054003d90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff064009ac0 tx=0x7ff0540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395bd73713ae1908 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:16.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.460+0000 7ff05ffff640 1 -- 172.21.15.111:0/1543320811 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff064031400 con 0x7ff054003970 2024-09-13T17:59:16.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.460+0000 7ff05ffff640 1 -- 172.21.15.111:0/1543320811 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06402d050 con 0x7ff054003970 2024-09-13T17:59:16.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.460+0000 7ff05ffff640 1 -- 172.21.15.111:0/1543320811 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff064031780 con 0x7ff054003970 2024-09-13T17:59:16.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.460+0000 7ff06f40d640 1 -- 172.21.15.111:0/1543320811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff054003970 msgr2=0x7ff054003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:16.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.460+0000 7ff06f40d640 1 --2- 172.21.15.111:0/1543320811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff054003970 0x7ff054003d90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff064009ac0 tx=0x7ff0540057f0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.460+0000 7ff06d9aa640 1 -- 172.21.15.111:0/1543320811 reap_dead start 2024-09-13T17:59:16.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.460+0000 7ff06f40d640 1 -- 172.21.15.111:0/1543320811 shutdown_connections 2024-09-13T17:59:16.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.460+0000 7ff06f40d640 1 -- 172.21.15.111:0/1543320811 >> 172.21.15.111:0/1543320811 conn(0x7ff068069ed0 msgr2=0x7ff06806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:16.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.460+0000 7ff06f40d640 1 -- 172.21.15.111:0/1543320811 shutdown_connections 2024-09-13T17:59:16.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.461+0000 7ff06f40d640 1 -- 172.21.15.111:0/1543320811 wait complete. 2024-09-13T17:59:16.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.462+0000 7ff06f40d640 1 Processor -- start 2024-09-13T17:59:16.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.462+0000 7ff06f40d640 1 -- start start 2024-09-13T17:59:16.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.462+0000 7ff06f40d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff054003970 0x7ff06815ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:16.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.462+0000 7ff06f40d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff06815a370 0x7ff0681547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:16.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.462+0000 7ff06f40d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff068159d50 0x7ff06806d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:16.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.462+0000 7ff06f40d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff068159020 con 0x7ff054003970 2024-09-13T17:59:16.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.462+0000 7ff06f40d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff068154ce0 con 0x7ff068159d50 2024-09-13T17:59:16.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.462+0000 7ff06f40d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06815a870 con 0x7ff06815a370 2024-09-13T17:59:16.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.462+0000 7ff06d1a9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff054003970 0x7ff06815ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:16.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.462+0000 7ff06d1a9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff054003970 0x7ff06815ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55366/0 (socket says 172.21.15.111:55366) 2024-09-13T17:59:16.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.462+0000 7ff06d1a9640 1 -- 172.21.15.111:0/814194846 learned_addr learned my addr 172.21.15.111:0/814194846 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:16.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.463+0000 7ff06d9aa640 1 --2- 172.21.15.111:0/814194846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff068159d50 0x7ff06806d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:16.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.463+0000 7ff06c9a8640 1 --2- 172.21.15.111:0/814194846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff06815a370 0x7ff0681547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:16.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.463+0000 7ff06d1a9640 1 -- 172.21.15.111:0/814194846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff06815a370 msgr2=0x7ff0681547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:16.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.463+0000 7ff06d1a9640 1 --2- 172.21.15.111:0/814194846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff06815a370 0x7ff0681547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.463+0000 7ff06d1a9640 1 -- 172.21.15.111:0/814194846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff068159d50 msgr2=0x7ff06806d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:16.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.463+0000 7ff06d1a9640 1 --2- 172.21.15.111:0/814194846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff068159d50 0x7ff06806d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:16.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.463+0000 7ff06d1a9640 1 -- 172.21.15.111:0/814194846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff068159270 con 0x7ff054003970 2024-09-13T17:59:16.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.463+0000 7ff06d9aa640 1 --2- 172.21.15.111:0/814194846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff068159d50 0x7ff06806d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:59:16.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.463+0000 7ff06d1a9640 1 --2- 172.21.15.111:0/814194846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff054003970 0x7ff06815ad00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff064034040 tx=0x7ff064004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:16.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.463+0000 7ff05dffb640 1 -- 172.21.15.111:0/814194846 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff064003bb0 con 0x7ff054003970 2024-09-13T17:59:16.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.463+0000 7ff05dffb640 1 -- 172.21.15.111:0/814194846 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff064003d10 con 0x7ff054003970 2024-09-13T17:59:16.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.463+0000 7ff06f40d640 1 -- 172.21.15.111:0/814194846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff068159550 con 0x7ff054003970 2024-09-13T17:59:16.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.463+0000 7ff05dffb640 1 -- 172.21.15.111:0/814194846 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff064044650 con 0x7ff054003970 2024-09-13T17:59:16.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.463+0000 7ff06f40d640 1 -- 172.21.15.111:0/814194846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff068155140 con 0x7ff054003970 2024-09-13T17:59:16.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.464+0000 7ff06f40d640 1 -- 172.21.15.111:0/814194846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06815ab50 con 0x7ff054003970 2024-09-13T17:59:16.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.465+0000 7ff05dffb640 1 -- 172.21.15.111:0/814194846 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff064038070 con 0x7ff054003970 2024-09-13T17:59:16.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.465+0000 7ff05dffb640 1 --2- 172.21.15.111:0/814194846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff04c06ac00 0x7ff04c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:16.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.465+0000 7ff05dffb640 1 -- 172.21.15.111:0/814194846 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7ff0640a9200 con 0x7ff054003970 2024-09-13T17:59:16.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.465+0000 7ff06c9a8640 1 --2- 172.21.15.111:0/814194846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff04c06ac00 0x7ff04c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:16.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.466+0000 7ff06c9a8640 1 --2- 172.21.15.111:0/814194846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff04c06ac00 0x7ff04c06d0c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff058001cf0 tx=0x7ff058001c80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:16.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.469+0000 7ff05dffb640 1 -- 172.21.15.111:0/814194846 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff0640745f0 con 0x7ff054003970 2024-09-13T17:59:16.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:16.694+0000 7ff06f40d640 1 -- 172.21.15.111:0/814194846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]} v 0) v1 -- 0x7ff0680c8dd0 con 0x7ff054003970 2024-09-13T17:59:17.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.060+0000 7ff05dffb640 1 -- 172.21.15.111:0/814194846 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]}]=0 v125) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff064077c10 con 0x7ff054003970 2024-09-13T17:59:17.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.062+0000 7ff06f40d640 1 -- 172.21.15.111:0/814194846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff04c06ac00 msgr2=0x7ff04c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.062+0000 7ff06f40d640 1 --2- 172.21.15.111:0/814194846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff04c06ac00 0x7ff04c06d0c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff058001cf0 tx=0x7ff058001c80 comp rx=0 tx=0).stop 2024-09-13T17:59:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.062+0000 7ff06f40d640 1 -- 172.21.15.111:0/814194846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff054003970 msgr2=0x7ff06815ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.062+0000 7ff06f40d640 1 --2- 172.21.15.111:0/814194846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff054003970 0x7ff06815ad00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff064034040 tx=0x7ff064004730 comp rx=0 tx=0).stop 2024-09-13T17:59:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.062+0000 7ff06f40d640 1 -- 172.21.15.111:0/814194846 shutdown_connections 2024-09-13T17:59:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.062+0000 7ff06f40d640 1 --2- 172.21.15.111:0/814194846 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff04c06ac00 0x7ff04c06d0c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.062+0000 7ff06f40d640 1 --2- 172.21.15.111:0/814194846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff068159d50 0x7ff06806d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.062+0000 7ff06f40d640 1 --2- 172.21.15.111:0/814194846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff06815a370 0x7ff0681547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.062+0000 7ff06f40d640 1 --2- 172.21.15.111:0/814194846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff054003970 0x7ff06815ad00 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.062+0000 7ff06f40d640 1 -- 172.21.15.111:0/814194846 >> 172.21.15.111:0/814194846 conn(0x7ff068069ed0 msgr2=0x7ff0680ade10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.062+0000 7ff06f40d640 1 -- 172.21.15.111:0/814194846 shutdown_connections 2024-09-13T17:59:17.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.062+0000 7ff06f40d640 1 -- 172.21.15.111:0/814194846 wait complete. 2024-09-13T17:59:17.075 INFO:tasks.cephfs_test_runner:MDS starts to handle client caps when it enters clientreplay stage. ... ok 2024-09-13T17:59:17.076 INFO:tasks.cephfs_test_runner: 2024-09-13T17:59:17.076 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-13T17:59:17.076 INFO:tasks.cephfs_test_runner:Ran 1 test in 65.905s 2024-09-13T17:59:17.076 INFO:tasks.cephfs_test_runner: 2024-09-13T17:59:17.076 INFO:tasks.cephfs_test_runner:OK 2024-09-13T17:59:17.076 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-13T17:59:17.085 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-13T17:59:17.101 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-13T17:59:17.101 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:59:17.101 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:59:17.118 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056025,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14842 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T17:59:17.118 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T17:59:17.118 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:59:17.173 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056025,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14842 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T17:59:17.174 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:59:17.174 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:59:17.194 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T17:59:17.194 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-09-13T17:59:17.194 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:59:17.253 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056014,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T17:59:17.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:59:17.254 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-09-13T17:59:17.269 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T17:59:17.269 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:59:17.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:59:17.269 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T17:59:17.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:59:17.335 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:59:17.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:59:17.400 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link delete ceph-brx 2024-09-13T17:59:17.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:59:17.497 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-09-13T17:59:17.516 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-13T17:59:17.516 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-13T17:59:17.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:59:17.517 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-13T17:59:17.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:59:17.587 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T17:59:17.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:59:17.655 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-09-13T17:59:17.753 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T17:59:17.765 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T17:59:17.766 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:59:17.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.863+0000 7f9a6ed45640 1 Processor -- start 2024-09-13T17:59:17.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.863+0000 7f9a6ed45640 1 -- start start 2024-09-13T17:59:17.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.864+0000 7f9a6ed45640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9a6806c4c0 0x7f9a6806c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.864+0000 7f9a6ed45640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9a680cbea0 0x7f9a680cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.864+0000 7f9a6ed45640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a6806dab0 con 0x7f9a680b2590 2024-09-13T17:59:17.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.864+0000 7f9a6ed45640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a6806deb0 con 0x7f9a6806c4c0 2024-09-13T17:59:17.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.864+0000 7f9a6ed45640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a680b2170 con 0x7f9a680cbea0 2024-09-13T17:59:17.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.864+0000 7f9a6cae1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9a6806c4c0 0x7f9a6806c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.864+0000 7f9a6cae1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9a6806c4c0 0x7f9a6806c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55372/0 (socket says 172.21.15.111:55372) 2024-09-13T17:59:17.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.864+0000 7f9a6cae1640 1 -- 172.21.15.111:0/2437334978 learned_addr learned my addr 172.21.15.111:0/2437334978 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:17.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.865+0000 7f9a6cae1640 1 -- 172.21.15.111:0/2437334978 >> v2:172.21.15.188:3300/0 conn(0x7f9a680cbea0 msgr2=0x7f9a680cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:17.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.865+0000 7f9a6cae1640 1 --2- 172.21.15.111:0/2437334978 >> v2:172.21.15.188:3300/0 conn(0x7f9a680cbea0 0x7f9a680cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.865+0000 7f9a6cae1640 1 -- 172.21.15.111:0/2437334978 >> v1:172.21.15.188:6789/0 conn(0x7f9a680b2590 legacy=0x7f9a6806d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:17.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.865+0000 7f9a6cae1640 1 -- 172.21.15.111:0/2437334978 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a680cc870 con 0x7f9a6806c4c0 2024-09-13T17:59:17.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.865+0000 7f9a6cae1640 1 --2- 172.21.15.111:0/2437334978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a6806c4c0 0x7f9a6806c8a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9a680c88d0 tx=0x7f9a6402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799bcf65e22ca8e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:17.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.865+0000 7f9a5f7fe640 1 -- 172.21.15.111:0/2437334978 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64032440 con 0x7f9a6806c4c0 2024-09-13T17:59:17.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.865+0000 7f9a5f7fe640 1 -- 172.21.15.111:0/2437334978 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a640325a0 con 0x7f9a6806c4c0 2024-09-13T17:59:17.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.865+0000 7f9a5f7fe640 1 -- 172.21.15.111:0/2437334978 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64032930 con 0x7f9a6806c4c0 2024-09-13T17:59:17.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.866+0000 7f9a6ed45640 1 -- 172.21.15.111:0/2437334978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a6806c4c0 msgr2=0x7f9a6806c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.866+0000 7f9a6ed45640 1 --2- 172.21.15.111:0/2437334978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a6806c4c0 0x7f9a6806c8a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9a680c88d0 tx=0x7f9a6402a230 comp rx=0 tx=0).stop 2024-09-13T17:59:17.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.866+0000 7f9a6ed45640 1 -- 172.21.15.111:0/2437334978 shutdown_connections 2024-09-13T17:59:17.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.866+0000 7f9a6ed45640 1 --2- 172.21.15.111:0/2437334978 >> v2:172.21.15.188:3300/0 conn(0x7f9a680cbea0 0x7f9a680cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.866+0000 7f9a6ed45640 1 --2- 172.21.15.111:0/2437334978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a6806c4c0 0x7f9a6806c8a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.866+0000 7f9a6ed45640 1 -- 172.21.15.111:0/2437334978 >> 172.21.15.111:0/2437334978 conn(0x7f9a68069ed0 msgr2=0x7f9a6806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:17.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.866+0000 7f9a6ed45640 1 -- 172.21.15.111:0/2437334978 shutdown_connections 2024-09-13T17:59:17.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.866+0000 7f9a6ed45640 1 -- 172.21.15.111:0/2437334978 wait complete. 2024-09-13T17:59:17.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.867+0000 7f9a6ed45640 1 Processor -- start 2024-09-13T17:59:17.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.867+0000 7f9a6ed45640 1 -- start start 2024-09-13T17:59:17.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.868+0000 7f9a6ed45640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a6806c4c0 0x7f9a6815bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.868+0000 7f9a6ed45640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a680b2590 0x7f9a680c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.868+0000 7f9a6ed45640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a680cbea0 0x7f9a681f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.868+0000 7f9a6ed45640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a680c62f0 con 0x7f9a680b2590 2024-09-13T17:59:17.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.868+0000 7f9a6ed45640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a680cd2e0 con 0x7f9a680cbea0 2024-09-13T17:59:17.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.868+0000 7f9a6ed45640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a681f2ef0 con 0x7f9a6806c4c0 2024-09-13T17:59:17.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.868+0000 7f9a5ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a680b2590 0x7f9a680c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.868+0000 7f9a5ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a680b2590 0x7f9a680c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55378/0 (socket says 172.21.15.111:55378) 2024-09-13T17:59:17.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.868+0000 7f9a5ffff640 1 -- 172.21.15.111:0/3898289034 learned_addr learned my addr 172.21.15.111:0/3898289034 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:17.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.868+0000 7f9a6d2e2640 1 --2- 172.21.15.111:0/3898289034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a680cbea0 0x7f9a681f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.868+0000 7f9a6cae1640 1 --2- 172.21.15.111:0/3898289034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a6806c4c0 0x7f9a6815bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.869+0000 7f9a5ffff640 1 -- 172.21.15.111:0/3898289034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a6806c4c0 msgr2=0x7f9a6815bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.869+0000 7f9a5ffff640 1 --2- 172.21.15.111:0/3898289034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a6806c4c0 0x7f9a6815bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.869+0000 7f9a5ffff640 1 -- 172.21.15.111:0/3898289034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a680cbea0 msgr2=0x7f9a681f29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.869+0000 7f9a5ffff640 1 --2- 172.21.15.111:0/3898289034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a680cbea0 0x7f9a681f29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.869+0000 7f9a5ffff640 1 -- 172.21.15.111:0/3898289034 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a640094d0 con 0x7f9a680b2590 2024-09-13T17:59:17.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.869+0000 7f9a5ffff640 1 --2- 172.21.15.111:0/3898289034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a680b2590 0x7f9a680c5db0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9a5800c410 tx=0x7f9a5800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:17.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.869+0000 7f9a5d7fa640 1 -- 172.21.15.111:0/3898289034 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a580035f0 con 0x7f9a680b2590 2024-09-13T17:59:17.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.869+0000 7f9a5d7fa640 1 -- 172.21.15.111:0/3898289034 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a58003750 con 0x7f9a680b2590 2024-09-13T17:59:17.873 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.869+0000 7f9a5d7fa640 1 -- 172.21.15.111:0/3898289034 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a58003b40 con 0x7f9a680b2590 2024-09-13T17:59:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.869+0000 7f9a6ed45640 1 -- 172.21.15.111:0/3898289034 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a681f31d0 con 0x7f9a680b2590 2024-09-13T17:59:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.869+0000 7f9a6ed45640 1 -- 172.21.15.111:0/3898289034 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a681f3790 con 0x7f9a680b2590 2024-09-13T17:59:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.870+0000 7f9a6ed45640 1 -- 172.21.15.111:0/3898289034 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a68072cb0 con 0x7f9a680b2590 2024-09-13T17:59:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.870+0000 7f9a5d7fa640 1 -- 172.21.15.111:0/3898289034 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9a58003ca0 con 0x7f9a680b2590 2024-09-13T17:59:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.871+0000 7f9a5d7fa640 1 --2- 172.21.15.111:0/3898289034 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9a5406ac30 0x7f9a5406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.871+0000 7f9a5d7fa640 1 -- 172.21.15.111:0/3898289034 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f9a58088230 con 0x7f9a680b2590 2024-09-13T17:59:17.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.871+0000 7f9a6cae1640 1 --2- 172.21.15.111:0/3898289034 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9a5406ac30 0x7f9a5406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.871+0000 7f9a6cae1640 1 --2- 172.21.15.111:0/3898289034 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9a5406ac30 0x7f9a5406d0f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9a64034000 tx=0x7f9a6402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:17.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:17.875+0000 7f9a5d7fa640 1 -- 172.21.15.111:0/3898289034 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f9a580536b0 con 0x7f9a680b2590 2024-09-13T17:59:18.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.033+0000 7f9a6ed45640 1 -- 172.21.15.111:0/3898289034 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a6806e3a0 con 0x7f9a5406ac30 2024-09-13T17:59:18.037 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.035+0000 7f9a5d7fa640 1 -- 172.21.15.111:0/3898289034 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51142 (secure 0 0 0) 0x7f9a64040070 con 0x7f9a5406ac30 2024-09-13T17:59:18.037 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:18.038 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-09-13T17:59:18.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.037+0000 7f9a6ed45640 1 -- 172.21.15.111:0/3898289034 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9a5406ac30 msgr2=0x7f9a5406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.037+0000 7f9a6ed45640 1 --2- 172.21.15.111:0/3898289034 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9a5406ac30 0x7f9a5406d0f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9a64034000 tx=0x7f9a6402e000 comp rx=0 tx=0).stop 2024-09-13T17:59:18.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.037+0000 7f9a6ed45640 1 -- 172.21.15.111:0/3898289034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a680b2590 msgr2=0x7f9a680c5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.037+0000 7f9a6ed45640 1 --2- 172.21.15.111:0/3898289034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a680b2590 0x7f9a680c5db0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9a5800c410 tx=0x7f9a5800c8d0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.037+0000 7f9a6ed45640 1 -- 172.21.15.111:0/3898289034 shutdown_connections 2024-09-13T17:59:18.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.037+0000 7f9a6ed45640 1 --2- 172.21.15.111:0/3898289034 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9a5406ac30 0x7f9a5406d0f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.037+0000 7f9a6ed45640 1 --2- 172.21.15.111:0/3898289034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a680cbea0 0x7f9a681f29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.037+0000 7f9a6ed45640 1 --2- 172.21.15.111:0/3898289034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a680b2590 0x7f9a680c5db0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.037+0000 7f9a6ed45640 1 --2- 172.21.15.111:0/3898289034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a6806c4c0 0x7f9a6815bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.037+0000 7f9a6ed45640 1 -- 172.21.15.111:0/3898289034 >> 172.21.15.111:0/3898289034 conn(0x7f9a68069ed0 msgr2=0x7f9a680af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:18.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.038+0000 7f9a6ed45640 1 -- 172.21.15.111:0/3898289034 shutdown_connections 2024-09-13T17:59:18.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.038+0000 7f9a6ed45640 1 -- 172.21.15.111:0/3898289034 wait complete. 2024-09-13T17:59:18.050 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-09-13T17:59:16.288918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221308,"kb_used_data":5148,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753412,"statfs":{"total":773094113280,"available":772867493888,"internally_reserved":0,"allocated":5271552,"data_stored":2970725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14667,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-86,"num_read_kb":-88,"num_write":-74,"num_write_kb":-39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001174"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":187,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:59:08.625238+0000","last_change":"2024-09-13T17:57:15.560506+0000","last_active":"2024-09-13T17:59:08.625238+0000","last_peered":"2024-09-13T17:59:08.625238+0000","last_clean":"2024-09-13T17:59:08.625238+0000","last_became_active":"2024-09-13T17:57:15.559612+0000","last_became_peered":"2024-09-13T17:57:15.559612+0000","last_unstale":"2024-09-13T17:59:08.625238+0000","last_undegraded":"2024-09-13T17:59:08.625238+0000","last_fullsized":"2024-09-13T17:59:08.625238+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_clean_scrub_stamp":"2024-09-13T17:57:14.317378+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:16:27.663312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":825873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72499999999999998,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64000000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71199999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67500000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.47099999999999997}]},{"osd":2,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66600000000000004,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70199999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.64200000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55800000000000005,"15min":0.56100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.748,"5min":0.879,"15min":0.879},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72299999999999998,"5min":1.034,"15min":1.034},"last":0.61599999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.629,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.40200000000000002}]},{"osd":5,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69599999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55800000000000005,"15min":0.56399999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71799999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.59399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":220339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.5}]},{"osd":3,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.45300000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.63400000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.61699999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":219869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.56599999999999995}]},{"osd":1,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.48699999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.37},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.45700000000000002}]},{"osd":6,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.53700000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.5}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":220361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.83299999999999996}]},{"osd":1,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.84699999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.79000000000000004}]},{"osd":3,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.73899999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.50900000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.86299999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":235469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.29099999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.252}]},{"osd":2,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.31}]},{"osd":4,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60299999999999998}]},{"osd":6,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.59199999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.58099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":219827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.36099999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.313}]},{"osd":3,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.34699999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.621}]},{"osd":7,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.67100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":219847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.29699999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.316}]},{"osd":3,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.33100000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.63}]},{"osd":5,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.753},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54100000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.55700000000000005}]},{"osd":7,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":809140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.26500000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.29999999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.28799999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.443}]},{"osd":5,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48899999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.42999999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.501}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:59:18.052 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:59:18.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.148+0000 7fbbc735a640 1 Processor -- start 2024-09-13T17:59:18.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.148+0000 7fbbc735a640 1 -- start start 2024-09-13T17:59:18.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.149+0000 7fbbc735a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbbc0071d30 0x7fbbc0070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.149+0000 7fbbc735a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc00abdf0 con 0x7fbbc0072110 2024-09-13T17:59:18.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.149+0000 7fbbc735a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc00b2280 con 0x7fbbc00c8920 2024-09-13T17:59:18.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.149+0000 7fbbc735a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc00ab1e0 con 0x7fbbc0071d30 2024-09-13T17:59:18.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.149+0000 7fbbc50f6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbbc0071d30 0x7fbbc0070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.149+0000 7fbbc48f5640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbbc0072110 0x7fbbc00b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43060/0 (socket says 172.21.15.111:43060) 2024-09-13T17:59:18.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.149+0000 7fbbc48f5640 1 -- 172.21.15.111:0/4057879925 learned_addr learned my addr 172.21.15.111:0/4057879925 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:18.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.149+0000 7fbbc50f6640 1 -- 172.21.15.111:0/4057879925 >> v1:172.21.15.188:6789/0 conn(0x7fbbc00c8920 legacy=0x7fbbc00ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:18.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.149+0000 7fbbc50f6640 1 -- 172.21.15.111:0/4057879925 >> v1:172.21.15.111:6789/0 conn(0x7fbbc0072110 legacy=0x7fbbc00b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.149+0000 7fbbb7fff640 1 -- 172.21.15.111:0/4057879925 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2565751052 0 0) 0x7fbbc00abdf0 con 0x7fbbc0072110 2024-09-13T17:59:18.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.150+0000 7fbbc50f6640 1 -- 172.21.15.111:0/4057879925 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc00ab4c0 con 0x7fbbc0071d30 2024-09-13T17:59:18.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.150+0000 7fbbc50f6640 1 --2- 172.21.15.111:0/4057879925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbc0071d30 0x7fbbc0070190 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0097a0 tx=0x7fbbbc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2febfa649b588301 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:18.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.150+0000 7fbbb7fff640 1 -- 172.21.15.111:0/4057879925 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc032440 con 0x7fbbc0071d30 2024-09-13T17:59:18.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.150+0000 7fbbb7fff640 1 -- 172.21.15.111:0/4057879925 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbbc0325a0 con 0x7fbbc0071d30 2024-09-13T17:59:18.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.150+0000 7fbbb7fff640 1 -- 172.21.15.111:0/4057879925 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc032930 con 0x7fbbc0071d30 2024-09-13T17:59:18.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.151+0000 7fbbc735a640 1 -- 172.21.15.111:0/4057879925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbc0071d30 msgr2=0x7fbbc0070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.151+0000 7fbbc735a640 1 --2- 172.21.15.111:0/4057879925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbc0071d30 0x7fbbc0070190 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0097a0 tx=0x7fbbbc02a1e0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.151+0000 7fbbc735a640 1 -- 172.21.15.111:0/4057879925 shutdown_connections 2024-09-13T17:59:18.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.151+0000 7fbbc735a640 1 --2- 172.21.15.111:0/4057879925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbc0071d30 0x7fbbc0070190 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.151+0000 7fbbc735a640 1 -- 172.21.15.111:0/4057879925 >> 172.21.15.111:0/4057879925 conn(0x7fbbc0069ed0 msgr2=0x7fbbc006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:18.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.151+0000 7fbbc735a640 1 -- 172.21.15.111:0/4057879925 shutdown_connections 2024-09-13T17:59:18.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.151+0000 7fbbc735a640 1 -- 172.21.15.111:0/4057879925 wait complete. 2024-09-13T17:59:18.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.152+0000 7fbbc735a640 1 Processor -- start 2024-09-13T17:59:18.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.152+0000 7fbbc735a640 1 -- start start 2024-09-13T17:59:18.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.153+0000 7fbbc735a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbc0071d30 0x7fbbc00972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.153+0000 7fbbc735a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbc0072110 0x7fbbc00c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.153+0000 7fbbc735a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbc00c8920 0x7fbbc00c3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.153+0000 7fbbc735a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc0157cf0 con 0x7fbbc00c8920 2024-09-13T17:59:18.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.153+0000 7fbbc735a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00977e0 con 0x7fbbc0072110 2024-09-13T17:59:18.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.153+0000 7fbbc735a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00c86e0 con 0x7fbbc0071d30 2024-09-13T17:59:18.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.153+0000 7fbbc58f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbc00c8920 0x7fbbc00c3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.153+0000 7fbbc58f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbc00c8920 0x7fbbc00c3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55396/0 (socket says 172.21.15.111:55396) 2024-09-13T17:59:18.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.153+0000 7fbbc58f7640 1 -- 172.21.15.111:0/1341641135 learned_addr learned my addr 172.21.15.111:0/1341641135 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:18.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.153+0000 7fbbc58f7640 1 -- 172.21.15.111:0/1341641135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbc0071d30 msgr2=0x7fbbc00972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.153+0000 7fbbc58f7640 1 --2- 172.21.15.111:0/1341641135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbc0071d30 0x7fbbc00972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.153+0000 7fbbc58f7640 1 -- 172.21.15.111:0/1341641135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbc0072110 msgr2=0x7fbbc00c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.153+0000 7fbbc58f7640 1 --2- 172.21.15.111:0/1341641135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbc0072110 0x7fbbc00c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.153+0000 7fbbc58f7640 1 -- 172.21.15.111:0/1341641135 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbbc009400 con 0x7fbbc00c8920 2024-09-13T17:59:18.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.154+0000 7fbbc58f7640 1 --2- 172.21.15.111:0/1341641135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbc00c8920 0x7fbbc00c3980 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800c7b0 tx=0x7fbbb800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:18.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.154+0000 7fbbb5ffb640 1 -- 172.21.15.111:0/1341641135 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8004b60 con 0x7fbbc00c8920 2024-09-13T17:59:18.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.154+0000 7fbbb5ffb640 1 -- 172.21.15.111:0/1341641135 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb8004cc0 con 0x7fbbc00c8920 2024-09-13T17:59:18.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.154+0000 7fbbc735a640 1 -- 172.21.15.111:0/1341641135 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc00c6700 con 0x7fbbc00c8920 2024-09-13T17:59:18.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.154+0000 7fbbb5ffb640 1 -- 172.21.15.111:0/1341641135 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb800aa90 con 0x7fbbc00c8920 2024-09-13T17:59:18.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.154+0000 7fbbc735a640 1 -- 172.21.15.111:0/1341641135 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc0157f00 con 0x7fbbc00c8920 2024-09-13T17:59:18.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.155+0000 7fbbc735a640 1 -- 172.21.15.111:0/1341641135 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc00c36a0 con 0x7fbbc00c8920 2024-09-13T17:59:18.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.155+0000 7fbbb5ffb640 1 -- 172.21.15.111:0/1341641135 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbbb800abf0 con 0x7fbbc00c8920 2024-09-13T17:59:18.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.155+0000 7fbbb5ffb640 1 --2- 172.21.15.111:0/1341641135 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbba406ac00 0x7fbba406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.156+0000 7fbbb5ffb640 1 -- 172.21.15.111:0/1341641135 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fbbb8089370 con 0x7fbbc00c8920 2024-09-13T17:59:18.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.156+0000 7fbbc50f6640 1 --2- 172.21.15.111:0/1341641135 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbba406ac00 0x7fbba406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.156+0000 7fbbc50f6640 1 --2- 172.21.15.111:0/1341641135 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbba406ac00 0x7fbba406d0c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc034000 tx=0x7fbbbc02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:18.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.160+0000 7fbbb5ffb640 1 -- 172.21.15.111:0/1341641135 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fbbc00c36a0 con 0x7fbbc00c8920 2024-09-13T17:59:18.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.320+0000 7fbbc735a640 1 -- 172.21.15.111:0/1341641135 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbbc00aba80 con 0x7fbba406ac00 2024-09-13T17:59:18.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.322+0000 7fbbb5ffb640 1 -- 172.21.15.111:0/1341641135 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51143 (secure 0 0 0) 0x7fbbbc006e10 con 0x7fbba406ac00 2024-09-13T17:59:18.324 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:18.324 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-09-13T17:59:18.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.323+0000 7fbbc735a640 1 -- 172.21.15.111:0/1341641135 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbba406ac00 msgr2=0x7fbba406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.323+0000 7fbbc735a640 1 --2- 172.21.15.111:0/1341641135 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbba406ac00 0x7fbba406d0c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc034000 tx=0x7fbbbc02c000 comp rx=0 tx=0).stop 2024-09-13T17:59:18.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.323+0000 7fbbc735a640 1 -- 172.21.15.111:0/1341641135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbc00c8920 msgr2=0x7fbbc00c3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.323+0000 7fbbc735a640 1 --2- 172.21.15.111:0/1341641135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbc00c8920 0x7fbbc00c3980 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800c7b0 tx=0x7fbbb800cc70 comp rx=0 tx=0).stop 2024-09-13T17:59:18.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.324+0000 7fbbc735a640 1 -- 172.21.15.111:0/1341641135 shutdown_connections 2024-09-13T17:59:18.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.324+0000 7fbbc735a640 1 --2- 172.21.15.111:0/1341641135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbc00c8920 0x7fbbc00c3980 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.324+0000 7fbbc735a640 1 --2- 172.21.15.111:0/1341641135 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbba406ac00 0x7fbba406d0c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.324+0000 7fbbc735a640 1 --2- 172.21.15.111:0/1341641135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbc0072110 0x7fbbc00c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.324+0000 7fbbc735a640 1 --2- 172.21.15.111:0/1341641135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbc0071d30 0x7fbbc00972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.324+0000 7fbbc735a640 1 -- 172.21.15.111:0/1341641135 >> 172.21.15.111:0/1341641135 conn(0x7fbbc0069ed0 msgr2=0x7fbbc00adf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:18.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.324+0000 7fbbc735a640 1 -- 172.21.15.111:0/1341641135 shutdown_connections 2024-09-13T17:59:18.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.324+0000 7fbbc735a640 1 -- 172.21.15.111:0/1341641135 wait complete. 2024-09-13T17:59:18.337 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-13T17:59:18.289277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221308,"kb_used_data":5148,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753412,"statfs":{"total":773094113280,"available":772867493888,"internally_reserved":0,"allocated":5271552,"data_stored":2970725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14667,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-86,"num_read_kb":-88,"num_write":-74,"num_write_kb":-39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001105"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":187,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:59:08.625238+0000","last_change":"2024-09-13T17:57:15.560506+0000","last_active":"2024-09-13T17:59:08.625238+0000","last_peered":"2024-09-13T17:59:08.625238+0000","last_clean":"2024-09-13T17:59:08.625238+0000","last_became_active":"2024-09-13T17:57:15.559612+0000","last_became_peered":"2024-09-13T17:57:15.559612+0000","last_unstale":"2024-09-13T17:59:08.625238+0000","last_undegraded":"2024-09-13T17:59:08.625238+0000","last_fullsized":"2024-09-13T17:59:08.625238+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_clean_scrub_stamp":"2024-09-13T17:57:14.317378+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:16:27.663312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":825873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72499999999999998,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64000000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71199999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67500000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.47099999999999997}]},{"osd":2,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66600000000000004,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70199999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.64200000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55800000000000005,"15min":0.56100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.748,"5min":0.879,"15min":0.879},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72299999999999998,"5min":1.034,"15min":1.034},"last":0.61599999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.629,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.40200000000000002}]},{"osd":5,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69599999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55800000000000005,"15min":0.56399999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71799999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.59399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":220339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.5}]},{"osd":3,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.45300000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.63400000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.61699999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":219869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.56599999999999995}]},{"osd":1,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.48699999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.37},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.45700000000000002}]},{"osd":6,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.53700000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.5}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":220361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.83299999999999996}]},{"osd":1,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.84699999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.79000000000000004}]},{"osd":3,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.73899999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.50900000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.86299999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":235469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.29099999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.252}]},{"osd":2,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.31}]},{"osd":4,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60299999999999998}]},{"osd":6,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.59199999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.58099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":219827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.36099999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.313}]},{"osd":3,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.34699999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.621}]},{"osd":7,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.67100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":219847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.29699999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.316}]},{"osd":3,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.33100000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.63}]},{"osd":5,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.753},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54100000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.55700000000000005}]},{"osd":7,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":809140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.26500000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.29999999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.28799999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.443}]},{"osd":5,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48899999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.42999999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.501}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:59:18.339 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T17:59:18.340 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:59:18.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.435+0000 7f1910101640 1 Processor -- start 2024-09-13T17:59:18.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.436+0000 7f1910101640 1 -- start start 2024-09-13T17:59:18.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.436+0000 7f1910101640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f190806c4c0 0x7f190806c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.436+0000 7f1910101640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f19080b2590 0x7f190806d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.436+0000 7f1910101640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190806e0c0 con 0x7f19080cbea0 2024-09-13T17:59:18.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.436+0000 7f1910101640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f190806deb0 con 0x7f19080b2590 2024-09-13T17:59:18.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.436+0000 7f1910101640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19080b2170 con 0x7f190806c4c0 2024-09-13T17:59:18.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190e69e640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f19080cbea0 0x7f19080cc350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43070/0 (socket says 172.21.15.111:43070) 2024-09-13T17:59:18.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190e69e640 1 -- 172.21.15.111:0/3984584127 learned_addr learned my addr 172.21.15.111:0/3984584127 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:18.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190d69c640 1 --2- 172.21.15.111:0/3984584127 >> v2:172.21.15.188:3300/0 conn(0x7f19080b2590 0x7f190806d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190ce9b640 1 -- 172.21.15.111:0/3984584127 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695532730 0 0) 0x7f190806e0c0 con 0x7f19080cbea0 2024-09-13T17:59:18.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190de9d640 1 --2- 172.21.15.111:0/3984584127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f190806c4c0 0x7f190806c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190ce9b640 1 -- 172.21.15.111:0/3984584127 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18f4003520 con 0x7f19080cbea0 2024-09-13T17:59:18.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190ce9b640 1 -- 172.21.15.111:0/3984584127 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1213770900 0 0) 0x7f18f4003520 con 0x7f19080cbea0 2024-09-13T17:59:18.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190ce9b640 1 -- 172.21.15.111:0/3984584127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f190806c4c0 msgr2=0x7f190806c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:18.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190ce9b640 1 --2- 172.21.15.111:0/3984584127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f190806c4c0 0x7f190806c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190ce9b640 1 -- 172.21.15.111:0/3984584127 >> v2:172.21.15.188:3300/0 conn(0x7f19080b2590 msgr2=0x7f190806d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:18.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190ce9b640 1 --2- 172.21.15.111:0/3984584127 >> v2:172.21.15.188:3300/0 conn(0x7f19080b2590 0x7f190806d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190ce9b640 1 -- 172.21.15.111:0/3984584127 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190806e3a0 con 0x7f19080cbea0 2024-09-13T17:59:18.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190ce9b640 1 -- 172.21.15.111:0/3984584127 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f19000019f0 con 0x7f19080cbea0 2024-09-13T17:59:18.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190ce9b640 1 -- 172.21.15.111:0/3984584127 >> v1:172.21.15.111:6789/0 conn(0x7f19080cbea0 legacy=0x7f19080cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190ce9b640 1 --2- 172.21.15.111:0/3984584127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18f4003970 0x7f18f4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190ce9b640 1 --2- 172.21.15.111:0/3984584127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f190806c4c0 0x7f18f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190ce9b640 1 --2- 172.21.15.111:0/3984584127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18f4004d00 0x7f18f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190ce9b640 1 -- 172.21.15.111:0/3984584127 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190806deb0 con 0x7f18f4004d00 2024-09-13T17:59:18.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190ce9b640 1 -- 172.21.15.111:0/3984584127 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19080b2170 con 0x7f18f4003970 2024-09-13T17:59:18.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190ce9b640 1 -- 172.21.15.111:0/3984584127 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f400b980 con 0x7f190806c4c0 2024-09-13T17:59:18.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190e69e640 1 --2- 172.21.15.111:0/3984584127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18f4004d00 0x7f18f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190de9d640 1 --2- 172.21.15.111:0/3984584127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18f4003970 0x7f18f4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.437+0000 7f190d69c640 1 --2- 172.21.15.111:0/3984584127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f190806c4c0 0x7f18f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.438+0000 7f190e69e640 1 -- 172.21.15.111:0/3984584127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f190806c4c0 msgr2=0x7f18f40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:18.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.438+0000 7f190e69e640 1 --2- 172.21.15.111:0/3984584127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f190806c4c0 0x7f18f40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.438+0000 7f190e69e640 1 -- 172.21.15.111:0/3984584127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18f4003970 msgr2=0x7f18f4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:18.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.438+0000 7f190e69e640 1 --2- 172.21.15.111:0/3984584127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18f4003970 0x7f18f4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.438+0000 7f190e69e640 1 -- 172.21.15.111:0/3984584127 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f4003520 con 0x7f18f4004d00 2024-09-13T17:59:18.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.438+0000 7f190e69e640 1 --2- 172.21.15.111:0/3984584127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18f4004d00 0x7f18f4005180 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f190000a830 tx=0x7f18f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27434e51ca8903a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:18.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.438+0000 7f190ce9b640 1 -- 172.21.15.111:0/3984584127 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190002ba50 con 0x7f18f4004d00 2024-09-13T17:59:18.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.438+0000 7f190ce9b640 1 -- 172.21.15.111:0/3984584127 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190002bbd0 con 0x7f18f4004d00 2024-09-13T17:59:18.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.438+0000 7f190ce9b640 1 -- 172.21.15.111:0/3984584127 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1900034470 con 0x7f18f4004d00 2024-09-13T17:59:18.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.439+0000 7f1910101640 1 -- 172.21.15.111:0/3984584127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18f4004d00 msgr2=0x7f18f4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.439+0000 7f1910101640 1 --2- 172.21.15.111:0/3984584127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18f4004d00 0x7f18f4005180 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f190000a830 tx=0x7f18f40057f0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.439+0000 7f190e69e640 1 -- 172.21.15.111:0/3984584127 reap_dead start 2024-09-13T17:59:18.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.439+0000 7f1910101640 1 -- 172.21.15.111:0/3984584127 shutdown_connections 2024-09-13T17:59:18.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.439+0000 7f1910101640 1 -- 172.21.15.111:0/3984584127 >> 172.21.15.111:0/3984584127 conn(0x7f1908069ed0 msgr2=0x7f190806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:18.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.439+0000 7f1910101640 1 -- 172.21.15.111:0/3984584127 shutdown_connections 2024-09-13T17:59:18.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.439+0000 7f1910101640 1 -- 172.21.15.111:0/3984584127 wait complete. 2024-09-13T17:59:18.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.440+0000 7f1910101640 1 Processor -- start 2024-09-13T17:59:18.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.440+0000 7f1910101640 1 -- start start 2024-09-13T17:59:18.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f1910101640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18f4004d00 0x7f19080c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f1910101640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19080c6530 0x7f19080c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f1910101640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1908098ef0 0x7f1908099370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f1910101640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19080998b0 con 0x7f1908098ef0 2024-09-13T17:59:18.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f1910101640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19080c54d0 con 0x7f18f4004d00 2024-09-13T17:59:18.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f1910101640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19080c5770 con 0x7f19080c6530 2024-09-13T17:59:18.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f190e69e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1908098ef0 0x7f1908099370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f190e69e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1908098ef0 0x7f1908099370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55414/0 (socket says 172.21.15.111:55414) 2024-09-13T17:59:18.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f190e69e640 1 -- 172.21.15.111:0/3877953773 learned_addr learned my addr 172.21.15.111:0/3877953773 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:18.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f190d69c640 1 --2- 172.21.15.111:0/3877953773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19080c6530 0x7f19080c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f190de9d640 1 --2- 172.21.15.111:0/3877953773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18f4004d00 0x7f19080c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f190e69e640 1 -- 172.21.15.111:0/3877953773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19080c6530 msgr2=0x7f19080c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f190e69e640 1 --2- 172.21.15.111:0/3877953773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19080c6530 0x7f19080c6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f190e69e640 1 -- 172.21.15.111:0/3877953773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18f4004d00 msgr2=0x7f19080c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f190e69e640 1 --2- 172.21.15.111:0/3877953773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18f4004d00 0x7f19080c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f190e69e640 1 -- 172.21.15.111:0/3877953773 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19080c5a50 con 0x7f1908098ef0 2024-09-13T17:59:18.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f190de9d640 1 --2- 172.21.15.111:0/3877953773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18f4004d00 0x7f19080c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:18.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.441+0000 7f190e69e640 1 --2- 172.21.15.111:0/3877953773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1908098ef0 0x7f1908099370 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1900001cb0 tx=0x7f190002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:18.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.442+0000 7f18fa7fc640 1 -- 172.21.15.111:0/3877953773 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190003b040 con 0x7f1908098ef0 2024-09-13T17:59:18.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.442+0000 7f18fa7fc640 1 -- 172.21.15.111:0/3877953773 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19000019e0 con 0x7f1908098ef0 2024-09-13T17:59:18.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.442+0000 7f1910101640 1 -- 172.21.15.111:0/3877953773 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19080c5d30 con 0x7f1908098ef0 2024-09-13T17:59:18.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.442+0000 7f18fa7fc640 1 -- 172.21.15.111:0/3877953773 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1900034a80 con 0x7f1908098ef0 2024-09-13T17:59:18.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.442+0000 7f1910101640 1 -- 172.21.15.111:0/3877953773 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19080c80d0 con 0x7f1908098ef0 2024-09-13T17:59:18.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.443+0000 7f1910101640 1 -- 172.21.15.111:0/3877953773 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19080c6250 con 0x7f1908098ef0 2024-09-13T17:59:18.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.443+0000 7f18fa7fc640 1 -- 172.21.15.111:0/3877953773 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1900031070 con 0x7f1908098ef0 2024-09-13T17:59:18.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.443+0000 7f18fa7fc640 1 --2- 172.21.15.111:0/3877953773 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f18ec06abc0 0x7f18ec06d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.444+0000 7f18fa7fc640 1 -- 172.21.15.111:0/3877953773 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f19000a9350 con 0x7f1908098ef0 2024-09-13T17:59:18.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.444+0000 7f190de9d640 1 --2- 172.21.15.111:0/3877953773 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f18ec06abc0 0x7f18ec06d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.444+0000 7f190de9d640 1 --2- 172.21.15.111:0/3877953773 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f18ec06abc0 0x7f18ec06d080 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1904009b40 tx=0x7f1904009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:18.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.448+0000 7f18fa7fc640 1 -- 172.21.15.111:0/3877953773 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f19080c6250 con 0x7f1908098ef0 2024-09-13T17:59:18.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.607+0000 7f1910101640 1 -- 172.21.15.111:0/3877953773 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f19080b29b0 con 0x7f18ec06abc0 2024-09-13T17:59:18.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.609+0000 7f18fa7fc640 1 -- 172.21.15.111:0/3877953773 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51143 (secure 0 0 0) 0x7f1904013a80 con 0x7f18ec06abc0 2024-09-13T17:59:18.612 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:18.612 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-09-13T17:59:18.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.612+0000 7f1910101640 1 -- 172.21.15.111:0/3877953773 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f18ec06abc0 msgr2=0x7f18ec06d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.612+0000 7f1910101640 1 --2- 172.21.15.111:0/3877953773 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f18ec06abc0 0x7f18ec06d080 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1904009b40 tx=0x7f1904009190 comp rx=0 tx=0).stop 2024-09-13T17:59:18.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.612+0000 7f1910101640 1 -- 172.21.15.111:0/3877953773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1908098ef0 msgr2=0x7f1908099370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.612+0000 7f1910101640 1 --2- 172.21.15.111:0/3877953773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1908098ef0 0x7f1908099370 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1900001cb0 tx=0x7f190002d070 comp rx=0 tx=0).stop 2024-09-13T17:59:18.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.612+0000 7f1910101640 1 -- 172.21.15.111:0/3877953773 shutdown_connections 2024-09-13T17:59:18.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.612+0000 7f1910101640 1 --2- 172.21.15.111:0/3877953773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1908098ef0 0x7f1908099370 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.612+0000 7f1910101640 1 --2- 172.21.15.111:0/3877953773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19080c6530 0x7f19080c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.612+0000 7f1910101640 1 --2- 172.21.15.111:0/3877953773 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f18ec06abc0 0x7f18ec06d080 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.612+0000 7f1910101640 1 --2- 172.21.15.111:0/3877953773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18f4004d00 0x7f19080c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.612+0000 7f1910101640 1 -- 172.21.15.111:0/3877953773 >> 172.21.15.111:0/3877953773 conn(0x7f1908069ed0 msgr2=0x7f19080af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:18.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.612+0000 7f1910101640 1 -- 172.21.15.111:0/3877953773 shutdown_connections 2024-09-13T17:59:18.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.612+0000 7f1910101640 1 -- 172.21.15.111:0/3877953773 wait complete. 2024-09-13T17:59:18.624 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-13T17:59:18.289277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221308,"kb_used_data":5148,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753412,"statfs":{"total":773094113280,"available":772867493888,"internally_reserved":0,"allocated":5271552,"data_stored":2970725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14667,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-86,"num_read_kb":-88,"num_write":-74,"num_write_kb":-39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001105"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":187,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:59:08.625238+0000","last_change":"2024-09-13T17:57:15.560506+0000","last_active":"2024-09-13T17:59:08.625238+0000","last_peered":"2024-09-13T17:59:08.625238+0000","last_clean":"2024-09-13T17:59:08.625238+0000","last_became_active":"2024-09-13T17:57:15.559612+0000","last_became_peered":"2024-09-13T17:57:15.559612+0000","last_unstale":"2024-09-13T17:59:08.625238+0000","last_undegraded":"2024-09-13T17:59:08.625238+0000","last_fullsized":"2024-09-13T17:59:08.625238+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_clean_scrub_stamp":"2024-09-13T17:57:14.317378+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:16:27.663312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":825873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72499999999999998,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64000000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71199999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67500000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.47099999999999997}]},{"osd":2,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66600000000000004,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70199999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.64200000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55800000000000005,"15min":0.56100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.748,"5min":0.879,"15min":0.879},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72299999999999998,"5min":1.034,"15min":1.034},"last":0.61599999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.629,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.40200000000000002}]},{"osd":5,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69599999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55800000000000005,"15min":0.56399999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71799999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.59399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":220339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.5}]},{"osd":3,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.45300000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.63400000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.61699999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":219869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.56599999999999995}]},{"osd":1,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.48699999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.37},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.45700000000000002}]},{"osd":6,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.53700000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.5}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":220361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.83299999999999996}]},{"osd":1,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.84699999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.79000000000000004}]},{"osd":3,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.73899999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.50900000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Sep 13 17:58:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.86299999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":235469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.29099999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.252}]},{"osd":2,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.31}]},{"osd":4,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60299999999999998}]},{"osd":6,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.59199999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.58099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":219827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.36099999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.313}]},{"osd":3,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.34699999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.621}]},{"osd":7,"last update":"Fri Sep 13 17:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.67100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":219847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.29699999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.316}]},{"osd":3,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.33100000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.63}]},{"osd":5,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.753},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54100000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.55700000000000005}]},{"osd":7,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":809140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.26500000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.29999999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.28799999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.443}]},{"osd":5,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48899999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.42999999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.501}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:59:18.626 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:59:18.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.722+0000 7ffac93ff640 1 Processor -- start 2024-09-13T17:59:18.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.723+0000 7ffac93ff640 1 -- start start 2024-09-13T17:59:18.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.723+0000 7ffac93ff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ffac40d0980 0x7ffac40d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.723+0000 7ffac93ff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffac40d2ab0 0x7ffac40cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.723+0000 7ffac93ff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac415d6e0 0x7ffac4160050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.723+0000 7ffac93ff640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac40cf810 con 0x7ffac40d0980 2024-09-13T17:59:18.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.723+0000 7ffac93ff640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac40d3270 con 0x7ffac40d2ab0 2024-09-13T17:59:18.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.723+0000 7ffac93ff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac40d12a0 con 0x7ffac415d6e0 2024-09-13T17:59:18.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.723+0000 7ffac2ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ffac40d0980 0x7ffac40d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.723+0000 7ffac2ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ffac40d0980 0x7ffac40d0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55428/0 (socket says 172.21.15.111:55428) 2024-09-13T17:59:18.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.723+0000 7ffac2ffd640 1 -- 172.21.15.111:0/4079461453 learned_addr learned my addr 172.21.15.111:0/4079461453 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:18.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.724+0000 7ffac2ffd640 1 -- 172.21.15.111:0/4079461453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac415d6e0 msgr2=0x7ffac4160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:18.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.724+0000 7ffac2ffd640 1 --2- 172.21.15.111:0/4079461453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac415d6e0 0x7ffac4160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.724+0000 7ffac2ffd640 1 -- 172.21.15.111:0/4079461453 >> v2:172.21.15.188:3300/0 conn(0x7ffac40d2ab0 msgr2=0x7ffac40cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:18.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.724+0000 7ffac2ffd640 1 --2- 172.21.15.111:0/4079461453 >> v2:172.21.15.188:3300/0 conn(0x7ffac40d2ab0 0x7ffac40cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.724+0000 7ffac2ffd640 1 -- 172.21.15.111:0/4079461453 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac40d2f90 con 0x7ffac40d0980 2024-09-13T17:59:18.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.724+0000 7ffac2ffd640 1 --2- 172.21.15.111:0/4079461453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40d0980 0x7ffac40d0d60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ffac40cc8f0 tx=0x7ffab402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46cdc65fc3ff8554 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:18.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.725+0000 7ffac1ffb640 1 -- 172.21.15.111:0/4079461453 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4032440 con 0x7ffac40d0980 2024-09-13T17:59:18.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.725+0000 7ffac1ffb640 1 -- 172.21.15.111:0/4079461453 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab40325a0 con 0x7ffac40d0980 2024-09-13T17:59:18.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.725+0000 7ffac1ffb640 1 -- 172.21.15.111:0/4079461453 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4032930 con 0x7ffac40d0980 2024-09-13T17:59:18.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.725+0000 7ffac93ff640 1 -- 172.21.15.111:0/4079461453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40d0980 msgr2=0x7ffac40d0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.725+0000 7ffac93ff640 1 --2- 172.21.15.111:0/4079461453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40d0980 0x7ffac40d0d60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ffac40cc8f0 tx=0x7ffab402a1e0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.725+0000 7ffac93ff640 1 -- 172.21.15.111:0/4079461453 shutdown_connections 2024-09-13T17:59:18.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.725+0000 7ffac93ff640 1 --2- 172.21.15.111:0/4079461453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac415d6e0 0x7ffac4160050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.725+0000 7ffac93ff640 1 --2- 172.21.15.111:0/4079461453 >> v2:172.21.15.188:3300/0 conn(0x7ffac40d2ab0 0x7ffac40cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.725+0000 7ffac93ff640 1 --2- 172.21.15.111:0/4079461453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40d0980 0x7ffac40d0d60 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.725+0000 7ffac93ff640 1 -- 172.21.15.111:0/4079461453 >> 172.21.15.111:0/4079461453 conn(0x7ffac4089f60 msgr2=0x7ffac40d23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:18.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.725+0000 7ffac93ff640 1 -- 172.21.15.111:0/4079461453 shutdown_connections 2024-09-13T17:59:18.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.725+0000 7ffac93ff640 1 -- 172.21.15.111:0/4079461453 wait complete. 2024-09-13T17:59:18.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.726+0000 7ffac93ff640 1 Processor -- start 2024-09-13T17:59:18.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.727+0000 7ffac93ff640 1 -- start start 2024-09-13T17:59:18.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.727+0000 7ffac93ff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40d0980 0x7ffac40c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.727+0000 7ffac93ff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac40d2ab0 0x7ffac4150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.727+0000 7ffac93ff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac415d6e0 0x7ffac40ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.727+0000 7ffac93ff640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac40c7330 con 0x7ffac40d0980 2024-09-13T17:59:18.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.727+0000 7ffac93ff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac40c6720 con 0x7ffac415d6e0 2024-09-13T17:59:18.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.727+0000 7ffac93ff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac40c6a00 con 0x7ffac40d2ab0 2024-09-13T17:59:18.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.727+0000 7ffac2ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40d0980 0x7ffac40c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.727+0000 7ffac2ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40d0980 0x7ffac40c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55438/0 (socket says 172.21.15.111:55438) 2024-09-13T17:59:18.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.727+0000 7ffac2ffd640 1 -- 172.21.15.111:0/568277548 learned_addr learned my addr 172.21.15.111:0/568277548 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:18.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.728+0000 7ffac27fc640 1 --2- 172.21.15.111:0/568277548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac40d2ab0 0x7ffac4150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.728+0000 7ffac37fe640 1 --2- 172.21.15.111:0/568277548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac415d6e0 0x7ffac40ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.728+0000 7ffac2ffd640 1 -- 172.21.15.111:0/568277548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac40d2ab0 msgr2=0x7ffac4150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.728+0000 7ffac2ffd640 1 --2- 172.21.15.111:0/568277548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac40d2ab0 0x7ffac4150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.728+0000 7ffac2ffd640 1 -- 172.21.15.111:0/568277548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac415d6e0 msgr2=0x7ffac40ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.728+0000 7ffac2ffd640 1 --2- 172.21.15.111:0/568277548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac415d6e0 0x7ffac40ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.728+0000 7ffac2ffd640 1 -- 172.21.15.111:0/568277548 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab40094d0 con 0x7ffac40d0980 2024-09-13T17:59:18.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.728+0000 7ffac27fc640 1 --2- 172.21.15.111:0/568277548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac40d2ab0 0x7ffac4150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:59:18.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.728+0000 7ffac37fe640 1 --2- 172.21.15.111:0/568277548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac415d6e0 0x7ffac40ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:59:18.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.728+0000 7ffac2ffd640 1 --2- 172.21.15.111:0/568277548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40d0980 0x7ffac40c5ed0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ffab4000c00 tx=0x7ffab40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:18.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.728+0000 7ffabb7fe640 1 -- 172.21.15.111:0/568277548 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4032440 con 0x7ffac40d0980 2024-09-13T17:59:18.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.728+0000 7ffabb7fe640 1 -- 172.21.15.111:0/568277548 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab4030b80 con 0x7ffac40d0980 2024-09-13T17:59:18.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.728+0000 7ffabb7fe640 1 -- 172.21.15.111:0/568277548 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab403b400 con 0x7ffac40d0980 2024-09-13T17:59:18.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.728+0000 7ffac93ff640 1 -- 172.21.15.111:0/568277548 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac40c6ce0 con 0x7ffac40d0980 2024-09-13T17:59:18.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.728+0000 7ffac93ff640 1 -- 172.21.15.111:0/568277548 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac40c6410 con 0x7ffac40d0980 2024-09-13T17:59:18.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.729+0000 7ffac93ff640 1 -- 172.21.15.111:0/568277548 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac40c6b40 con 0x7ffac40d0980 2024-09-13T17:59:18.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.729+0000 7ffabb7fe640 1 -- 172.21.15.111:0/568277548 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ffab4039040 con 0x7ffac40d0980 2024-09-13T17:59:18.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.730+0000 7ffabb7fe640 1 --2- 172.21.15.111:0/568277548 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffaa406ac00 0x7ffaa406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.730+0000 7ffabb7fe640 1 -- 172.21.15.111:0/568277548 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7ffab40a9b10 con 0x7ffac40d0980 2024-09-13T17:59:18.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.730+0000 7ffac27fc640 1 --2- 172.21.15.111:0/568277548 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffaa406ac00 0x7ffaa406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.730+0000 7ffac27fc640 1 --2- 172.21.15.111:0/568277548 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffaa406ac00 0x7ffaa406d0c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffaac006fd0 tx=0x7ffaac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:18.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.734+0000 7ffabb7fe640 1 -- 172.21.15.111:0/568277548 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ffac40c6b40 con 0x7ffac40d0980 2024-09-13T17:59:18.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.896+0000 7ffac93ff640 1 -- 172.21.15.111:0/568277548 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffac40c6b40 con 0x7ffac40d0980 2024-09-13T17:59:18.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.897+0000 7ffabb7fe640 1 -- 172.21.15.111:0/568277548 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+13381 (secure 0 0 0) 0x7ffac40c6b40 con 0x7ffac40d0980 2024-09-13T17:59:18.899 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:18.899 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":45,"fsid":"2a8c13ef-d427-4b48-90b3-65459ec79a47","created":"2024-09-13T17:57:02.267632+0000","modified":"2024-09-13T17:59:16.046311+0000","last_up_change":"2024-09-13T17:57:13.315807+0000","last_in_change":"2024-09-13T17:57:04.324664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:57:14.294908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"20368658-282f-4098-bfcc-4c95d0bf0def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6816","nonce":4024408960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6819","nonce":4024408960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6823","nonce":4024408960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4024408960},{"type":"v1","addr":"172.21.15.111:6821","nonce":4024408960}]},"public_addr":"172.21.15.111:6816/4024408960","cluster_addr":"172.21.15.111:6819/4024408960","heartbeat_back_addr":"172.21.15.111:6823/4024408960","heartbeat_front_addr":"172.21.15.111:6821/4024408960","state":["exists","up"]},{"osd":1,"uuid":"46c9d9c2-beda-4b6d-bdf4-62b24fe98259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6809","nonce":2899699337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6811","nonce":2899699337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6815","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6817","nonce":2899699337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2899699337},{"type":"v1","addr":"172.21.15.111:6813","nonce":2899699337}]},"public_addr":"172.21.15.111:6809/2899699337","cluster_addr":"172.21.15.111:6811/2899699337","heartbeat_back_addr":"172.21.15.111:6817/2899699337","heartbeat_front_addr":"172.21.15.111:6813/2899699337","state":["exists","up"]},{"osd":2,"uuid":"5a3f2c5b-9f7c-4c5f-8f9f-36d7b4521279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6801","nonce":2426719794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6803","nonce":2426719794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6807","nonce":2426719794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2426719794},{"type":"v1","addr":"172.21.15.111:6805","nonce":2426719794}]},"public_addr":"172.21.15.111:6801/2426719794","cluster_addr":"172.21.15.111:6803/2426719794","heartbeat_back_addr":"172.21.15.111:6807/2426719794","heartbeat_front_addr":"172.21.15.111:6805/2426719794","state":["exists","up"]},{"osd":3,"uuid":"9bc539c8-40b1-4070-9967-891d80f266a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6825","nonce":3696423188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6827","nonce":3696423188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6831","nonce":3696423188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3696423188},{"type":"v1","addr":"172.21.15.111:6829","nonce":3696423188}]},"public_addr":"172.21.15.111:6825/3696423188","cluster_addr":"172.21.15.111:6827/3696423188","heartbeat_back_addr":"172.21.15.111:6831/3696423188","heartbeat_front_addr":"172.21.15.111:6829/3696423188","state":["exists","up"]},{"osd":4,"uuid":"a7264062-0f38-44a0-acc9-796dfac73619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6817","nonce":1492113133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6819","nonce":1492113133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6823","nonce":1492113133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1492113133},{"type":"v1","addr":"172.21.15.188:6821","nonce":1492113133}]},"public_addr":"172.21.15.188:6817/1492113133","cluster_addr":"172.21.15.188:6819/1492113133","heartbeat_back_addr":"172.21.15.188:6823/1492113133","heartbeat_front_addr":"172.21.15.188:6821/1492113133","state":["exists","up"]},{"osd":5,"uuid":"98e5b234-4ed0-47c0-a6cc-1d63574e67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6809","nonce":3407929299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6811","nonce":3407929299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6815","nonce":3407929299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3407929299},{"type":"v1","addr":"172.21.15.188:6813","nonce":3407929299}]},"public_addr":"172.21.15.188:6809/3407929299","cluster_addr":"172.21.15.188:6811/3407929299","heartbeat_back_addr":"172.21.15.188:6815/3407929299","heartbeat_front_addr":"172.21.15.188:6813/3407929299","state":["exists","up"]},{"osd":6,"uuid":"efddf674-a908-4929-a8b3-0f0de44358ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6801","nonce":1593506388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6803","nonce":1593506388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6807","nonce":1593506388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1593506388},{"type":"v1","addr":"172.21.15.188:6805","nonce":1593506388}]},"public_addr":"172.21.15.188:6801/1593506388","cluster_addr":"172.21.15.188:6803/1593506388","heartbeat_back_addr":"172.21.15.188:6807/1593506388","heartbeat_front_addr":"172.21.15.188:6805/1593506388","state":["exists","up"]},{"osd":7,"uuid":"82233e9f-d2fb-438b-8800-785cb1f3c44a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6825","nonce":2518120865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6827","nonce":2518120865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6831","nonce":2518120865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2518120865},{"type":"v1","addr":"172.21.15.188:6829","nonce":2518120865}]},"public_addr":"172.21.15.188:6825/2518120865","cluster_addr":"172.21.15.188:6827/2518120865","heartbeat_back_addr":"172.21.15.188:6831/2518120865","heartbeat_front_addr":"172.21.15.188:6829/2518120865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.528045+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.606038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.468517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.494347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.495554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.491665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.469866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:57:09.496893+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6835/214034224":"2024-09-14T17:59:13.034441+0000","172.21.15.111:6834/214034224":"2024-09-14T17:59:13.034441+0000","172.21.15.111:6837/982359636":"2024-09-14T17:59:01.542287+0000","172.21.15.111:6836/982359636":"2024-09-14T17:59:01.542287+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:59:18.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.899+0000 7ffac93ff640 1 -- 172.21.15.111:0/568277548 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffaa406ac00 msgr2=0x7ffaa406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.899+0000 7ffac93ff640 1 --2- 172.21.15.111:0/568277548 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffaa406ac00 0x7ffaa406d0c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffaac006fd0 tx=0x7ffaac008040 comp rx=0 tx=0).stop 2024-09-13T17:59:18.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.899+0000 7ffac93ff640 1 -- 172.21.15.111:0/568277548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40d0980 msgr2=0x7ffac40c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.899+0000 7ffac93ff640 1 --2- 172.21.15.111:0/568277548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40d0980 0x7ffac40c5ed0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ffab4000c00 tx=0x7ffab40046c0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.899+0000 7ffac93ff640 1 -- 172.21.15.111:0/568277548 shutdown_connections 2024-09-13T17:59:18.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.899+0000 7ffac93ff640 1 --2- 172.21.15.111:0/568277548 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ffaa406ac00 0x7ffaa406d0c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.899+0000 7ffac93ff640 1 --2- 172.21.15.111:0/568277548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac415d6e0 0x7ffac40ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.899+0000 7ffac93ff640 1 --2- 172.21.15.111:0/568277548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac40d2ab0 0x7ffac4150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.899+0000 7ffac93ff640 1 --2- 172.21.15.111:0/568277548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffac40d0980 0x7ffac40c5ed0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.899+0000 7ffac93ff640 1 -- 172.21.15.111:0/568277548 >> 172.21.15.111:0/568277548 conn(0x7ffac4089f60 msgr2=0x7ffac40cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:18.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.899+0000 7ffac93ff640 1 -- 172.21.15.111:0/568277548 shutdown_connections 2024-09-13T17:59:18.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:18.899+0000 7ffac93ff640 1 -- 172.21.15.111:0/568277548 wait complete. 2024-09-13T17:59:19.912 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T17:59:19.913 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:59:20.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.028+0000 7fa241f1f640 1 Processor -- start 2024-09-13T17:59:20.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.028+0000 7fa241f1f640 1 -- start start 2024-09-13T17:59:20.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.028+0000 7fa241f1f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa23c0b1370 0x7fa23c0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.028+0000 7fa241f1f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa23c0d1650 0x7fa23c0d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.028+0000 7fa241f1f640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa23c0d1f40 con 0x7fa23c0cd500 2024-09-13T17:59:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.028+0000 7fa241f1f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa23c0d2190 con 0x7fa23c0d1650 2024-09-13T17:59:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.028+0000 7fa241f1f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c0ccb20 con 0x7fa23c0b1370 2024-09-13T17:59:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.029+0000 7fa23b7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa23c0cd500 0x7fa23c0b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43084/0 (socket says 172.21.15.111:43084) 2024-09-13T17:59:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.029+0000 7fa23b7fe640 1 -- 172.21.15.111:0/3619250904 learned_addr learned my addr 172.21.15.111:0/3619250904 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.029+0000 7fa23affd640 1 --2- 172.21.15.111:0/3619250904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa23c0b1370 0x7fa23c0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.029+0000 7fa23bfff640 1 --2- 172.21.15.111:0/3619250904 >> v2:172.21.15.188:3300/0 conn(0x7fa23c0d1650 0x7fa23c0d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.029+0000 7fa23a7fc640 1 -- 172.21.15.111:0/3619250904 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1856465250 0 0) 0x7fa23c0d1f40 con 0x7fa23c0cd500 2024-09-13T17:59:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.029+0000 7fa23a7fc640 1 -- 172.21.15.111:0/3619250904 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa224003520 con 0x7fa23c0cd500 2024-09-13T17:59:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.029+0000 7fa23bfff640 1 -- 172.21.15.111:0/3619250904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa23c0b1370 msgr2=0x7fa23c0b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.029+0000 7fa23bfff640 1 --2- 172.21.15.111:0/3619250904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa23c0b1370 0x7fa23c0b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.029+0000 7fa23bfff640 1 -- 172.21.15.111:0/3619250904 >> v1:172.21.15.111:6789/0 conn(0x7fa23c0cd500 legacy=0x7fa23c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.029+0000 7fa23a7fc640 1 -- 172.21.15.111:0/3619250904 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 259026239 0 0) 0x7fa224003520 con 0x7fa23c0cd500 2024-09-13T17:59:20.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.029+0000 7fa23bfff640 1 -- 172.21.15.111:0/3619250904 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c0cce00 con 0x7fa23c0d1650 2024-09-13T17:59:20.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.029+0000 7fa23affd640 1 --2- 172.21.15.111:0/3619250904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa23c0b1370 0x7fa23c0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:59:20.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.030+0000 7fa23bfff640 1 --2- 172.21.15.111:0/3619250904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa23c0d1650 0x7fa23c0d2f50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa234009dc0 tx=0x7fa23402a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a133e7c028b38e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.030+0000 7fa23a7fc640 1 -- 172.21.15.111:0/3619250904 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c001670 con 0x7fa23c0d1650 2024-09-13T17:59:20.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.030+0000 7fa23a7fc640 1 -- 172.21.15.111:0/3619250904 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23402d050 con 0x7fa23c0d1650 2024-09-13T17:59:20.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.030+0000 7fa23a7fc640 1 -- 172.21.15.111:0/3619250904 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa234007760 con 0x7fa23c0d1650 2024-09-13T17:59:20.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.031+0000 7fa241f1f640 1 -- 172.21.15.111:0/3619250904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa23c0d1650 msgr2=0x7fa23c0d2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.031+0000 7fa241f1f640 1 --2- 172.21.15.111:0/3619250904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa23c0d1650 0x7fa23c0d2f50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa234009dc0 tx=0x7fa23402a610 comp rx=0 tx=0).stop 2024-09-13T17:59:20.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.031+0000 7fa241f1f640 1 -- 172.21.15.111:0/3619250904 shutdown_connections 2024-09-13T17:59:20.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.031+0000 7fa241f1f640 1 --2- 172.21.15.111:0/3619250904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa23c0d1650 0x7fa23c0d2f50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.031+0000 7fa241f1f640 1 --2- 172.21.15.111:0/3619250904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa23c0b1370 0x7fa23c0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.031+0000 7fa241f1f640 1 -- 172.21.15.111:0/3619250904 >> 172.21.15.111:0/3619250904 conn(0x7fa23c089f60 msgr2=0x7fa23c0d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:20.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.031+0000 7fa241f1f640 1 -- 172.21.15.111:0/3619250904 shutdown_connections 2024-09-13T17:59:20.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.031+0000 7fa241f1f640 1 -- 172.21.15.111:0/3619250904 wait complete. 2024-09-13T17:59:20.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.032+0000 7fa241f1f640 1 Processor -- start 2024-09-13T17:59:20.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.033+0000 7fa241f1f640 1 -- start start 2024-09-13T17:59:20.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.033+0000 7fa241f1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa23c0cd500 0x7fa23c1f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.033+0000 7fa241f1f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa23c0d1650 0x7fa23c0c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.033+0000 7fa241f1f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa23c1535c0 0x7fa23c0c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.033+0000 7fa241f1f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c0ca400 con 0x7fa23c0cd500 2024-09-13T17:59:20.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.033+0000 7fa241f1f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c0b60e0 con 0x7fa23c1535c0 2024-09-13T17:59:20.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.033+0000 7fa241f1f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c0b0a90 con 0x7fa23c0d1650 2024-09-13T17:59:20.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.033+0000 7fa23b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa23c0cd500 0x7fa23c1f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.033+0000 7fa23b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa23c0cd500 0x7fa23c1f1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55440/0 (socket says 172.21.15.111:55440) 2024-09-13T17:59:20.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.033+0000 7fa23b7fe640 1 -- 172.21.15.111:0/236065859 learned_addr learned my addr 172.21.15.111:0/236065859 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:20.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.034+0000 7fa23affd640 1 --2- 172.21.15.111:0/236065859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa23c0d1650 0x7fa23c0c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.034+0000 7fa23bfff640 1 --2- 172.21.15.111:0/236065859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa23c1535c0 0x7fa23c0c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.034+0000 7fa23b7fe640 1 -- 172.21.15.111:0/236065859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa23c0d1650 msgr2=0x7fa23c0c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.034+0000 7fa23b7fe640 1 --2- 172.21.15.111:0/236065859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa23c0d1650 0x7fa23c0c95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.034+0000 7fa23b7fe640 1 -- 172.21.15.111:0/236065859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa23c1535c0 msgr2=0x7fa23c0c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.034+0000 7fa23b7fe640 1 --2- 172.21.15.111:0/236065859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa23c1535c0 0x7fa23c0c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.034+0000 7fa23b7fe640 1 -- 172.21.15.111:0/236065859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa230009400 con 0x7fa23c0cd500 2024-09-13T17:59:20.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.034+0000 7fa23b7fe640 1 --2- 172.21.15.111:0/236065859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa23c0cd500 0x7fa23c1f1d20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00c770 tx=0x7fa22c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.034+0000 7fa223fff640 1 -- 172.21.15.111:0/236065859 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c005ad0 con 0x7fa23c0cd500 2024-09-13T17:59:20.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.034+0000 7fa223fff640 1 -- 172.21.15.111:0/236065859 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22c005c50 con 0x7fa23c0cd500 2024-09-13T17:59:20.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.034+0000 7fa241f1f640 1 -- 172.21.15.111:0/236065859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa234009a70 con 0x7fa23c0cd500 2024-09-13T17:59:20.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.034+0000 7fa223fff640 1 -- 172.21.15.111:0/236065859 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c007070 con 0x7fa23c0cd500 2024-09-13T17:59:20.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.034+0000 7fa241f1f640 1 -- 172.21.15.111:0/236065859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23c0b07b0 con 0x7fa23c0cd500 2024-09-13T17:59:20.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.035+0000 7fa241f1f640 1 -- 172.21.15.111:0/236065859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa23c0cdaf0 con 0x7fa23c0cd500 2024-09-13T17:59:20.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.036+0000 7fa223fff640 1 -- 172.21.15.111:0/236065859 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa22c0071d0 con 0x7fa23c0cd500 2024-09-13T17:59:20.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.036+0000 7fa223fff640 1 --2- 172.21.15.111:0/236065859 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa22806ac00 0x7fa22806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.036+0000 7fa223fff640 1 -- 172.21.15.111:0/236065859 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fa22c088a50 con 0x7fa23c0cd500 2024-09-13T17:59:20.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.036+0000 7fa23affd640 1 --2- 172.21.15.111:0/236065859 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa22806ac00 0x7fa22806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.036+0000 7fa223fff640 1 --2- 172.21.15.111:0/236065859 >> [v2:172.21.15.111:6814/4024408960,v1:172.21.15.111:6816/4024408960] conn(0x7fa228070fa0 0x7fa2280733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.036+0000 7fa223fff640 1 -- 172.21.15.111:0/236065859 --> [v2:172.21.15.111:6814/4024408960,v1:172.21.15.111:6816/4024408960] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa228073ab0 con 0x7fa228070fa0 2024-09-13T17:59:20.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.036+0000 7fa223fff640 1 -- 172.21.15.111:0/236065859 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa22c08d2c0 con 0x7fa23c0cd500 2024-09-13T17:59:20.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.036+0000 7fa23bfff640 1 --2- 172.21.15.111:0/236065859 >> [v2:172.21.15.111:6814/4024408960,v1:172.21.15.111:6816/4024408960] conn(0x7fa228070fa0 0x7fa2280733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.037+0000 7fa23affd640 1 --2- 172.21.15.111:0/236065859 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa22806ac00 0x7fa22806d0c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa23c06b770 tx=0x7fa230008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.037+0000 7fa23bfff640 1 --2- 172.21.15.111:0/236065859 >> [v2:172.21.15.111:6814/4024408960,v1:172.21.15.111:6816/4024408960] conn(0x7fa228070fa0 0x7fa2280733e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.037+0000 7fa223fff640 1 -- 172.21.15.111:0/236065859 <== osd.0 v2:172.21.15.111:6814/4024408960 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa228073ab0 con 0x7fa228070fa0 2024-09-13T17:59:20.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.049+0000 7fa241f1f640 1 -- 172.21.15.111:0/236065859 --> [v2:172.21.15.111:6814/4024408960,v1:172.21.15.111:6816/4024408960] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa23c0d1ce0 con 0x7fa228070fa0 2024-09-13T17:59:20.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.050+0000 7fa223fff640 1 -- 172.21.15.111:0/236065859 <== osd.0 v2:172.21.15.111:6814/4024408960 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa23c0d1ce0 con 0x7fa228070fa0 2024-09-13T17:59:20.052 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T17:59:20.052 INFO:teuthology.orchestra.run.smithi111.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:59:20.052 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T17:59:20.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.050+0000 7fa241f1f640 1 -- 172.21.15.111:0/236065859 >> [v2:172.21.15.111:6814/4024408960,v1:172.21.15.111:6816/4024408960] conn(0x7fa228070fa0 msgr2=0x7fa2280733e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.050+0000 7fa241f1f640 1 --2- 172.21.15.111:0/236065859 >> [v2:172.21.15.111:6814/4024408960,v1:172.21.15.111:6816/4024408960] conn(0x7fa228070fa0 0x7fa2280733e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.050+0000 7fa241f1f640 1 -- 172.21.15.111:0/236065859 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa22806ac00 msgr2=0x7fa22806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.050+0000 7fa241f1f640 1 --2- 172.21.15.111:0/236065859 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa22806ac00 0x7fa22806d0c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa23c06b770 tx=0x7fa230008040 comp rx=0 tx=0).stop 2024-09-13T17:59:20.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.050+0000 7fa241f1f640 1 -- 172.21.15.111:0/236065859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa23c0cd500 msgr2=0x7fa23c1f1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.050+0000 7fa241f1f640 1 --2- 172.21.15.111:0/236065859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa23c0cd500 0x7fa23c1f1d20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00c770 tx=0x7fa22c00cc30 comp rx=0 tx=0).stop 2024-09-13T17:59:20.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.050+0000 7fa23bfff640 1 -- 172.21.15.111:0/236065859 reap_dead start 2024-09-13T17:59:20.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.050+0000 7fa241f1f640 1 -- 172.21.15.111:0/236065859 shutdown_connections 2024-09-13T17:59:20.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.050+0000 7fa241f1f640 1 -- 172.21.15.111:0/236065859 >> 172.21.15.111:0/236065859 conn(0x7fa23c089f60 msgr2=0x7fa23c15ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:20.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.050+0000 7fa241f1f640 1 -- 172.21.15.111:0/236065859 shutdown_connections 2024-09-13T17:59:20.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.050+0000 7fa241f1f640 1 -- 172.21.15.111:0/236065859 wait complete. 2024-09-13T17:59:20.063 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T17:59:20.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.161+0000 7f81404c9640 1 Processor -- start 2024-09-13T17:59:20.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.161+0000 7f81404c9640 1 -- start start 2024-09-13T17:59:20.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.162+0000 7f81404c9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f813815c500 0x7f81380d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.162+0000 7f81404c9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f81380d2590 0x7f81380cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.162+0000 7f81404c9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81380cd450 con 0x7f81380cde20 2024-09-13T17:59:20.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.162+0000 7f81404c9640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81380d29f0 con 0x7f81380d2590 2024-09-13T17:59:20.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.162+0000 7f81404c9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81380d0cf0 con 0x7f813815c500 2024-09-13T17:59:20.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.162+0000 7f813da64640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f81380d2590 0x7f81380cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.162+0000 7f813da64640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f81380d2590 0x7f81380cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55454/0 (socket says 172.21.15.111:55454) 2024-09-13T17:59:20.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.162+0000 7f813da64640 1 -- 172.21.15.111:0/211294566 learned_addr learned my addr 172.21.15.111:0/211294566 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:20.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.163+0000 7f813e265640 1 --2- 172.21.15.111:0/211294566 >> v2:172.21.15.188:3300/0 conn(0x7f813815c500 0x7f81380d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.163+0000 7f813da64640 1 -- 172.21.15.111:0/211294566 >> v2:172.21.15.188:3300/0 conn(0x7f813815c500 msgr2=0x7f81380d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:20.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.163+0000 7f813da64640 1 --2- 172.21.15.111:0/211294566 >> v2:172.21.15.188:3300/0 conn(0x7f813815c500 0x7f81380d2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.163+0000 7f813da64640 1 -- 172.21.15.111:0/211294566 >> v1:172.21.15.188:6789/0 conn(0x7f81380cde20 legacy=0x7f81380d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:20.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.163+0000 7f813da64640 1 -- 172.21.15.111:0/211294566 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81380d0fd0 con 0x7f81380d2590 2024-09-13T17:59:20.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.163+0000 7f813e265640 1 --2- 172.21.15.111:0/211294566 >> v2:172.21.15.188:3300/0 conn(0x7f813815c500 0x7f81380d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:59:20.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.163+0000 7f813da64640 1 --2- 172.21.15.111:0/211294566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81380d2590 0x7f81380cd8e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f81380cfec0 tx=0x7f813402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d6995778806dbe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.164+0000 7f813d263640 1 -- 172.21.15.111:0/211294566 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8134032440 con 0x7f81380d2590 2024-09-13T17:59:20.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.164+0000 7f813d263640 1 -- 172.21.15.111:0/211294566 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81340325a0 con 0x7f81380d2590 2024-09-13T17:59:20.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.164+0000 7f813d263640 1 -- 172.21.15.111:0/211294566 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8134032930 con 0x7f81380d2590 2024-09-13T17:59:20.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.164+0000 7f81404c9640 1 -- 172.21.15.111:0/211294566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81380d2590 msgr2=0x7f81380cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.164+0000 7f81404c9640 1 --2- 172.21.15.111:0/211294566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81380d2590 0x7f81380cd8e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f81380cfec0 tx=0x7f813402a270 comp rx=0 tx=0).stop 2024-09-13T17:59:20.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.164+0000 7f81404c9640 1 -- 172.21.15.111:0/211294566 shutdown_connections 2024-09-13T17:59:20.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.164+0000 7f81404c9640 1 --2- 172.21.15.111:0/211294566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81380d2590 0x7f81380cd8e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.164+0000 7f81404c9640 1 --2- 172.21.15.111:0/211294566 >> v2:172.21.15.188:3300/0 conn(0x7f813815c500 0x7f81380d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.164+0000 7f81404c9640 1 -- 172.21.15.111:0/211294566 >> 172.21.15.111:0/211294566 conn(0x7f8138089f60 msgr2=0x7f813809b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:20.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.164+0000 7f81404c9640 1 -- 172.21.15.111:0/211294566 shutdown_connections 2024-09-13T17:59:20.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.164+0000 7f81404c9640 1 -- 172.21.15.111:0/211294566 wait complete. 2024-09-13T17:59:20.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.166+0000 7f81404c9640 1 Processor -- start 2024-09-13T17:59:20.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.166+0000 7f81404c9640 1 -- start start 2024-09-13T17:59:20.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.166+0000 7f81404c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81380cde20 0x7f813815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.166+0000 7f81404c9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81380d2590 0x7f81381f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.166+0000 7f81404c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813815c500 0x7f813815e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.166+0000 7f81404c9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81380b6230 con 0x7f81380d2590 2024-09-13T17:59:20.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.166+0000 7f81404c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813809fef0 con 0x7f81380cde20 2024-09-13T17:59:20.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.166+0000 7f81404c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81380b55b0 con 0x7f813815c500 2024-09-13T17:59:20.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.166+0000 7f813da64640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81380d2590 0x7f81381f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.166+0000 7f813da64640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81380d2590 0x7f81381f21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55466/0 (socket says 172.21.15.111:55466) 2024-09-13T17:59:20.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.166+0000 7f813da64640 1 -- 172.21.15.111:0/2237710609 learned_addr learned my addr 172.21.15.111:0/2237710609 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:20.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.167+0000 7f813e265640 1 --2- 172.21.15.111:0/2237710609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81380cde20 0x7f813815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.167+0000 7f813ea66640 1 --2- 172.21.15.111:0/2237710609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813815c500 0x7f813815e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.167+0000 7f813da64640 1 -- 172.21.15.111:0/2237710609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813815c500 msgr2=0x7f813815e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.167+0000 7f813da64640 1 --2- 172.21.15.111:0/2237710609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813815c500 0x7f813815e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.167+0000 7f813da64640 1 -- 172.21.15.111:0/2237710609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81380cde20 msgr2=0x7f813815a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.167+0000 7f813da64640 1 --2- 172.21.15.111:0/2237710609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81380cde20 0x7f813815a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.167+0000 7f813da64640 1 -- 172.21.15.111:0/2237710609 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81340094d0 con 0x7f81380d2590 2024-09-13T17:59:20.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.167+0000 7f813da64640 1 --2- 172.21.15.111:0/2237710609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81380d2590 0x7f81381f21b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8134000c00 tx=0x7f81340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.167+0000 7f812affd640 1 -- 172.21.15.111:0/2237710609 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8134030b80 con 0x7f81380d2590 2024-09-13T17:59:20.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.167+0000 7f812affd640 1 -- 172.21.15.111:0/2237710609 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8134030ce0 con 0x7f81380d2590 2024-09-13T17:59:20.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.167+0000 7f81404c9640 1 -- 172.21.15.111:0/2237710609 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81380b5890 con 0x7f81380d2590 2024-09-13T17:59:20.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.167+0000 7f812affd640 1 -- 172.21.15.111:0/2237710609 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813403d680 con 0x7f81380d2590 2024-09-13T17:59:20.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.167+0000 7f81404c9640 1 -- 172.21.15.111:0/2237710609 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81380b5e50 con 0x7f81380d2590 2024-09-13T17:59:20.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.168+0000 7f81404c9640 1 -- 172.21.15.111:0/2237710609 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81380b6370 con 0x7f81380d2590 2024-09-13T17:59:20.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.169+0000 7f812affd640 1 -- 172.21.15.111:0/2237710609 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8134039040 con 0x7f81380d2590 2024-09-13T17:59:20.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.169+0000 7f812affd640 1 --2- 172.21.15.111:0/2237710609 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f811c06ac00 0x7f811c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.170+0000 7f812affd640 1 -- 172.21.15.111:0/2237710609 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f81340a9e60 con 0x7f81380d2590 2024-09-13T17:59:20.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.170+0000 7f813e265640 1 --2- 172.21.15.111:0/2237710609 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f811c06ac00 0x7f811c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.170+0000 7f813e265640 1 --2- 172.21.15.111:0/2237710609 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f811c06ac00 0x7f811c06d0c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f812c00aac0 tx=0x7f812c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.174+0000 7f812affd640 1 -- 172.21.15.111:0/2237710609 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f81380b6370 con 0x7f81380d2590 2024-09-13T17:59:20.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.342+0000 7f81404c9640 1 -- 172.21.15.111:0/2237710609 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f81380d12b0 con 0x7f811c06ac00 2024-09-13T17:59:20.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.342+0000 7f812affd640 1 -- 172.21.15.111:0/2237710609 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f812c00c040 con 0x7f811c06ac00 2024-09-13T17:59:20.344 INFO:teuthology.orchestra.run.smithi111.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T17:59:20.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.344+0000 7f81404c9640 1 -- 172.21.15.111:0/2237710609 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f811c06ac00 msgr2=0x7f811c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.344+0000 7f81404c9640 1 --2- 172.21.15.111:0/2237710609 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f811c06ac00 0x7f811c06d0c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f812c00aac0 tx=0x7f812c008040 comp rx=0 tx=0).stop 2024-09-13T17:59:20.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.344+0000 7f81404c9640 1 -- 172.21.15.111:0/2237710609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81380d2590 msgr2=0x7f81381f21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.344+0000 7f81404c9640 1 --2- 172.21.15.111:0/2237710609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81380d2590 0x7f81381f21b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8134000c00 tx=0x7f81340046c0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.344+0000 7f81404c9640 1 -- 172.21.15.111:0/2237710609 shutdown_connections 2024-09-13T17:59:20.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.344+0000 7f81404c9640 1 --2- 172.21.15.111:0/2237710609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813815c500 0x7f813815e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.344+0000 7f81404c9640 1 --2- 172.21.15.111:0/2237710609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81380d2590 0x7f81381f21b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.344+0000 7f81404c9640 1 --2- 172.21.15.111:0/2237710609 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f811c06ac00 0x7f811c06d0c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.344+0000 7f81404c9640 1 --2- 172.21.15.111:0/2237710609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81380cde20 0x7f813815a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.344+0000 7f81404c9640 1 -- 172.21.15.111:0/2237710609 >> 172.21.15.111:0/2237710609 conn(0x7f8138089f60 msgr2=0x7f81380b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:20.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.345+0000 7f81404c9640 1 -- 172.21.15.111:0/2237710609 shutdown_connections 2024-09-13T17:59:20.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.345+0000 7f81404c9640 1 -- 172.21.15.111:0/2237710609 wait complete. 2024-09-13T17:59:20.360 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T17:59:20.361 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:59:20.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.464+0000 7f073527d640 1 Processor -- start 2024-09-13T17:59:20.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.464+0000 7f073527d640 1 -- start start 2024-09-13T17:59:20.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.465+0000 7f073527d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f073006b9f0 0x7f07300989d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.465+0000 7f073527d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0730098f10 0x7f07300cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.465+0000 7f073527d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07300caac0 con 0x7f073015c830 2024-09-13T17:59:20.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.465+0000 7f073527d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07300993c0 con 0x7f073006b9f0 2024-09-13T17:59:20.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.465+0000 7f073527d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07300cb080 con 0x7f0730098f10 2024-09-13T17:59:20.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.465+0000 7f072e7fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f073006b9f0 0x7f07300989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.465+0000 7f072e7fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f073006b9f0 0x7f07300989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55482/0 (socket says 172.21.15.111:55482) 2024-09-13T17:59:20.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.465+0000 7f072e7fc640 1 -- 172.21.15.111:0/3087941499 learned_addr learned my addr 172.21.15.111:0/3087941499 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:20.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.466+0000 7f072f7fe640 1 --2- 172.21.15.111:0/3087941499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0730098f10 0x7f07300cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.466+0000 7f072e7fc640 1 -- 172.21.15.111:0/3087941499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0730098f10 msgr2=0x7f07300cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:20.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.466+0000 7f072e7fc640 1 --2- 172.21.15.111:0/3087941499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0730098f10 0x7f07300cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.466+0000 7f072e7fc640 1 -- 172.21.15.111:0/3087941499 >> v1:172.21.15.188:6789/0 conn(0x7f073015c830 legacy=0x7f0730099590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.466+0000 7f072e7fc640 1 -- 172.21.15.111:0/3087941499 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07300b2d40 con 0x7f073006b9f0 2024-09-13T17:59:20.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.467+0000 7f072e7fc640 1 --2- 172.21.15.111:0/3087941499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f073006b9f0 0x7f07300989d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0724009870 tx=0x7f072402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5d98232deb4435 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.467+0000 7f072dffb640 1 -- 172.21.15.111:0/3087941499 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0724032440 con 0x7f073006b9f0 2024-09-13T17:59:20.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.467+0000 7f072dffb640 1 -- 172.21.15.111:0/3087941499 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07240325a0 con 0x7f073006b9f0 2024-09-13T17:59:20.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.467+0000 7f072dffb640 1 -- 172.21.15.111:0/3087941499 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0724032930 con 0x7f073006b9f0 2024-09-13T17:59:20.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.467+0000 7f073527d640 1 -- 172.21.15.111:0/3087941499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f073006b9f0 msgr2=0x7f07300989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.467+0000 7f073527d640 1 --2- 172.21.15.111:0/3087941499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f073006b9f0 0x7f07300989d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0724009870 tx=0x7f072402a270 comp rx=0 tx=0).stop 2024-09-13T17:59:20.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.467+0000 7f073527d640 1 -- 172.21.15.111:0/3087941499 shutdown_connections 2024-09-13T17:59:20.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.467+0000 7f073527d640 1 --2- 172.21.15.111:0/3087941499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f073006b9f0 0x7f07300989d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.467+0000 7f073527d640 1 --2- 172.21.15.111:0/3087941499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0730098f10 0x7f07300cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.467+0000 7f073527d640 1 -- 172.21.15.111:0/3087941499 >> 172.21.15.111:0/3087941499 conn(0x7f07300b1f60 msgr2=0x7f07300b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:20.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.467+0000 7f073527d640 1 -- 172.21.15.111:0/3087941499 shutdown_connections 2024-09-13T17:59:20.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.468+0000 7f073527d640 1 -- 172.21.15.111:0/3087941499 wait complete. 2024-09-13T17:59:20.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.469+0000 7f073527d640 1 Processor -- start 2024-09-13T17:59:20.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.469+0000 7f073527d640 1 -- start start 2024-09-13T17:59:20.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.469+0000 7f073527d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0730098f10 0x7f073015efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.469+0000 7f073527d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f073015c830 0x7f073015f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f073527d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f073015b670 0x7f07300c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f073527d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07300ca810 con 0x7f0730098f10 2024-09-13T17:59:20.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f073527d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07300c8730 con 0x7f073015c830 2024-09-13T17:59:20.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f073527d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07300c8a10 con 0x7f073015b670 2024-09-13T17:59:20.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f072effd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0730098f10 0x7f073015efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f072effd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0730098f10 0x7f073015efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55492/0 (socket says 172.21.15.111:55492) 2024-09-13T17:59:20.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f072effd640 1 -- 172.21.15.111:0/2633816329 learned_addr learned my addr 172.21.15.111:0/2633816329 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:20.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f072f7fe640 1 --2- 172.21.15.111:0/2633816329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f073015b670 0x7f07300c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f072e7fc640 1 --2- 172.21.15.111:0/2633816329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f073015c830 0x7f073015f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f072f7fe640 1 -- 172.21.15.111:0/2633816329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f073015c830 msgr2=0x7f073015f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f072f7fe640 1 --2- 172.21.15.111:0/2633816329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f073015c830 0x7f073015f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f072f7fe640 1 -- 172.21.15.111:0/2633816329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0730098f10 msgr2=0x7f073015efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f072f7fe640 1 --2- 172.21.15.111:0/2633816329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0730098f10 0x7f073015efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f072f7fe640 1 -- 172.21.15.111:0/2633816329 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07240094d0 con 0x7f073015b670 2024-09-13T17:59:20.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f072e7fc640 1 --2- 172.21.15.111:0/2633816329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f073015c830 0x7f073015f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:59:20.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f072effd640 1 --2- 172.21.15.111:0/2633816329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0730098f10 0x7f073015efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:59:20.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f072f7fe640 1 --2- 172.21.15.111:0/2633816329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f073015b670 0x7f07300c9b30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f072000efc0 tx=0x7f072000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.470+0000 7f07177fe640 1 -- 172.21.15.111:0/2633816329 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720009c00 con 0x7f073015b670 2024-09-13T17:59:20.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.471+0000 7f07177fe640 1 -- 172.21.15.111:0/2633816329 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072000f040 con 0x7f073015b670 2024-09-13T17:59:20.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.471+0000 7f073527d640 1 -- 172.21.15.111:0/2633816329 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07300c8cf0 con 0x7f073015b670 2024-09-13T17:59:20.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.471+0000 7f07177fe640 1 -- 172.21.15.111:0/2633816329 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07200146c0 con 0x7f073015b670 2024-09-13T17:59:20.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.471+0000 7f073527d640 1 -- 172.21.15.111:0/2633816329 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07300c94f0 con 0x7f073015b670 2024-09-13T17:59:20.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.471+0000 7f073527d640 1 -- 172.21.15.111:0/2633816329 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f07300c88a0 con 0x7f073015b670 2024-09-13T17:59:20.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.472+0000 7f07177fe640 1 -- 172.21.15.111:0/2633816329 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0720014820 con 0x7f073015b670 2024-09-13T17:59:20.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.473+0000 7f07177fe640 1 --2- 172.21.15.111:0/2633816329 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f071c06ac00 0x7f071c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.473+0000 7f07177fe640 1 -- 172.21.15.111:0/2633816329 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f072008a940 con 0x7f073015b670 2024-09-13T17:59:20.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.473+0000 7f072effd640 1 --2- 172.21.15.111:0/2633816329 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f071c06ac00 0x7f071c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.473+0000 7f07177fe640 1 --2- 172.21.15.111:0/2633816329 >> [v2:172.21.15.111:6808/2899699337,v1:172.21.15.111:6809/2899699337] conn(0x7f071c070fa0 0x7f071c0733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.473+0000 7f07177fe640 1 -- 172.21.15.111:0/2633816329 --> [v2:172.21.15.111:6808/2899699337,v1:172.21.15.111:6809/2899699337] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f071c073ab0 con 0x7f071c070fa0 2024-09-13T17:59:20.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.473+0000 7f07177fe640 1 -- 172.21.15.111:0/2633816329 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f072008acb0 con 0x7f073015b670 2024-09-13T17:59:20.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.473+0000 7f072e7fc640 1 --2- 172.21.15.111:0/2633816329 >> [v2:172.21.15.111:6808/2899699337,v1:172.21.15.111:6809/2899699337] conn(0x7f071c070fa0 0x7f071c0733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.473+0000 7f072effd640 1 --2- 172.21.15.111:0/2633816329 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f071c06ac00 0x7f071c06d0c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f073006ead0 tx=0x7f0728009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.473+0000 7f072e7fc640 1 --2- 172.21.15.111:0/2633816329 >> [v2:172.21.15.111:6808/2899699337,v1:172.21.15.111:6809/2899699337] conn(0x7f071c070fa0 0x7f071c0733e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.474+0000 7f07177fe640 1 -- 172.21.15.111:0/2633816329 <== osd.1 v2:172.21.15.111:6808/2899699337 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f071c073ab0 con 0x7f071c070fa0 2024-09-13T17:59:20.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.486+0000 7f073527d640 1 -- 172.21.15.111:0/2633816329 --> [v2:172.21.15.111:6808/2899699337,v1:172.21.15.111:6809/2899699337] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f07300ca950 con 0x7f071c070fa0 2024-09-13T17:59:20.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.486+0000 7f07177fe640 1 -- 172.21.15.111:0/2633816329 <== osd.1 v2:172.21.15.111:6808/2899699337 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f07300ca950 con 0x7f071c070fa0 2024-09-13T17:59:20.488 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T17:59:20.488 INFO:teuthology.orchestra.run.smithi111.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:59:20.488 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T17:59:20.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.486+0000 7f073527d640 1 -- 172.21.15.111:0/2633816329 >> [v2:172.21.15.111:6808/2899699337,v1:172.21.15.111:6809/2899699337] conn(0x7f071c070fa0 msgr2=0x7f071c0733e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.486+0000 7f073527d640 1 --2- 172.21.15.111:0/2633816329 >> [v2:172.21.15.111:6808/2899699337,v1:172.21.15.111:6809/2899699337] conn(0x7f071c070fa0 0x7f071c0733e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.486+0000 7f073527d640 1 -- 172.21.15.111:0/2633816329 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f071c06ac00 msgr2=0x7f071c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.486+0000 7f073527d640 1 --2- 172.21.15.111:0/2633816329 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f071c06ac00 0x7f071c06d0c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f073006ead0 tx=0x7f0728009040 comp rx=0 tx=0).stop 2024-09-13T17:59:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.486+0000 7f073527d640 1 -- 172.21.15.111:0/2633816329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f073015b670 msgr2=0x7f07300c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.486+0000 7f073527d640 1 --2- 172.21.15.111:0/2633816329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f073015b670 0x7f07300c9b30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f072000efc0 tx=0x7f072000c490 comp rx=0 tx=0).stop 2024-09-13T17:59:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.486+0000 7f072f7fe640 1 -- 172.21.15.111:0/2633816329 reap_dead start 2024-09-13T17:59:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.487+0000 7f073527d640 1 -- 172.21.15.111:0/2633816329 shutdown_connections 2024-09-13T17:59:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.487+0000 7f073527d640 1 -- 172.21.15.111:0/2633816329 >> 172.21.15.111:0/2633816329 conn(0x7f07300b1f60 msgr2=0x7f07300b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.487+0000 7f073527d640 1 -- 172.21.15.111:0/2633816329 shutdown_connections 2024-09-13T17:59:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.487+0000 7f073527d640 1 -- 172.21.15.111:0/2633816329 wait complete. 2024-09-13T17:59:20.499 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T17:59:20.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.598+0000 7f88ed325640 1 Processor -- start 2024-09-13T17:59:20.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.598+0000 7f88ed325640 1 -- start start 2024-09-13T17:59:20.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.599+0000 7f88ed325640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f88e8159310 0x7f88e80ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.599+0000 7f88ed325640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88e80b2b40 0x7f88e80ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.599+0000 7f88ed325640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e80ccbe0 con 0x7f88e80ab680 2024-09-13T17:59:20.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.599+0000 7f88ed325640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e80ad940 con 0x7f88e8159310 2024-09-13T17:59:20.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.599+0000 7f88ed325640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e80b1810 con 0x7f88e80b2b40 2024-09-13T17:59:20.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.599+0000 7f88e6ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f88e8159310 0x7f88e80ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.599+0000 7f88e77fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f88e80ab680 0x7f88e80ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43088/0 (socket says 172.21.15.111:43088) 2024-09-13T17:59:20.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.599+0000 7f88e77fe640 1 -- 172.21.15.111:0/2571318033 learned_addr learned my addr 172.21.15.111:0/2571318033 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:20.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.599+0000 7f88e67fc640 1 --2- 172.21.15.111:0/2571318033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88e80b2b40 0x7f88e80ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.599+0000 7f88e6ffd640 1 -- 172.21.15.111:0/2571318033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88e80b2b40 msgr2=0x7f88e80ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:20.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.599+0000 7f88e6ffd640 1 --2- 172.21.15.111:0/2571318033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88e80b2b40 0x7f88e80ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.599+0000 7f88e6ffd640 1 -- 172.21.15.111:0/2571318033 >> v1:172.21.15.111:6789/0 conn(0x7f88e80ab680 legacy=0x7f88e80ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.599+0000 7f88e5ffb640 1 -- 172.21.15.111:0/2571318033 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3412461037 0 0) 0x7f88e80ccbe0 con 0x7f88e80ab680 2024-09-13T17:59:20.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.599+0000 7f88e6ffd640 1 -- 172.21.15.111:0/2571318033 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e806d9e0 con 0x7f88e8159310 2024-09-13T17:59:20.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.600+0000 7f88e6ffd640 1 --2- 172.21.15.111:0/2571318033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88e8159310 0x7f88e80ad400 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f88d800ba40 tx=0x7f88d802c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c572343083c3b8f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.600+0000 7f88e5ffb640 1 -- 172.21.15.111:0/2571318033 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8031710 con 0x7f88e8159310 2024-09-13T17:59:20.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.600+0000 7f88e5ffb640 1 -- 172.21.15.111:0/2571318033 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d802f050 con 0x7f88e8159310 2024-09-13T17:59:20.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.600+0000 7f88e5ffb640 1 -- 172.21.15.111:0/2571318033 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8031a90 con 0x7f88e8159310 2024-09-13T17:59:20.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.600+0000 7f88ed325640 1 -- 172.21.15.111:0/2571318033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88e8159310 msgr2=0x7f88e80ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.600+0000 7f88ed325640 1 --2- 172.21.15.111:0/2571318033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88e8159310 0x7f88e80ad400 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f88d800ba40 tx=0x7f88d802c7b0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.601+0000 7f88ed325640 1 -- 172.21.15.111:0/2571318033 shutdown_connections 2024-09-13T17:59:20.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.601+0000 7f88ed325640 1 --2- 172.21.15.111:0/2571318033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88e80b2b40 0x7f88e80ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.601+0000 7f88ed325640 1 --2- 172.21.15.111:0/2571318033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88e8159310 0x7f88e80ad400 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.601+0000 7f88ed325640 1 -- 172.21.15.111:0/2571318033 >> 172.21.15.111:0/2571318033 conn(0x7f88e806a0b0 msgr2=0x7f88e806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:20.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.601+0000 7f88ed325640 1 -- 172.21.15.111:0/2571318033 shutdown_connections 2024-09-13T17:59:20.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.601+0000 7f88ed325640 1 -- 172.21.15.111:0/2571318033 wait complete. 2024-09-13T17:59:20.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.602+0000 7f88ed325640 1 Processor -- start 2024-09-13T17:59:20.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.602+0000 7f88ed325640 1 -- start start 2024-09-13T17:59:20.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88ed325640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88e80ab680 0x7f88e8157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88ed325640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88e80b2b40 0x7f88e8154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88ed325640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88e8159310 0x7f88e80974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88ed325640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e81edf70 con 0x7f88e8159310 2024-09-13T17:59:20.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88ed325640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e8151e00 con 0x7f88e80b2b40 2024-09-13T17:59:20.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88ed325640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e8156100 con 0x7f88e80ab680 2024-09-13T17:59:20.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88e77fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88e8159310 0x7f88e80974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88e77fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88e8159310 0x7f88e80974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55508/0 (socket says 172.21.15.111:55508) 2024-09-13T17:59:20.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88e77fe640 1 -- 172.21.15.111:0/4204760983 learned_addr learned my addr 172.21.15.111:0/4204760983 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:20.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88e6ffd640 1 --2- 172.21.15.111:0/4204760983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88e80ab680 0x7f88e8157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88e67fc640 1 --2- 172.21.15.111:0/4204760983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88e80b2b40 0x7f88e8154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88e77fe640 1 -- 172.21.15.111:0/4204760983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88e80ab680 msgr2=0x7f88e8157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88e77fe640 1 --2- 172.21.15.111:0/4204760983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88e80ab680 0x7f88e8157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88e77fe640 1 -- 172.21.15.111:0/4204760983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88e80b2b40 msgr2=0x7f88e8154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88e77fe640 1 --2- 172.21.15.111:0/4204760983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88e80b2b40 0x7f88e8154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88e77fe640 1 -- 172.21.15.111:0/4204760983 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d800b6a0 con 0x7f88e8159310 2024-09-13T17:59:20.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88e77fe640 1 --2- 172.21.15.111:0/4204760983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88e8159310 0x7f88e80974f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f88e000c770 tx=0x7f88e000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.603+0000 7f88cf7fe640 1 -- 172.21.15.111:0/4204760983 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e000ce80 con 0x7f88e8159310 2024-09-13T17:59:20.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.604+0000 7f88cf7fe640 1 -- 172.21.15.111:0/4204760983 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e000a640 con 0x7f88e8159310 2024-09-13T17:59:20.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.604+0000 7f88ed325640 1 -- 172.21.15.111:0/4204760983 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e81563e0 con 0x7f88e8159310 2024-09-13T17:59:20.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.604+0000 7f88cf7fe640 1 -- 172.21.15.111:0/4204760983 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e000a9f0 con 0x7f88e8159310 2024-09-13T17:59:20.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.604+0000 7f88ed325640 1 -- 172.21.15.111:0/4204760983 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e8152050 con 0x7f88e8159310 2024-09-13T17:59:20.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.605+0000 7f88ed325640 1 -- 172.21.15.111:0/4204760983 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e8152270 con 0x7f88e8159310 2024-09-13T17:59:20.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.605+0000 7f88cf7fe640 1 -- 172.21.15.111:0/4204760983 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f88e000ab50 con 0x7f88e8159310 2024-09-13T17:59:20.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.605+0000 7f88cf7fe640 1 --2- 172.21.15.111:0/4204760983 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f88d406ac00 0x7f88d406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.605+0000 7f88cf7fe640 1 -- 172.21.15.111:0/4204760983 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f88e00888f0 con 0x7f88e8159310 2024-09-13T17:59:20.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.605+0000 7f88e6ffd640 1 --2- 172.21.15.111:0/4204760983 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f88d406ac00 0x7f88d406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.606+0000 7f88e6ffd640 1 --2- 172.21.15.111:0/4204760983 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f88d406ac00 0x7f88d406d0c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f88e80afdf0 tx=0x7f88d8031000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.610+0000 7f88cf7fe640 1 -- 172.21.15.111:0/4204760983 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f88e0053d70 con 0x7f88e8159310 2024-09-13T17:59:20.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.774+0000 7f88ed325640 1 -- 172.21.15.111:0/4204760983 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f88e806dfa0 con 0x7f88d406ac00 2024-09-13T17:59:20.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.774+0000 7f88cf7fe640 1 -- 172.21.15.111:0/4204760983 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f88d8042070 con 0x7f88d406ac00 2024-09-13T17:59:20.776 INFO:teuthology.orchestra.run.smithi111.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T17:59:20.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.776+0000 7f88ed325640 1 -- 172.21.15.111:0/4204760983 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f88d406ac00 msgr2=0x7f88d406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.776+0000 7f88ed325640 1 --2- 172.21.15.111:0/4204760983 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f88d406ac00 0x7f88d406d0c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f88e80afdf0 tx=0x7f88d8031000 comp rx=0 tx=0).stop 2024-09-13T17:59:20.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.776+0000 7f88ed325640 1 -- 172.21.15.111:0/4204760983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88e8159310 msgr2=0x7f88e80974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.776+0000 7f88ed325640 1 --2- 172.21.15.111:0/4204760983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88e8159310 0x7f88e80974f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f88e000c770 tx=0x7f88e000cc30 comp rx=0 tx=0).stop 2024-09-13T17:59:20.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.776+0000 7f88ed325640 1 -- 172.21.15.111:0/4204760983 shutdown_connections 2024-09-13T17:59:20.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.776+0000 7f88ed325640 1 --2- 172.21.15.111:0/4204760983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88e8159310 0x7f88e80974f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.776+0000 7f88ed325640 1 --2- 172.21.15.111:0/4204760983 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f88d406ac00 0x7f88d406d0c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.776+0000 7f88ed325640 1 --2- 172.21.15.111:0/4204760983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88e80b2b40 0x7f88e8154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.776+0000 7f88ed325640 1 --2- 172.21.15.111:0/4204760983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88e80ab680 0x7f88e8157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.776+0000 7f88ed325640 1 -- 172.21.15.111:0/4204760983 >> 172.21.15.111:0/4204760983 conn(0x7f88e806a0b0 msgr2=0x7f88e80ad040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:20.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.776+0000 7f88ed325640 1 -- 172.21.15.111:0/4204760983 shutdown_connections 2024-09-13T17:59:20.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.777+0000 7f88ed325640 1 -- 172.21.15.111:0/4204760983 wait complete. 2024-09-13T17:59:20.789 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T17:59:20.790 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:59:20.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.889+0000 7ff2da518640 1 Processor -- start 2024-09-13T17:59:20.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.889+0000 7ff2da518640 1 -- start start 2024-09-13T17:59:20.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.890+0000 7ff2da518640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff2d40cdff0 0x7ff2d406c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.890+0000 7ff2da518640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d40cd450 con 0x7ff2d40d2760 2024-09-13T17:59:20.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.890+0000 7ff2da518640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d40d0cf0 con 0x7ff2d415c500 2024-09-13T17:59:20.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.890+0000 7ff2da518640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d40d0fd0 con 0x7ff2d40cdff0 2024-09-13T17:59:20.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.890+0000 7ff2d37fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff2d40d2760 0x7ff2d40cd8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43102/0 (socket says 172.21.15.111:43102) 2024-09-13T17:59:20.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.890+0000 7ff2d37fe640 1 -- 172.21.15.111:0/1440738557 learned_addr learned my addr 172.21.15.111:0/1440738557 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:20.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.890+0000 7ff2d2ffd640 1 -- 172.21.15.111:0/1440738557 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552629330 0 0) 0x7ff2d40cd450 con 0x7ff2d40d2760 2024-09-13T17:59:20.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.890+0000 7ff2d2ffd640 1 -- 172.21.15.111:0/1440738557 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c0003520 con 0x7ff2d40d2760 2024-09-13T17:59:20.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.890+0000 7ff2d8ab5640 1 --2- 172.21.15.111:0/1440738557 >> v2:172.21.15.188:3300/0 conn(0x7ff2d40cdff0 0x7ff2d406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d2ffd640 1 -- 172.21.15.111:0/1440738557 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3172191093 0 0) 0x7ff2c0003520 con 0x7ff2d40d2760 2024-09-13T17:59:20.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d2ffd640 1 -- 172.21.15.111:0/1440738557 >> v2:172.21.15.188:3300/0 conn(0x7ff2d40cdff0 msgr2=0x7ff2d406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:20.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d2ffd640 1 --2- 172.21.15.111:0/1440738557 >> v2:172.21.15.188:3300/0 conn(0x7ff2d40cdff0 0x7ff2d406c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d2ffd640 1 -- 172.21.15.111:0/1440738557 >> v1:172.21.15.188:6789/0 conn(0x7ff2d415c500 legacy=0x7ff2d40d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d2ffd640 1 -- 172.21.15.111:0/1440738557 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d40cd0e0 con 0x7ff2d40d2760 2024-09-13T17:59:20.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d2ffd640 1 -- 172.21.15.111:0/1440738557 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7ff2bc001670 con 0x7ff2d40d2760 2024-09-13T17:59:20.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d2ffd640 1 -- 172.21.15.111:0/1440738557 >> v1:172.21.15.111:6789/0 conn(0x7ff2d40d2760 legacy=0x7ff2d40cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d2ffd640 1 --2- 172.21.15.111:0/1440738557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2c0003970 0x7ff2c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d2ffd640 1 --2- 172.21.15.111:0/1440738557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2c00047c0 0x7ff2c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d2ffd640 1 --2- 172.21.15.111:0/1440738557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c0005180 0x7ff2c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d2ffd640 1 -- 172.21.15.111:0/1440738557 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d40d0fd0 con 0x7ff2c0005180 2024-09-13T17:59:20.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d2ffd640 1 -- 172.21.15.111:0/1440738557 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c000c020 con 0x7ff2c0003970 2024-09-13T17:59:20.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d2ffd640 1 -- 172.21.15.111:0/1440738557 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c000c1e0 con 0x7ff2c00047c0 2024-09-13T17:59:20.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d8ab5640 1 --2- 172.21.15.111:0/1440738557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c0005180 0x7ff2c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d37fe640 1 --2- 172.21.15.111:0/1440738557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2c00047c0 0x7ff2c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d3fff640 1 --2- 172.21.15.111:0/1440738557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2c0003970 0x7ff2c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d8ab5640 1 -- 172.21.15.111:0/1440738557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2c00047c0 msgr2=0x7ff2c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:20.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d8ab5640 1 --2- 172.21.15.111:0/1440738557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2c00047c0 0x7ff2c0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d8ab5640 1 -- 172.21.15.111:0/1440738557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2c0003970 msgr2=0x7ff2c0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:20.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d8ab5640 1 --2- 172.21.15.111:0/1440738557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2c0003970 0x7ff2c0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d8ab5640 1 -- 172.21.15.111:0/1440738557 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c0003520 con 0x7ff2c0005180 2024-09-13T17:59:20.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.891+0000 7ff2d37fe640 1 --2- 172.21.15.111:0/1440738557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2c00047c0 0x7ff2c0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:20.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.892+0000 7ff2d8ab5640 1 -- 172.21.15.111:0/1440738557 reap_dead start 2024-09-13T17:59:20.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.892+0000 7ff2d8ab5640 1 --2- 172.21.15.111:0/1440738557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c0005180 0x7ff2c000b980 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009fc0 tx=0x7ff2c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db3eeb3c9298c90 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.892+0000 7ff2d2ffd640 1 -- 172.21.15.111:0/1440738557 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc003b90 con 0x7ff2c0005180 2024-09-13T17:59:20.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.892+0000 7ff2d2ffd640 1 -- 172.21.15.111:0/1440738557 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc00f040 con 0x7ff2c0005180 2024-09-13T17:59:20.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.892+0000 7ff2d2ffd640 1 -- 172.21.15.111:0/1440738557 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc004020 con 0x7ff2c0005180 2024-09-13T17:59:20.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.893+0000 7ff2da518640 1 -- 172.21.15.111:0/1440738557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c0005180 msgr2=0x7ff2c000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.893+0000 7ff2da518640 1 --2- 172.21.15.111:0/1440738557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c0005180 0x7ff2c000b980 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009fc0 tx=0x7ff2c00057f0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.893+0000 7ff2da518640 1 -- 172.21.15.111:0/1440738557 shutdown_connections 2024-09-13T17:59:20.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.893+0000 7ff2da518640 1 --2- 172.21.15.111:0/1440738557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c0005180 0x7ff2c000b980 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.893+0000 7ff2da518640 1 -- 172.21.15.111:0/1440738557 >> 172.21.15.111:0/1440738557 conn(0x7ff2d4089f60 msgr2=0x7ff2d409b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:20.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.893+0000 7ff2da518640 1 -- 172.21.15.111:0/1440738557 shutdown_connections 2024-09-13T17:59:20.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.893+0000 7ff2da518640 1 -- 172.21.15.111:0/1440738557 wait complete. 2024-09-13T17:59:20.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.894+0000 7ff2da518640 1 Processor -- start 2024-09-13T17:59:20.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.894+0000 7ff2da518640 1 -- start start 2024-09-13T17:59:20.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2da518640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2c0005180 0x7ff2d415cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2da518640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d40c8d10 0x7ff2d415a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2da518640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2d41f26b0 0x7ff2d40b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2da518640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d415ddb0 con 0x7ff2d41f26b0 2024-09-13T17:59:20.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2da518640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d415d130 con 0x7ff2d40c8d10 2024-09-13T17:59:20.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2da518640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d415d410 con 0x7ff2c0005180 2024-09-13T17:59:20.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2d8ab5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2d41f26b0 0x7ff2d40b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2d8ab5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2d41f26b0 0x7ff2d40b2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55516/0 (socket says 172.21.15.111:55516) 2024-09-13T17:59:20.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2d8ab5640 1 -- 172.21.15.111:0/10176978 learned_addr learned my addr 172.21.15.111:0/10176978 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:20.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2d37fe640 1 --2- 172.21.15.111:0/10176978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d40c8d10 0x7ff2d415a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2d3fff640 1 --2- 172.21.15.111:0/10176978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2c0005180 0x7ff2d415cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2d8ab5640 1 -- 172.21.15.111:0/10176978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2c0005180 msgr2=0x7ff2d415cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2d8ab5640 1 --2- 172.21.15.111:0/10176978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2c0005180 0x7ff2d415cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2d8ab5640 1 -- 172.21.15.111:0/10176978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d40c8d10 msgr2=0x7ff2d415a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2d8ab5640 1 --2- 172.21.15.111:0/10176978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d40c8d10 0x7ff2d415a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2d8ab5640 1 -- 172.21.15.111:0/10176978 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d415d6f0 con 0x7ff2d41f26b0 2024-09-13T17:59:20.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2d37fe640 1 --2- 172.21.15.111:0/10176978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d40c8d10 0x7ff2d415a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:20.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2d8ab5640 1 --2- 172.21.15.111:0/10176978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2d41f26b0 0x7ff2d40b2910 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00e230 tx=0x7ff2cc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.895+0000 7ff2d0ff9640 1 -- 172.21.15.111:0/10176978 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc03c470 con 0x7ff2d41f26b0 2024-09-13T17:59:20.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.896+0000 7ff2d0ff9640 1 -- 172.21.15.111:0/10176978 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc03c5f0 con 0x7ff2d41f26b0 2024-09-13T17:59:20.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.896+0000 7ff2da518640 1 -- 172.21.15.111:0/10176978 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d415d9d0 con 0x7ff2d41f26b0 2024-09-13T17:59:20.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.896+0000 7ff2d0ff9640 1 -- 172.21.15.111:0/10176978 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc03c9a0 con 0x7ff2d41f26b0 2024-09-13T17:59:20.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.896+0000 7ff2da518640 1 -- 172.21.15.111:0/10176978 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d40c9190 con 0x7ff2d41f26b0 2024-09-13T17:59:20.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.897+0000 7ff2da518640 1 -- 172.21.15.111:0/10176978 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff2d40d12b0 con 0x7ff2d41f26b0 2024-09-13T17:59:20.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.897+0000 7ff2d0ff9640 1 -- 172.21.15.111:0/10176978 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff2cc03cb00 con 0x7ff2d41f26b0 2024-09-13T17:59:20.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.897+0000 7ff2d0ff9640 1 --2- 172.21.15.111:0/10176978 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff2b406ac30 0x7ff2b406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.897+0000 7ff2d0ff9640 1 -- 172.21.15.111:0/10176978 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7ff2cc0a9e30 con 0x7ff2d41f26b0 2024-09-13T17:59:20.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.897+0000 7ff2d3fff640 1 --2- 172.21.15.111:0/10176978 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff2b406ac30 0x7ff2b406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.897+0000 7ff2d0ff9640 1 --2- 172.21.15.111:0/10176978 >> [v2:172.21.15.111:6800/2426719794,v1:172.21.15.111:6801/2426719794] conn(0x7ff2b4070fd0 0x7ff2b4073410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:20.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.897+0000 7ff2d0ff9640 1 -- 172.21.15.111:0/10176978 --> [v2:172.21.15.111:6800/2426719794,v1:172.21.15.111:6801/2426719794] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff2b4073ae0 con 0x7ff2b4070fd0 2024-09-13T17:59:20.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.897+0000 7ff2d0ff9640 1 -- 172.21.15.111:0/10176978 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2cc0ac4f0 con 0x7ff2d41f26b0 2024-09-13T17:59:20.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.898+0000 7ff2d37fe640 1 --2- 172.21.15.111:0/10176978 >> [v2:172.21.15.111:6800/2426719794,v1:172.21.15.111:6801/2426719794] conn(0x7ff2b4070fd0 0x7ff2b4073410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:20.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.898+0000 7ff2d3fff640 1 --2- 172.21.15.111:0/10176978 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff2b406ac30 0x7ff2b406d0f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4000c00 tx=0x7ff2c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.898+0000 7ff2d37fe640 1 --2- 172.21.15.111:0/10176978 >> [v2:172.21.15.111:6800/2426719794,v1:172.21.15.111:6801/2426719794] conn(0x7ff2b4070fd0 0x7ff2b4073410 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:20.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.899+0000 7ff2d0ff9640 1 -- 172.21.15.111:0/10176978 <== osd.2 v2:172.21.15.111:6800/2426719794 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff2b4073ae0 con 0x7ff2b4070fd0 2024-09-13T17:59:20.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.911+0000 7ff2da518640 1 -- 172.21.15.111:0/10176978 --> [v2:172.21.15.111:6800/2426719794,v1:172.21.15.111:6801/2426719794] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff2d4071b50 con 0x7ff2b4070fd0 2024-09-13T17:59:20.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.911+0000 7ff2d0ff9640 1 -- 172.21.15.111:0/10176978 <== osd.2 v2:172.21.15.111:6800/2426719794 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff2d4071b50 con 0x7ff2b4070fd0 2024-09-13T17:59:20.913 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T17:59:20.913 INFO:teuthology.orchestra.run.smithi111.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:59:20.913 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T17:59:20.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.911+0000 7ff2da518640 1 -- 172.21.15.111:0/10176978 >> [v2:172.21.15.111:6800/2426719794,v1:172.21.15.111:6801/2426719794] conn(0x7ff2b4070fd0 msgr2=0x7ff2b4073410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.911+0000 7ff2da518640 1 --2- 172.21.15.111:0/10176978 >> [v2:172.21.15.111:6800/2426719794,v1:172.21.15.111:6801/2426719794] conn(0x7ff2b4070fd0 0x7ff2b4073410 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:20.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.911+0000 7ff2da518640 1 -- 172.21.15.111:0/10176978 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff2b406ac30 msgr2=0x7ff2b406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.911+0000 7ff2da518640 1 --2- 172.21.15.111:0/10176978 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7ff2b406ac30 0x7ff2b406d0f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4000c00 tx=0x7ff2c4009040 comp rx=0 tx=0).stop 2024-09-13T17:59:20.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.911+0000 7ff2da518640 1 -- 172.21.15.111:0/10176978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2d41f26b0 msgr2=0x7ff2d40b2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:20.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.911+0000 7ff2da518640 1 --2- 172.21.15.111:0/10176978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2d41f26b0 0x7ff2d40b2910 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00e230 tx=0x7ff2cc009750 comp rx=0 tx=0).stop 2024-09-13T17:59:20.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.911+0000 7ff2d8ab5640 1 -- 172.21.15.111:0/10176978 reap_dead start 2024-09-13T17:59:20.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.911+0000 7ff2da518640 1 -- 172.21.15.111:0/10176978 shutdown_connections 2024-09-13T17:59:20.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.911+0000 7ff2da518640 1 -- 172.21.15.111:0/10176978 >> 172.21.15.111:0/10176978 conn(0x7ff2d4089f60 msgr2=0x7ff2d40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:20.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.911+0000 7ff2da518640 1 -- 172.21.15.111:0/10176978 shutdown_connections 2024-09-13T17:59:20.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:20.911+0000 7ff2da518640 1 -- 172.21.15.111:0/10176978 wait complete. 2024-09-13T17:59:20.923 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T17:59:21.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.021+0000 7f6da49ce640 1 Processor -- start 2024-09-13T17:59:21.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.021+0000 7f6da49ce640 1 -- start start 2024-09-13T17:59:21.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.022+0000 7f6da49ce640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6da015c500 0x7f6da00d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.022+0000 7f6da49ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6da00d2590 0x7f6da00cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.022+0000 7f6da49ce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6da00cde20 0x7f6da00d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.022+0000 7f6da49ce640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da00cd450 con 0x7f6da015c500 2024-09-13T17:59:21.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.022+0000 7f6da49ce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da00d29f0 con 0x7f6da00cde20 2024-09-13T17:59:21.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.022+0000 7f6da49ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da00d0cf0 con 0x7f6da00d2590 2024-09-13T17:59:21.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.022+0000 7f6d9e59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6da015c500 0x7f6da00d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.022+0000 7f6d9e59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6da015c500 0x7f6da00d2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55520/0 (socket says 172.21.15.111:55520) 2024-09-13T17:59:21.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.022+0000 7f6d9e59c640 1 -- 172.21.15.111:0/1781850351 learned_addr learned my addr 172.21.15.111:0/1781850351 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:21.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.022+0000 7f6d9e59c640 1 -- 172.21.15.111:0/1781850351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6da00d2590 msgr2=0x7f6da00cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:21.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.022+0000 7f6d9e59c640 1 --2- 172.21.15.111:0/1781850351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6da00d2590 0x7f6da00cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.022+0000 7f6d9e59c640 1 -- 172.21.15.111:0/1781850351 >> v2:172.21.15.188:3300/0 conn(0x7f6da00cde20 msgr2=0x7f6da00d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:21.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.022+0000 7f6d9e59c640 1 --2- 172.21.15.111:0/1781850351 >> v2:172.21.15.188:3300/0 conn(0x7f6da00cde20 0x7f6da00d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.022+0000 7f6d9e59c640 1 -- 172.21.15.111:0/1781850351 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da00d0fd0 con 0x7f6da015c500 2024-09-13T17:59:21.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.023+0000 7f6d9e59c640 1 --2- 172.21.15.111:0/1781850351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da015c500 0x7f6da00d2050 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6da00b1e40 tx=0x7f6d9002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162ed2175f0b19b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:21.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.023+0000 7f6d9d59a640 1 -- 172.21.15.111:0/1781850351 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90032440 con 0x7f6da015c500 2024-09-13T17:59:21.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.023+0000 7f6d9d59a640 1 -- 172.21.15.111:0/1781850351 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d900325a0 con 0x7f6da015c500 2024-09-13T17:59:21.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.023+0000 7f6d9d59a640 1 -- 172.21.15.111:0/1781850351 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90032930 con 0x7f6da015c500 2024-09-13T17:59:21.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.024+0000 7f6da49ce640 1 -- 172.21.15.111:0/1781850351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da015c500 msgr2=0x7f6da00d2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.024+0000 7f6da49ce640 1 --2- 172.21.15.111:0/1781850351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da015c500 0x7f6da00d2050 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6da00b1e40 tx=0x7f6d9002a1e0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.024+0000 7f6da49ce640 1 -- 172.21.15.111:0/1781850351 shutdown_connections 2024-09-13T17:59:21.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.024+0000 7f6da49ce640 1 --2- 172.21.15.111:0/1781850351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6da00d2590 0x7f6da00cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.024+0000 7f6da49ce640 1 --2- 172.21.15.111:0/1781850351 >> v2:172.21.15.188:3300/0 conn(0x7f6da00cde20 0x7f6da00d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.024+0000 7f6da49ce640 1 --2- 172.21.15.111:0/1781850351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da015c500 0x7f6da00d2050 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.024+0000 7f6da49ce640 1 -- 172.21.15.111:0/1781850351 >> 172.21.15.111:0/1781850351 conn(0x7f6da0089f60 msgr2=0x7f6da009b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:21.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.024+0000 7f6da49ce640 1 -- 172.21.15.111:0/1781850351 shutdown_connections 2024-09-13T17:59:21.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.024+0000 7f6da49ce640 1 -- 172.21.15.111:0/1781850351 wait complete. 2024-09-13T17:59:21.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.025+0000 7f6da49ce640 1 Processor -- start 2024-09-13T17:59:21.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.025+0000 7f6da49ce640 1 -- start start 2024-09-13T17:59:21.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.026+0000 7f6da49ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da00cde20 0x7f6da014ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.026+0000 7f6da49ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6da00d2590 0x7f6da00cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.026+0000 7f6da49ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6da015c500 0x7f6da00cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.026+0000 7f6da49ce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da00cae30 con 0x7f6da00cde20 2024-09-13T17:59:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.026+0000 7f6da49ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da00c9f10 con 0x7f6da00d2590 2024-09-13T17:59:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.026+0000 7f6da49ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da00ca1b0 con 0x7f6da015c500 2024-09-13T17:59:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.026+0000 7f6d9e59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da00cde20 0x7f6da014ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.026+0000 7f6d9e59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da00cde20 0x7f6da014ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55528/0 (socket says 172.21.15.111:55528) 2024-09-13T17:59:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.026+0000 7f6d9e59c640 1 -- 172.21.15.111:0/2178911699 learned_addr learned my addr 172.21.15.111:0/2178911699 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.026+0000 7f6d9e59c640 1 -- 172.21.15.111:0/2178911699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6da015c500 msgr2=0x7f6da00cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.026+0000 7f6d9e59c640 1 --2- 172.21.15.111:0/2178911699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6da015c500 0x7f6da00cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.026+0000 7f6d9e59c640 1 -- 172.21.15.111:0/2178911699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6da00d2590 msgr2=0x7f6da00cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.026+0000 7f6d9e59c640 1 --2- 172.21.15.111:0/2178911699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6da00d2590 0x7f6da00cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.026+0000 7f6d9e59c640 1 -- 172.21.15.111:0/2178911699 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d900094d0 con 0x7f6da00cde20 2024-09-13T17:59:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.027+0000 7f6d9e59c640 1 --2- 172.21.15.111:0/2178911699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da00cde20 0x7f6da014ec10 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6d90000c00 tx=0x7f6d900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.027+0000 7f6d96ffd640 1 -- 172.21.15.111:0/2178911699 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90032440 con 0x7f6da00cde20 2024-09-13T17:59:21.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.027+0000 7f6d96ffd640 1 -- 172.21.15.111:0/2178911699 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d90030b80 con 0x7f6da00cde20 2024-09-13T17:59:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.027+0000 7f6da49ce640 1 -- 172.21.15.111:0/2178911699 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da00ca490 con 0x7f6da00cde20 2024-09-13T17:59:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.027+0000 7f6d96ffd640 1 -- 172.21.15.111:0/2178911699 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9003b400 con 0x7f6da00cde20 2024-09-13T17:59:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.027+0000 7f6da49ce640 1 -- 172.21.15.111:0/2178911699 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da00caa50 con 0x7f6da00cde20 2024-09-13T17:59:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.028+0000 7f6da49ce640 1 -- 172.21.15.111:0/2178911699 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da00caf70 con 0x7f6da00cde20 2024-09-13T17:59:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.028+0000 7f6d96ffd640 1 -- 172.21.15.111:0/2178911699 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6d90039040 con 0x7f6da00cde20 2024-09-13T17:59:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.029+0000 7f6d96ffd640 1 --2- 172.21.15.111:0/2178911699 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6d8006ac00 0x7f6d8006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.029+0000 7f6d9dd9b640 1 --2- 172.21.15.111:0/2178911699 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6d8006ac00 0x7f6d8006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.029+0000 7f6d96ffd640 1 -- 172.21.15.111:0/2178911699 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f6d900a9a50 con 0x7f6da00cde20 2024-09-13T17:59:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.029+0000 7f6d9dd9b640 1 --2- 172.21.15.111:0/2178911699 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6d8006ac00 0x7f6d8006d0c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6da00cfdf0 tx=0x7f6d88006c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:21.035 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.033+0000 7f6d96ffd640 1 -- 172.21.15.111:0/2178911699 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f6d90074e20 con 0x7f6da00cde20 2024-09-13T17:59:21.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.208+0000 7f6da49ce640 1 -- 172.21.15.111:0/2178911699 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6da00ce2b0 con 0x7f6d8006ac00 2024-09-13T17:59:21.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.209+0000 7f6d96ffd640 1 -- 172.21.15.111:0/2178911699 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6d88013400 con 0x7f6d8006ac00 2024-09-13T17:59:21.211 INFO:teuthology.orchestra.run.smithi111.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T17:59:21.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.212+0000 7f6da49ce640 1 -- 172.21.15.111:0/2178911699 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6d8006ac00 msgr2=0x7f6d8006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.212+0000 7f6da49ce640 1 --2- 172.21.15.111:0/2178911699 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6d8006ac00 0x7f6d8006d0c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6da00cfdf0 tx=0x7f6d88006c90 comp rx=0 tx=0).stop 2024-09-13T17:59:21.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.212+0000 7f6da49ce640 1 -- 172.21.15.111:0/2178911699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da00cde20 msgr2=0x7f6da014ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.213+0000 7f6da49ce640 1 --2- 172.21.15.111:0/2178911699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da00cde20 0x7f6da014ec10 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6d90000c00 tx=0x7f6d900046c0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.213+0000 7f6da49ce640 1 -- 172.21.15.111:0/2178911699 shutdown_connections 2024-09-13T17:59:21.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.213+0000 7f6da49ce640 1 --2- 172.21.15.111:0/2178911699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6da015c500 0x7f6da00cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.213+0000 7f6da49ce640 1 --2- 172.21.15.111:0/2178911699 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f6d8006ac00 0x7f6d8006d0c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.213+0000 7f6da49ce640 1 --2- 172.21.15.111:0/2178911699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6da00d2590 0x7f6da00cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.213+0000 7f6da49ce640 1 --2- 172.21.15.111:0/2178911699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da00cde20 0x7f6da014ec10 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.213+0000 7f6da49ce640 1 -- 172.21.15.111:0/2178911699 >> 172.21.15.111:0/2178911699 conn(0x7f6da0089f60 msgr2=0x7f6da006c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:21.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.213+0000 7f6da49ce640 1 -- 172.21.15.111:0/2178911699 shutdown_connections 2024-09-13T17:59:21.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.213+0000 7f6da49ce640 1 -- 172.21.15.111:0/2178911699 wait complete. 2024-09-13T17:59:21.225 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T17:59:21.226 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:59:21.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.326+0000 7f5a0e29c640 1 Processor -- start 2024-09-13T17:59:21.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.326+0000 7f5a0e29c640 1 -- start start 2024-09-13T17:59:21.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.326+0000 7f5a0e29c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5a0815c500 0x7f5a080d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.326+0000 7f5a0e29c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5a080d2590 0x7f5a080cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.326+0000 7f5a0e29c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a080cd450 con 0x7f5a080cde20 2024-09-13T17:59:21.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.326+0000 7f5a0e29c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a080d29f0 con 0x7f5a0815c500 2024-09-13T17:59:21.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.326+0000 7f5a0e29c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a080d0cf0 con 0x7f5a080d2590 2024-09-13T17:59:21.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.327+0000 7f5a07fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5a0815c500 0x7f5a080d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.327+0000 7f5a07fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5a0815c500 0x7f5a080d2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55538/0 (socket says 172.21.15.111:55538) 2024-09-13T17:59:21.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.327+0000 7f5a07fff640 1 -- 172.21.15.111:0/3594771139 learned_addr learned my addr 172.21.15.111:0/3594771139 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:21.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.327+0000 7f5a07fff640 1 -- 172.21.15.111:0/3594771139 >> v2:172.21.15.188:3300/0 conn(0x7f5a080d2590 msgr2=0x7f5a080cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:21.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.327+0000 7f5a07fff640 1 --2- 172.21.15.111:0/3594771139 >> v2:172.21.15.188:3300/0 conn(0x7f5a080d2590 0x7f5a080cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.327+0000 7f5a07fff640 1 -- 172.21.15.111:0/3594771139 >> v1:172.21.15.188:6789/0 conn(0x7f5a080cde20 legacy=0x7f5a080d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:21.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.327+0000 7f5a07fff640 1 -- 172.21.15.111:0/3594771139 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a080d0fd0 con 0x7f5a0815c500 2024-09-13T17:59:21.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.328+0000 7f5a07fff640 1 --2- 172.21.15.111:0/3594771139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a0815c500 0x7f5a080d2050 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5a0806ce50 tx=0x7f59f802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60911d1fabdb34e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:21.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.328+0000 7f5a077fe640 1 -- 172.21.15.111:0/3594771139 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8032440 con 0x7f5a0815c500 2024-09-13T17:59:21.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.328+0000 7f5a077fe640 1 -- 172.21.15.111:0/3594771139 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f80325a0 con 0x7f5a0815c500 2024-09-13T17:59:21.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.328+0000 7f5a077fe640 1 -- 172.21.15.111:0/3594771139 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8032930 con 0x7f5a0815c500 2024-09-13T17:59:21.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.328+0000 7f5a0e29c640 1 -- 172.21.15.111:0/3594771139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a0815c500 msgr2=0x7f5a080d2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.328+0000 7f5a0e29c640 1 --2- 172.21.15.111:0/3594771139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a0815c500 0x7f5a080d2050 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5a0806ce50 tx=0x7f59f802a230 comp rx=0 tx=0).stop 2024-09-13T17:59:21.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.328+0000 7f5a0e29c640 1 -- 172.21.15.111:0/3594771139 shutdown_connections 2024-09-13T17:59:21.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.328+0000 7f5a0e29c640 1 --2- 172.21.15.111:0/3594771139 >> v2:172.21.15.188:3300/0 conn(0x7f5a080d2590 0x7f5a080cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.328+0000 7f5a0e29c640 1 --2- 172.21.15.111:0/3594771139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a0815c500 0x7f5a080d2050 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.328+0000 7f5a0e29c640 1 -- 172.21.15.111:0/3594771139 >> 172.21.15.111:0/3594771139 conn(0x7f5a08089f60 msgr2=0x7f5a0809b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:21.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.328+0000 7f5a0e29c640 1 -- 172.21.15.111:0/3594771139 shutdown_connections 2024-09-13T17:59:21.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.328+0000 7f5a0e29c640 1 -- 172.21.15.111:0/3594771139 wait complete. 2024-09-13T17:59:21.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.330+0000 7f5a0e29c640 1 Processor -- start 2024-09-13T17:59:21.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.330+0000 7f5a0e29c640 1 -- start start 2024-09-13T17:59:21.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.330+0000 7f5a0e29c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a080cde20 0x7f5a0815d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.330+0000 7f5a0e29c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a080d2590 0x7f5a0815f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.330+0000 7f5a0e29c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a0815c500 0x7f5a080c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.330+0000 7f5a0e29c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0815fa80 con 0x7f5a080cde20 2024-09-13T17:59:21.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.330+0000 7f5a0e29c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a080cad60 con 0x7f5a0815c500 2024-09-13T17:59:21.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.330+0000 7f5a0e29c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a080caa80 con 0x7f5a080d2590 2024-09-13T17:59:21.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.330+0000 7f5a07fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a080cde20 0x7f5a0815d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.330+0000 7f5a07fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a080cde20 0x7f5a0815d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55542/0 (socket says 172.21.15.111:55542) 2024-09-13T17:59:21.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.330+0000 7f5a07fff640 1 -- 172.21.15.111:0/3605797458 learned_addr learned my addr 172.21.15.111:0/3605797458 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:21.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.331+0000 7f5a07fff640 1 -- 172.21.15.111:0/3605797458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a080d2590 msgr2=0x7f5a0815f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.331+0000 7f5a07fff640 1 --2- 172.21.15.111:0/3605797458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a080d2590 0x7f5a0815f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.331+0000 7f5a07fff640 1 -- 172.21.15.111:0/3605797458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a0815c500 msgr2=0x7f5a080c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.331+0000 7f5a07fff640 1 --2- 172.21.15.111:0/3605797458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a0815c500 0x7f5a080c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.331+0000 7f5a07fff640 1 -- 172.21.15.111:0/3605797458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f80094d0 con 0x7f5a080cde20 2024-09-13T17:59:21.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.331+0000 7f5a07fff640 1 --2- 172.21.15.111:0/3605797458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a080cde20 0x7f5a0815d6c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f59f8000c00 tx=0x7f59f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:21.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.331+0000 7f5a057fa640 1 -- 172.21.15.111:0/3605797458 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8032440 con 0x7f5a080cde20 2024-09-13T17:59:21.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.331+0000 7f5a057fa640 1 -- 172.21.15.111:0/3605797458 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f8030b80 con 0x7f5a080cde20 2024-09-13T17:59:21.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.331+0000 7f5a0e29c640 1 -- 172.21.15.111:0/3605797458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0815cbe0 con 0x7f5a080cde20 2024-09-13T17:59:21.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.331+0000 7f5a057fa640 1 -- 172.21.15.111:0/3605797458 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f803b400 con 0x7f5a080cde20 2024-09-13T17:59:21.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.331+0000 7f5a0e29c640 1 -- 172.21.15.111:0/3605797458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0815d1a0 con 0x7f5a080cde20 2024-09-13T17:59:21.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.332+0000 7f5a0e29c640 1 -- 172.21.15.111:0/3605797458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5a080d12b0 con 0x7f5a080cde20 2024-09-13T17:59:21.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.332+0000 7f5a057fa640 1 -- 172.21.15.111:0/3605797458 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f59f8039040 con 0x7f5a080cde20 2024-09-13T17:59:21.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.333+0000 7f5a057fa640 1 --2- 172.21.15.111:0/3605797458 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f59e806ac00 0x7f59e806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.333+0000 7f5a057fa640 1 -- 172.21.15.111:0/3605797458 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f59f80a9b10 con 0x7f5a080cde20 2024-09-13T17:59:21.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.333+0000 7f59fffff640 1 --2- 172.21.15.111:0/3605797458 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f59e806ac00 0x7f59e806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.333+0000 7f5a057fa640 1 --2- 172.21.15.111:0/3605797458 >> [v2:172.21.15.111:6824/3696423188,v1:172.21.15.111:6825/3696423188] conn(0x7f59e8070fa0 0x7f59e80733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.333+0000 7f5a057fa640 1 -- 172.21.15.111:0/3605797458 --> [v2:172.21.15.111:6824/3696423188,v1:172.21.15.111:6825/3696423188] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59e8073ab0 con 0x7f59e8070fa0 2024-09-13T17:59:21.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.333+0000 7f5a057fa640 1 -- 172.21.15.111:0/3605797458 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59f80ae2c0 con 0x7f5a080cde20 2024-09-13T17:59:21.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.333+0000 7f5a0c839640 1 --2- 172.21.15.111:0/3605797458 >> [v2:172.21.15.111:6824/3696423188,v1:172.21.15.111:6825/3696423188] conn(0x7f59e8070fa0 0x7f59e80733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.333+0000 7f59fffff640 1 --2- 172.21.15.111:0/3605797458 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f59e806ac00 0x7f59e806d0c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5a080cfd00 tx=0x7f59f0006c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:21.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.333+0000 7f5a0c839640 1 --2- 172.21.15.111:0/3605797458 >> [v2:172.21.15.111:6824/3696423188,v1:172.21.15.111:6825/3696423188] conn(0x7f59e8070fa0 0x7f59e80733e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:21.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.334+0000 7f5a057fa640 1 -- 172.21.15.111:0/3605797458 <== osd.3 v2:172.21.15.111:6824/3696423188 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f59e8073ab0 con 0x7f59e8070fa0 2024-09-13T17:59:21.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.346+0000 7f5a0e29c640 1 -- 172.21.15.111:0/3605797458 --> [v2:172.21.15.111:6824/3696423188,v1:172.21.15.111:6825/3696423188] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5a080ce220 con 0x7f59e8070fa0 2024-09-13T17:59:21.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.346+0000 7f5a057fa640 1 -- 172.21.15.111:0/3605797458 <== osd.3 v2:172.21.15.111:6824/3696423188 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5a00006b10 con 0x7f59e8070fa0 2024-09-13T17:59:21.348 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T17:59:21.348 INFO:teuthology.orchestra.run.smithi111.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:59:21.348 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T17:59:21.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.346+0000 7f5a0e29c640 1 -- 172.21.15.111:0/3605797458 >> [v2:172.21.15.111:6824/3696423188,v1:172.21.15.111:6825/3696423188] conn(0x7f59e8070fa0 msgr2=0x7f59e80733e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.346+0000 7f5a0e29c640 1 --2- 172.21.15.111:0/3605797458 >> [v2:172.21.15.111:6824/3696423188,v1:172.21.15.111:6825/3696423188] conn(0x7f59e8070fa0 0x7f59e80733e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.346+0000 7f5a0e29c640 1 -- 172.21.15.111:0/3605797458 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f59e806ac00 msgr2=0x7f59e806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.346+0000 7f5a0e29c640 1 --2- 172.21.15.111:0/3605797458 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f59e806ac00 0x7f59e806d0c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5a080cfd00 tx=0x7f59f0006c90 comp rx=0 tx=0).stop 2024-09-13T17:59:21.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.346+0000 7f5a0e29c640 1 -- 172.21.15.111:0/3605797458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a080cde20 msgr2=0x7f5a0815d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.346+0000 7f5a0e29c640 1 --2- 172.21.15.111:0/3605797458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a080cde20 0x7f5a0815d6c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f59f8000c00 tx=0x7f59f80046c0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.346+0000 7f5a0c839640 1 -- 172.21.15.111:0/3605797458 reap_dead start 2024-09-13T17:59:21.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.347+0000 7f5a0e29c640 1 -- 172.21.15.111:0/3605797458 shutdown_connections 2024-09-13T17:59:21.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.347+0000 7f5a0e29c640 1 -- 172.21.15.111:0/3605797458 >> 172.21.15.111:0/3605797458 conn(0x7f5a08089f60 msgr2=0x7f5a080cffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:21.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.347+0000 7f5a0e29c640 1 -- 172.21.15.111:0/3605797458 shutdown_connections 2024-09-13T17:59:21.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.347+0000 7f5a0e29c640 1 -- 172.21.15.111:0/3605797458 wait complete. 2024-09-13T17:59:21.358 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T17:59:21.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.456+0000 7f9b41cb1640 1 Processor -- start 2024-09-13T17:59:21.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.456+0000 7f9b41cb1640 1 -- start start 2024-09-13T17:59:21.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.457+0000 7f9b41cb1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9b3c0d2760 0x7f9b3c0cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.457+0000 7f9b41cb1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9b3c0cde20 0x7f9b3c0d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.457+0000 7f9b41cb1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b3c0cd450 con 0x7f9b3c15c500 2024-09-13T17:59:21.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.457+0000 7f9b41cb1640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3c0d0cf0 con 0x7f9b3c0d2760 2024-09-13T17:59:21.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.457+0000 7f9b41cb1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3c0d0fd0 con 0x7f9b3c0cde20 2024-09-13T17:59:21.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.457+0000 7f9b33fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9b3c0d2760 0x7f9b3c0cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.457+0000 7f9b33fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9b3c0d2760 0x7f9b3c0cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55558/0 (socket says 172.21.15.111:55558) 2024-09-13T17:59:21.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.457+0000 7f9b33fff640 1 -- 172.21.15.111:0/301150475 learned_addr learned my addr 172.21.15.111:0/301150475 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:21.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.457+0000 7f9b3bfff640 1 --2- 172.21.15.111:0/301150475 >> v2:172.21.15.188:3300/0 conn(0x7f9b3c0cde20 0x7f9b3c0d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.457+0000 7f9b33fff640 1 -- 172.21.15.111:0/301150475 >> v2:172.21.15.188:3300/0 conn(0x7f9b3c0cde20 msgr2=0x7f9b3c0d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:21.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.457+0000 7f9b33fff640 1 --2- 172.21.15.111:0/301150475 >> v2:172.21.15.188:3300/0 conn(0x7f9b3c0cde20 0x7f9b3c0d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.457+0000 7f9b33fff640 1 -- 172.21.15.111:0/301150475 >> v1:172.21.15.188:6789/0 conn(0x7f9b3c15c500 legacy=0x7f9b3c0d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.457+0000 7f9b33fff640 1 -- 172.21.15.111:0/301150475 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c0cd0e0 con 0x7f9b3c0d2760 2024-09-13T17:59:21.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.458+0000 7f9b33fff640 1 --2- 172.21.15.111:0/301150475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b3c0d2760 0x7f9b3c0cd8e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c0cca00 tx=0x7f9b2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434f9af0314ce504 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:21.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.458+0000 7f9b3affd640 1 -- 172.21.15.111:0/301150475 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b24032440 con 0x7f9b3c0d2760 2024-09-13T17:59:21.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.458+0000 7f9b3affd640 1 -- 172.21.15.111:0/301150475 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b240325a0 con 0x7f9b3c0d2760 2024-09-13T17:59:21.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.458+0000 7f9b3affd640 1 -- 172.21.15.111:0/301150475 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b24032930 con 0x7f9b3c0d2760 2024-09-13T17:59:21.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.458+0000 7f9b41cb1640 1 -- 172.21.15.111:0/301150475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b3c0d2760 msgr2=0x7f9b3c0cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.458+0000 7f9b41cb1640 1 --2- 172.21.15.111:0/301150475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b3c0d2760 0x7f9b3c0cd8e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c0cca00 tx=0x7f9b2402a270 comp rx=0 tx=0).stop 2024-09-13T17:59:21.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.458+0000 7f9b41cb1640 1 -- 172.21.15.111:0/301150475 shutdown_connections 2024-09-13T17:59:21.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.459+0000 7f9b41cb1640 1 --2- 172.21.15.111:0/301150475 >> v2:172.21.15.188:3300/0 conn(0x7f9b3c0cde20 0x7f9b3c0d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.459+0000 7f9b41cb1640 1 --2- 172.21.15.111:0/301150475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b3c0d2760 0x7f9b3c0cd8e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.459+0000 7f9b41cb1640 1 -- 172.21.15.111:0/301150475 >> 172.21.15.111:0/301150475 conn(0x7f9b3c089f60 msgr2=0x7f9b3c09b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:21.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.459+0000 7f9b41cb1640 1 -- 172.21.15.111:0/301150475 shutdown_connections 2024-09-13T17:59:21.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.459+0000 7f9b41cb1640 1 -- 172.21.15.111:0/301150475 wait complete. 2024-09-13T17:59:21.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.460+0000 7f9b41cb1640 1 Processor -- start 2024-09-13T17:59:21.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.460+0000 7f9b41cb1640 1 -- start start 2024-09-13T17:59:21.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.460+0000 7f9b41cb1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b3c0cde20 0x7f9b3c14f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.460+0000 7f9b41cb1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b3c0d2760 0x7f9b3c0cc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.460+0000 7f9b41cb1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b3c15c500 0x7f9b3c0aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.460+0000 7f9b41cb1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c14fba0 con 0x7f9b3c15c500 2024-09-13T17:59:21.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.460+0000 7f9b41cb1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0cc590 con 0x7f9b3c0cde20 2024-09-13T17:59:21.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.460+0000 7f9b41cb1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c15cc80 con 0x7f9b3c0d2760 2024-09-13T17:59:21.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.460+0000 7f9b3bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b3c15c500 0x7f9b3c0aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.461+0000 7f9b3bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b3c15c500 0x7f9b3c0aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55568/0 (socket says 172.21.15.111:55568) 2024-09-13T17:59:21.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.461+0000 7f9b3bfff640 1 -- 172.21.15.111:0/3618807264 learned_addr learned my addr 172.21.15.111:0/3618807264 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:21.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.461+0000 7f9b33fff640 1 --2- 172.21.15.111:0/3618807264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b3c0d2760 0x7f9b3c0cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.461+0000 7f9b3b7fe640 1 --2- 172.21.15.111:0/3618807264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b3c0cde20 0x7f9b3c14f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.461+0000 7f9b3bfff640 1 -- 172.21.15.111:0/3618807264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b3c0d2760 msgr2=0x7f9b3c0cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.461+0000 7f9b3bfff640 1 --2- 172.21.15.111:0/3618807264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b3c0d2760 0x7f9b3c0cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.461+0000 7f9b3bfff640 1 -- 172.21.15.111:0/3618807264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b3c0cde20 msgr2=0x7f9b3c14f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.461+0000 7f9b3bfff640 1 --2- 172.21.15.111:0/3618807264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b3c0cde20 0x7f9b3c14f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.461+0000 7f9b3bfff640 1 -- 172.21.15.111:0/3618807264 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b240094d0 con 0x7f9b3c15c500 2024-09-13T17:59:21.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.461+0000 7f9b3bfff640 1 --2- 172.21.15.111:0/3618807264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b3c15c500 0x7f9b3c0aef80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9b34007e40 tx=0x7f9b3400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:21.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.461+0000 7f9b38ff9640 1 -- 172.21.15.111:0/3618807264 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34014620 con 0x7f9b3c15c500 2024-09-13T17:59:21.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.461+0000 7f9b38ff9640 1 -- 172.21.15.111:0/3618807264 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3400ee90 con 0x7f9b3c15c500 2024-09-13T17:59:21.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.461+0000 7f9b41cb1640 1 -- 172.21.15.111:0/3618807264 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3c15cf60 con 0x7f9b3c15c500 2024-09-13T17:59:21.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.461+0000 7f9b38ff9640 1 -- 172.21.15.111:0/3618807264 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b340149e0 con 0x7f9b3c15c500 2024-09-13T17:59:21.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.461+0000 7f9b41cb1640 1 -- 172.21.15.111:0/3618807264 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3c15d520 con 0x7f9b3c15c500 2024-09-13T17:59:21.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.462+0000 7f9b41cb1640 1 -- 172.21.15.111:0/3618807264 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b3c0d08e0 con 0x7f9b3c15c500 2024-09-13T17:59:21.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.462+0000 7f9b38ff9640 1 -- 172.21.15.111:0/3618807264 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9b34014b40 con 0x7f9b3c15c500 2024-09-13T17:59:21.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.463+0000 7f9b38ff9640 1 --2- 172.21.15.111:0/3618807264 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9b1c06ac00 0x7f9b1c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.463+0000 7f9b38ff9640 1 -- 172.21.15.111:0/3618807264 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f9b34089a40 con 0x7f9b3c15c500 2024-09-13T17:59:21.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.463+0000 7f9b3b7fe640 1 --2- 172.21.15.111:0/3618807264 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9b1c06ac00 0x7f9b1c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.463+0000 7f9b3b7fe640 1 --2- 172.21.15.111:0/3618807264 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9b1c06ac00 0x7f9b1c06d0c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c000f60 tx=0x7f9b2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:21.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.467+0000 7f9b38ff9640 1 -- 172.21.15.111:0/3618807264 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f9b3408be60 con 0x7f9b3c15c500 2024-09-13T17:59:21.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.629+0000 7f9b41cb1640 1 -- 172.21.15.111:0/3618807264 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b3c0ce220 con 0x7f9b1c06ac00 2024-09-13T17:59:21.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.629+0000 7f9b38ff9640 1 -- 172.21.15.111:0/3618807264 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9b2c001d10 con 0x7f9b1c06ac00 2024-09-13T17:59:21.631 INFO:teuthology.orchestra.run.smithi111.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T17:59:21.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.631+0000 7f9b41cb1640 1 -- 172.21.15.111:0/3618807264 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9b1c06ac00 msgr2=0x7f9b1c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.631+0000 7f9b41cb1640 1 --2- 172.21.15.111:0/3618807264 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9b1c06ac00 0x7f9b1c06d0c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c000f60 tx=0x7f9b2c009040 comp rx=0 tx=0).stop 2024-09-13T17:59:21.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.631+0000 7f9b41cb1640 1 -- 172.21.15.111:0/3618807264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b3c15c500 msgr2=0x7f9b3c0aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.631+0000 7f9b41cb1640 1 --2- 172.21.15.111:0/3618807264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b3c15c500 0x7f9b3c0aef80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9b34007e40 tx=0x7f9b3400e4b0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.631+0000 7f9b41cb1640 1 -- 172.21.15.111:0/3618807264 shutdown_connections 2024-09-13T17:59:21.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.631+0000 7f9b41cb1640 1 --2- 172.21.15.111:0/3618807264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b3c15c500 0x7f9b3c0aef80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.631+0000 7f9b41cb1640 1 --2- 172.21.15.111:0/3618807264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b3c0d2760 0x7f9b3c0cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.631+0000 7f9b41cb1640 1 --2- 172.21.15.111:0/3618807264 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f9b1c06ac00 0x7f9b1c06d0c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.631+0000 7f9b41cb1640 1 --2- 172.21.15.111:0/3618807264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b3c0cde20 0x7f9b3c14f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.631+0000 7f9b41cb1640 1 -- 172.21.15.111:0/3618807264 >> 172.21.15.111:0/3618807264 conn(0x7f9b3c089f60 msgr2=0x7f9b3c0b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:21.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.631+0000 7f9b41cb1640 1 -- 172.21.15.111:0/3618807264 shutdown_connections 2024-09-13T17:59:21.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.631+0000 7f9b41cb1640 1 -- 172.21.15.111:0/3618807264 wait complete. 2024-09-13T17:59:21.644 INFO:tasks.ceph:Scrubbing osd.4 2024-09-13T17:59:21.644 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:59:21.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.743+0000 7fd8200d7640 1 Processor -- start 2024-09-13T17:59:21.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.743+0000 7fd8200d7640 1 -- start start 2024-09-13T17:59:21.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd8200d7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd8180cec00 0x7fd818164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd8200d7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd81815a790 con 0x7fd8180b20a0 2024-09-13T17:59:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd8200d7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8180b2680 con 0x7fd81815a3b0 2024-09-13T17:59:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd8200d7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8180b2960 con 0x7fd8180cec00 2024-09-13T17:59:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81d672640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd8180b20a0 0x7fd8180ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43106/0 (socket says 172.21.15.111:43106) 2024-09-13T17:59:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81d672640 1 -- 172.21.15.111:0/1224254471 learned_addr learned my addr 172.21.15.111:0/1224254471 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81ce71640 1 -- 172.21.15.111:0/1224254471 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682226239 0 0) 0x7fd81815a790 con 0x7fd8180b20a0 2024-09-13T17:59:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81ce71640 1 -- 172.21.15.111:0/1224254471 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd804003520 con 0x7fd8180b20a0 2024-09-13T17:59:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81e674640 1 --2- 172.21.15.111:0/1224254471 >> v2:172.21.15.188:3300/0 conn(0x7fd8180cec00 0x7fd818164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81ce71640 1 -- 172.21.15.111:0/1224254471 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 751905267 0 0) 0x7fd804003520 con 0x7fd8180b20a0 2024-09-13T17:59:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81ce71640 1 -- 172.21.15.111:0/1224254471 >> v2:172.21.15.188:3300/0 conn(0x7fd8180cec00 msgr2=0x7fd818164ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81ce71640 1 --2- 172.21.15.111:0/1224254471 >> v2:172.21.15.188:3300/0 conn(0x7fd8180cec00 0x7fd818164ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81ce71640 1 -- 172.21.15.111:0/1224254471 >> v1:172.21.15.188:6789/0 conn(0x7fd81815a3b0 legacy=0x7fd8180b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81ce71640 1 -- 172.21.15.111:0/1224254471 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8180b2c40 con 0x7fd8180b20a0 2024-09-13T17:59:21.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81ce71640 1 -- 172.21.15.111:0/1224254471 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7fd808001690 con 0x7fd8180b20a0 2024-09-13T17:59:21.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81ce71640 1 -- 172.21.15.111:0/1224254471 >> v1:172.21.15.111:6789/0 conn(0x7fd8180b20a0 legacy=0x7fd8180ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81ce71640 1 --2- 172.21.15.111:0/1224254471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd804003970 0x7fd804003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81ce71640 1 --2- 172.21.15.111:0/1224254471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8040047c0 0x7fd804004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81ce71640 1 --2- 172.21.15.111:0/1224254471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd804005180 0x7fd80400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81ce71640 1 -- 172.21.15.111:0/1224254471 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8180b2960 con 0x7fd804005180 2024-09-13T17:59:21.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81ce71640 1 -- 172.21.15.111:0/1224254471 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80400c0d0 con 0x7fd804003970 2024-09-13T17:59:21.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.744+0000 7fd81ce71640 1 -- 172.21.15.111:0/1224254471 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd80400c290 con 0x7fd8040047c0 2024-09-13T17:59:21.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.745+0000 7fd81e674640 1 --2- 172.21.15.111:0/1224254471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd804005180 0x7fd80400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.745+0000 7fd81de73640 1 --2- 172.21.15.111:0/1224254471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd804003970 0x7fd804003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.745+0000 7fd81d672640 1 --2- 172.21.15.111:0/1224254471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8040047c0 0x7fd804004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.745+0000 7fd81e674640 1 -- 172.21.15.111:0/1224254471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8040047c0 msgr2=0x7fd804004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:21.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.745+0000 7fd81e674640 1 --2- 172.21.15.111:0/1224254471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8040047c0 0x7fd804004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.745+0000 7fd81e674640 1 -- 172.21.15.111:0/1224254471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd804003970 msgr2=0x7fd804003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:21.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.745+0000 7fd81e674640 1 --2- 172.21.15.111:0/1224254471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd804003970 0x7fd804003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.745+0000 7fd81e674640 1 -- 172.21.15.111:0/1224254471 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd804003520 con 0x7fd804005180 2024-09-13T17:59:21.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.745+0000 7fd81e674640 1 -- 172.21.15.111:0/1224254471 reap_dead start 2024-09-13T17:59:21.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.745+0000 7fd81e674640 1 --2- 172.21.15.111:0/1224254471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd804005180 0x7fd80400b980 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd81000ba10 tx=0x7fd8040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d4070d06e6fae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:21.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.746+0000 7fd81ce71640 1 -- 172.21.15.111:0/1224254471 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81002ae00 con 0x7fd804005180 2024-09-13T17:59:21.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.746+0000 7fd81ce71640 1 -- 172.21.15.111:0/1224254471 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81002e070 con 0x7fd804005180 2024-09-13T17:59:21.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.746+0000 7fd81ce71640 1 -- 172.21.15.111:0/1224254471 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810034640 con 0x7fd804005180 2024-09-13T17:59:21.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.746+0000 7fd8200d7640 1 -- 172.21.15.111:0/1224254471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd804005180 msgr2=0x7fd80400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.746+0000 7fd8200d7640 1 --2- 172.21.15.111:0/1224254471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd804005180 0x7fd80400b980 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd81000ba10 tx=0x7fd8040057f0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.746+0000 7fd8200d7640 1 -- 172.21.15.111:0/1224254471 shutdown_connections 2024-09-13T17:59:21.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.746+0000 7fd8200d7640 1 --2- 172.21.15.111:0/1224254471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd804005180 0x7fd80400b980 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.746+0000 7fd8200d7640 1 -- 172.21.15.111:0/1224254471 >> 172.21.15.111:0/1224254471 conn(0x7fd81806a0b0 msgr2=0x7fd81806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.746+0000 7fd8200d7640 1 -- 172.21.15.111:0/1224254471 shutdown_connections 2024-09-13T17:59:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.746+0000 7fd8200d7640 1 -- 172.21.15.111:0/1224254471 wait complete. 2024-09-13T17:59:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.747+0000 7fd8200d7640 1 Processor -- start 2024-09-13T17:59:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.747+0000 7fd8200d7640 1 -- start start 2024-09-13T17:59:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.748+0000 7fd8200d7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd804005180 0x7fd8180aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.748+0000 7fd8200d7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8181ef920 0x7fd8181555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.748+0000 7fd8200d7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd81815c320 0x7fd81815c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.748+0000 7fd8200d7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81815cce0 con 0x7fd804005180 2024-09-13T17:59:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.748+0000 7fd8200d7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd818155b30 con 0x7fd81815c320 2024-09-13T17:59:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.748+0000 7fd8200d7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8180af300 con 0x7fd8181ef920 2024-09-13T17:59:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.748+0000 7fd81de73640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd804005180 0x7fd8180aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.748+0000 7fd81de73640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd804005180 0x7fd8180aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55586/0 (socket says 172.21.15.111:55586) 2024-09-13T17:59:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.748+0000 7fd81de73640 1 -- 172.21.15.111:0/1871315978 learned_addr learned my addr 172.21.15.111:0/1871315978 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.748+0000 7fd81d672640 1 --2- 172.21.15.111:0/1871315978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8181ef920 0x7fd8181555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.748+0000 7fd81e674640 1 --2- 172.21.15.111:0/1871315978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd81815c320 0x7fd81815c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.749+0000 7fd81de73640 1 -- 172.21.15.111:0/1871315978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8181ef920 msgr2=0x7fd8181555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.749+0000 7fd81de73640 1 --2- 172.21.15.111:0/1871315978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8181ef920 0x7fd8181555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.749+0000 7fd81de73640 1 -- 172.21.15.111:0/1871315978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd81815c320 msgr2=0x7fd81815c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.749+0000 7fd81de73640 1 --2- 172.21.15.111:0/1871315978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd81815c320 0x7fd81815c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.749+0000 7fd81de73640 1 -- 172.21.15.111:0/1871315978 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8180af5e0 con 0x7fd804005180 2024-09-13T17:59:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.749+0000 7fd81d672640 1 --2- 172.21.15.111:0/1871315978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8181ef920 0x7fd8181555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.749+0000 7fd81e674640 1 --2- 172.21.15.111:0/1871315978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd81815c320 0x7fd81815c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.749+0000 7fd81de73640 1 --2- 172.21.15.111:0/1871315978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd804005180 0x7fd8180aed20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd81400f8b0 tx=0x7fd81400fd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.749+0000 7fd80e7fc640 1 -- 172.21.15.111:0/1871315978 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd814006d60 con 0x7fd804005180 2024-09-13T17:59:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.749+0000 7fd80e7fc640 1 -- 172.21.15.111:0/1871315978 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd814003970 con 0x7fd804005180 2024-09-13T17:59:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.749+0000 7fd8200d7640 1 -- 172.21.15.111:0/1871315978 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8180af8c0 con 0x7fd804005180 2024-09-13T17:59:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.749+0000 7fd80e7fc640 1 -- 172.21.15.111:0/1871315978 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd814003d20 con 0x7fd804005180 2024-09-13T17:59:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.749+0000 7fd8200d7640 1 -- 172.21.15.111:0/1871315978 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8180afe80 con 0x7fd804005180 2024-09-13T17:59:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.750+0000 7fd8200d7640 1 -- 172.21.15.111:0/1871315978 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd8180ab4c0 con 0x7fd804005180 2024-09-13T17:59:21.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.750+0000 7fd80e7fc640 1 -- 172.21.15.111:0/1871315978 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd814005110 con 0x7fd804005180 2024-09-13T17:59:21.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.751+0000 7fd80e7fc640 1 --2- 172.21.15.111:0/1871315978 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd7fc06ac30 0x7fd7fc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.751+0000 7fd80e7fc640 1 -- 172.21.15.111:0/1871315978 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fd814089650 con 0x7fd804005180 2024-09-13T17:59:21.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.751+0000 7fd81d672640 1 --2- 172.21.15.111:0/1871315978 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd7fc06ac30 0x7fd7fc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.751+0000 7fd80e7fc640 1 --2- 172.21.15.111:0/1871315978 >> [v2:172.21.15.188:6816/1492113133,v1:172.21.15.188:6817/1492113133] conn(0x7fd7fc070fd0 0x7fd7fc073410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.751+0000 7fd80e7fc640 1 -- 172.21.15.111:0/1871315978 --> [v2:172.21.15.188:6816/1492113133,v1:172.21.15.188:6817/1492113133] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd7fc073ae0 con 0x7fd7fc070fd0 2024-09-13T17:59:21.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.751+0000 7fd80e7fc640 1 -- 172.21.15.111:0/1871315978 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8140899c0 con 0x7fd804005180 2024-09-13T17:59:21.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.751+0000 7fd81e674640 1 --2- 172.21.15.111:0/1871315978 >> [v2:172.21.15.188:6816/1492113133,v1:172.21.15.188:6817/1492113133] conn(0x7fd7fc070fd0 0x7fd7fc073410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.751+0000 7fd81d672640 1 --2- 172.21.15.111:0/1871315978 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd7fc06ac30 0x7fd7fc06d0f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd818070a70 tx=0x7fd808009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:21.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.752+0000 7fd81e674640 1 --2- 172.21.15.111:0/1871315978 >> [v2:172.21.15.188:6816/1492113133,v1:172.21.15.188:6817/1492113133] conn(0x7fd7fc070fd0 0x7fd7fc073410 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:21.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.752+0000 7fd80e7fc640 1 -- 172.21.15.111:0/1871315978 <== osd.4 v2:172.21.15.188:6816/1492113133 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fd7fc073ae0 con 0x7fd7fc070fd0 2024-09-13T17:59:21.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.764+0000 7fd8200d7640 1 -- 172.21.15.111:0/1871315978 --> [v2:172.21.15.188:6816/1492113133,v1:172.21.15.188:6817/1492113133] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd8180b0010 con 0x7fd7fc070fd0 2024-09-13T17:59:21.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.765+0000 7fd80e7fc640 1 -- 172.21.15.111:0/1871315978 <== osd.4 v2:172.21.15.188:6816/1492113133 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd8180b0010 con 0x7fd7fc070fd0 2024-09-13T17:59:21.766 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T17:59:21.767 INFO:teuthology.orchestra.run.smithi111.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:59:21.767 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T17:59:21.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.765+0000 7fd8200d7640 1 -- 172.21.15.111:0/1871315978 >> [v2:172.21.15.188:6816/1492113133,v1:172.21.15.188:6817/1492113133] conn(0x7fd7fc070fd0 msgr2=0x7fd7fc073410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.765+0000 7fd8200d7640 1 --2- 172.21.15.111:0/1871315978 >> [v2:172.21.15.188:6816/1492113133,v1:172.21.15.188:6817/1492113133] conn(0x7fd7fc070fd0 0x7fd7fc073410 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.765+0000 7fd8200d7640 1 -- 172.21.15.111:0/1871315978 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd7fc06ac30 msgr2=0x7fd7fc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.765+0000 7fd8200d7640 1 --2- 172.21.15.111:0/1871315978 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd7fc06ac30 0x7fd7fc06d0f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd818070a70 tx=0x7fd808009040 comp rx=0 tx=0).stop 2024-09-13T17:59:21.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.765+0000 7fd8200d7640 1 -- 172.21.15.111:0/1871315978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd804005180 msgr2=0x7fd8180aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.765+0000 7fd8200d7640 1 --2- 172.21.15.111:0/1871315978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd804005180 0x7fd8180aed20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd81400f8b0 tx=0x7fd81400fd70 comp rx=0 tx=0).stop 2024-09-13T17:59:21.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.765+0000 7fd81e674640 1 -- 172.21.15.111:0/1871315978 reap_dead start 2024-09-13T17:59:21.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.765+0000 7fd8200d7640 1 -- 172.21.15.111:0/1871315978 shutdown_connections 2024-09-13T17:59:21.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.765+0000 7fd8200d7640 1 -- 172.21.15.111:0/1871315978 >> 172.21.15.111:0/1871315978 conn(0x7fd81806a0b0 msgr2=0x7fd81806a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:21.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.765+0000 7fd8200d7640 1 -- 172.21.15.111:0/1871315978 shutdown_connections 2024-09-13T17:59:21.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.765+0000 7fd8200d7640 1 -- 172.21.15.111:0/1871315978 wait complete. 2024-09-13T17:59:21.777 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-13T17:59:21.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.873+0000 7fc17bdf4640 1 Processor -- start 2024-09-13T17:59:21.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.873+0000 7fc17bdf4640 1 -- start start 2024-09-13T17:59:21.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.874+0000 7fc17bdf4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1740cdff0 0x7fc17406c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.874+0000 7fc17bdf4640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1740cd450 con 0x7fc17415c500 2024-09-13T17:59:21.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.874+0000 7fc17bdf4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1740d0cf0 con 0x7fc1740d2760 2024-09-13T17:59:21.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.874+0000 7fc17bdf4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1740d0fd0 con 0x7fc1740cdff0 2024-09-13T17:59:21.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.874+0000 7fc179b90640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc17415c500 0x7fc1740d2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43122/0 (socket says 172.21.15.111:43122) 2024-09-13T17:59:21.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.874+0000 7fc179b90640 1 -- 172.21.15.111:0/593761073 learned_addr learned my addr 172.21.15.111:0/593761073 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:21.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.874+0000 7fc17a391640 1 --2- 172.21.15.111:0/593761073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1740cdff0 0x7fc17406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.874+0000 7fc178b8e640 1 -- 172.21.15.111:0/593761073 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189794389 0 0) 0x7fc1740cd450 con 0x7fc17415c500 2024-09-13T17:59:21.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.874+0000 7fc178b8e640 1 -- 172.21.15.111:0/593761073 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc160003520 con 0x7fc17415c500 2024-09-13T17:59:21.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc178b8e640 1 -- 172.21.15.111:0/593761073 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2941724741 0 0) 0x7fc160003520 con 0x7fc17415c500 2024-09-13T17:59:21.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc178b8e640 1 -- 172.21.15.111:0/593761073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1740cdff0 msgr2=0x7fc17406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:21.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc178b8e640 1 --2- 172.21.15.111:0/593761073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1740cdff0 0x7fc17406c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc178b8e640 1 -- 172.21.15.111:0/593761073 >> v1:172.21.15.188:6789/0 conn(0x7fc1740d2760 legacy=0x7fc1740cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc178b8e640 1 -- 172.21.15.111:0/593761073 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1740cd0e0 con 0x7fc17415c500 2024-09-13T17:59:21.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc178b8e640 1 -- 172.21.15.111:0/593761073 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7fc170001670 con 0x7fc17415c500 2024-09-13T17:59:21.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc178b8e640 1 -- 172.21.15.111:0/593761073 >> v1:172.21.15.111:6789/0 conn(0x7fc17415c500 legacy=0x7fc1740d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc178b8e640 1 --2- 172.21.15.111:0/593761073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1740cdff0 0x7fc1600039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc178b8e640 1 --2- 172.21.15.111:0/593761073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc160003f20 0x7fc1600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc178b8e640 1 --2- 172.21.15.111:0/593761073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc160004d00 0x7fc160005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc178b8e640 1 -- 172.21.15.111:0/593761073 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1740d0fd0 con 0x7fc160004d00 2024-09-13T17:59:21.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc178b8e640 1 -- 172.21.15.111:0/593761073 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16000ba60 con 0x7fc160003f20 2024-09-13T17:59:21.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc178b8e640 1 -- 172.21.15.111:0/593761073 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16000bc20 con 0x7fc1740cdff0 2024-09-13T17:59:21.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc17a391640 1 --2- 172.21.15.111:0/593761073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc160004d00 0x7fc160005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc179b90640 1 --2- 172.21.15.111:0/593761073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1740cdff0 0x7fc1600039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc17938f640 1 --2- 172.21.15.111:0/593761073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc160003f20 0x7fc1600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc17a391640 1 -- 172.21.15.111:0/593761073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1740cdff0 msgr2=0x7fc1600039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:21.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc17a391640 1 --2- 172.21.15.111:0/593761073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1740cdff0 0x7fc1600039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc17a391640 1 -- 172.21.15.111:0/593761073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc160003f20 msgr2=0x7fc1600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:21.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc17a391640 1 --2- 172.21.15.111:0/593761073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc160003f20 0x7fc1600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc17a391640 1 -- 172.21.15.111:0/593761073 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc160003520 con 0x7fc160004d00 2024-09-13T17:59:21.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.875+0000 7fc17938f640 1 --2- 172.21.15.111:0/593761073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc160003f20 0x7fc1600047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:21.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.876+0000 7fc17a391640 1 --2- 172.21.15.111:0/593761073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc160004d00 0x7fc160005180 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009fc0 tx=0x7fc1600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99adbc8053b2b51 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:21.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.876+0000 7fc178b8e640 1 -- 172.21.15.111:0/593761073 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c031640 con 0x7fc160004d00 2024-09-13T17:59:21.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.876+0000 7fc178b8e640 1 -- 172.21.15.111:0/593761073 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c02d070 con 0x7fc160004d00 2024-09-13T17:59:21.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.876+0000 7fc178b8e640 1 -- 172.21.15.111:0/593761073 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c0319e0 con 0x7fc160004d00 2024-09-13T17:59:21.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.876+0000 7fc17bdf4640 1 -- 172.21.15.111:0/593761073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc160004d00 msgr2=0x7fc160005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.876+0000 7fc17bdf4640 1 --2- 172.21.15.111:0/593761073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc160004d00 0x7fc160005180 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009fc0 tx=0x7fc1600057f0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.876+0000 7fc17a391640 1 -- 172.21.15.111:0/593761073 reap_dead start 2024-09-13T17:59:21.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.877+0000 7fc17bdf4640 1 -- 172.21.15.111:0/593761073 shutdown_connections 2024-09-13T17:59:21.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.877+0000 7fc17bdf4640 1 -- 172.21.15.111:0/593761073 >> 172.21.15.111:0/593761073 conn(0x7fc174089f60 msgr2=0x7fc17409b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:21.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.877+0000 7fc17bdf4640 1 -- 172.21.15.111:0/593761073 shutdown_connections 2024-09-13T17:59:21.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.877+0000 7fc17bdf4640 1 -- 172.21.15.111:0/593761073 wait complete. 2024-09-13T17:59:21.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.878+0000 7fc17bdf4640 1 Processor -- start 2024-09-13T17:59:21.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.878+0000 7fc17bdf4640 1 -- start start 2024-09-13T17:59:21.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.878+0000 7fc17bdf4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc160004d00 0x7fc174150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.878+0000 7fc17bdf4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc17415cd10 0x7fc17415d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.878+0000 7fc17bdf4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1740b5260 0x7fc1740ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc17bdf4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17415d7a0 con 0x7fc17415cd10 2024-09-13T17:59:21.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc17bdf4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1740b83b0 con 0x7fc160004d00 2024-09-13T17:59:21.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc17bdf4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17415dcb0 con 0x7fc1740b5260 2024-09-13T17:59:21.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc17938f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc17415cd10 0x7fc17415d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc17938f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc17415cd10 0x7fc17415d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55604/0 (socket says 172.21.15.111:55604) 2024-09-13T17:59:21.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc17938f640 1 -- 172.21.15.111:0/2395364096 learned_addr learned my addr 172.21.15.111:0/2395364096 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:21.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc179b90640 1 --2- 172.21.15.111:0/2395364096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc160004d00 0x7fc174150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc17a391640 1 --2- 172.21.15.111:0/2395364096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1740b5260 0x7fc1740ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc17938f640 1 -- 172.21.15.111:0/2395364096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1740b5260 msgr2=0x7fc1740ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc17938f640 1 --2- 172.21.15.111:0/2395364096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1740b5260 0x7fc1740ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc17938f640 1 -- 172.21.15.111:0/2395364096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc160004d00 msgr2=0x7fc174150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:21.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc17938f640 1 --2- 172.21.15.111:0/2395364096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc160004d00 0x7fc174150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:21.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc17938f640 1 -- 172.21.15.111:0/2395364096 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17415d9d0 con 0x7fc17415cd10 2024-09-13T17:59:21.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc17938f640 1 --2- 172.21.15.111:0/2395364096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc17415cd10 0x7fc17415d260 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc16400e550 tx=0x7fc16400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:21.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc16a7fc640 1 -- 172.21.15.111:0/2395364096 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1640052e0 con 0x7fc17415cd10 2024-09-13T17:59:21.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc16a7fc640 1 -- 172.21.15.111:0/2395364096 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc164005440 con 0x7fc17415cd10 2024-09-13T17:59:21.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc17bdf4640 1 -- 172.21.15.111:0/2395364096 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1740ca4b0 con 0x7fc17415cd10 2024-09-13T17:59:21.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc16a7fc640 1 -- 172.21.15.111:0/2395364096 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164003830 con 0x7fc17415cd10 2024-09-13T17:59:21.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.879+0000 7fc17bdf4640 1 -- 172.21.15.111:0/2395364096 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1740caa70 con 0x7fc17415cd10 2024-09-13T17:59:21.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.880+0000 7fc17bdf4640 1 -- 172.21.15.111:0/2395364096 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1740cac30 con 0x7fc17415cd10 2024-09-13T17:59:21.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.881+0000 7fc16a7fc640 1 -- 172.21.15.111:0/2395364096 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc164003990 con 0x7fc17415cd10 2024-09-13T17:59:21.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.881+0000 7fc16a7fc640 1 --2- 172.21.15.111:0/2395364096 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc15806ac00 0x7fc15806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:21.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.881+0000 7fc16a7fc640 1 -- 172.21.15.111:0/2395364096 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fc164088d60 con 0x7fc17415cd10 2024-09-13T17:59:21.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.881+0000 7fc179b90640 1 --2- 172.21.15.111:0/2395364096 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc15806ac00 0x7fc15806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:21.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.882+0000 7fc179b90640 1 --2- 172.21.15.111:0/2395364096 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc15806ac00 0x7fc15806d0c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc1700017a0 tx=0x7fc170001710 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:21.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:21.885+0000 7fc16a7fc640 1 -- 172.21.15.111:0/2395364096 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc164054150 con 0x7fc17415cd10 2024-09-13T17:59:22.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.046+0000 7fc17bdf4640 1 -- 172.21.15.111:0/2395364096 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1740d12b0 con 0x7fc15806ac00 2024-09-13T17:59:22.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.047+0000 7fc16a7fc640 1 -- 172.21.15.111:0/2395364096 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc170004d90 con 0x7fc15806ac00 2024-09-13T17:59:22.049 INFO:teuthology.orchestra.run.smithi111.stderr:instructed osd(s) 4 to deep-scrub 2024-09-13T17:59:22.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.048+0000 7fc17bdf4640 1 -- 172.21.15.111:0/2395364096 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc15806ac00 msgr2=0x7fc15806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.048+0000 7fc17bdf4640 1 --2- 172.21.15.111:0/2395364096 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc15806ac00 0x7fc15806d0c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc1700017a0 tx=0x7fc170001710 comp rx=0 tx=0).stop 2024-09-13T17:59:22.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.048+0000 7fc17bdf4640 1 -- 172.21.15.111:0/2395364096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc17415cd10 msgr2=0x7fc17415d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.048+0000 7fc17bdf4640 1 --2- 172.21.15.111:0/2395364096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc17415cd10 0x7fc17415d260 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc16400e550 tx=0x7fc16400ea10 comp rx=0 tx=0).stop 2024-09-13T17:59:22.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.048+0000 7fc17bdf4640 1 -- 172.21.15.111:0/2395364096 shutdown_connections 2024-09-13T17:59:22.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.048+0000 7fc17bdf4640 1 --2- 172.21.15.111:0/2395364096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1740b5260 0x7fc1740ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.048+0000 7fc17bdf4640 1 --2- 172.21.15.111:0/2395364096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc17415cd10 0x7fc17415d260 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.048+0000 7fc17bdf4640 1 --2- 172.21.15.111:0/2395364096 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fc15806ac00 0x7fc15806d0c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.048+0000 7fc17bdf4640 1 --2- 172.21.15.111:0/2395364096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc160004d00 0x7fc174150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.048+0000 7fc17bdf4640 1 -- 172.21.15.111:0/2395364096 >> 172.21.15.111:0/2395364096 conn(0x7fc174089f60 msgr2=0x7fc1740b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:22.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.049+0000 7fc17bdf4640 1 -- 172.21.15.111:0/2395364096 shutdown_connections 2024-09-13T17:59:22.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.049+0000 7fc17bdf4640 1 -- 172.21.15.111:0/2395364096 wait complete. 2024-09-13T17:59:22.061 INFO:tasks.ceph:Scrubbing osd.5 2024-09-13T17:59:22.061 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:59:22.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.160+0000 7effe048e640 1 Processor -- start 2024-09-13T17:59:22.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.160+0000 7effe048e640 1 -- start start 2024-09-13T17:59:22.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.161+0000 7effe048e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7effd815a3b0 0x7effd80b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.161+0000 7effe048e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effd80b1720 0x7effd80ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.161+0000 7effe048e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7effd80cea30 0x7effd806c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.161+0000 7effe048e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd80ceec0 con 0x7effd80cea30 2024-09-13T17:59:22.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.161+0000 7effe048e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd815a790 con 0x7effd815a3b0 2024-09-13T17:59:22.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.161+0000 7effe048e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd80b2680 con 0x7effd80b1720 2024-09-13T17:59:22.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.161+0000 7effdea2b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7effd80cea30 0x7effd806c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.161+0000 7effdea2b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7effd80cea30 0x7effd806c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55614/0 (socket says 172.21.15.111:55614) 2024-09-13T17:59:22.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.161+0000 7effdea2b640 1 -- 172.21.15.111:0/1699040908 learned_addr learned my addr 172.21.15.111:0/1699040908 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:22.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.161+0000 7effde22a640 1 --2- 172.21.15.111:0/1699040908 >> v2:172.21.15.188:3300/0 conn(0x7effd815a3b0 0x7effd80b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.161+0000 7effdda29640 1 --2- 172.21.15.111:0/1699040908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effd80b1720 0x7effd80ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.161+0000 7effdea2b640 1 -- 172.21.15.111:0/1699040908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effd80b1720 msgr2=0x7effd80ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:22.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.161+0000 7effdea2b640 1 --2- 172.21.15.111:0/1699040908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effd80b1720 0x7effd80ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.161+0000 7effdea2b640 1 -- 172.21.15.111:0/1699040908 >> v2:172.21.15.188:3300/0 conn(0x7effd815a3b0 msgr2=0x7effd80b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:22.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.161+0000 7effdea2b640 1 --2- 172.21.15.111:0/1699040908 >> v2:172.21.15.188:3300/0 conn(0x7effd815a3b0 0x7effd80b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.161+0000 7effdea2b640 1 -- 172.21.15.111:0/1699040908 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd80b2960 con 0x7effd80cea30 2024-09-13T17:59:22.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.162+0000 7effdea2b640 1 --2- 172.21.15.111:0/1699040908 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effd80cea30 0x7effd806c640 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7effd80cbd70 tx=0x7effd002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88275d19804141a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:22.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.162+0000 7effdd228640 1 -- 172.21.15.111:0/1699040908 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0031600 con 0x7effd80cea30 2024-09-13T17:59:22.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.162+0000 7effdd228640 1 -- 172.21.15.111:0/1699040908 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd002d050 con 0x7effd80cea30 2024-09-13T17:59:22.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.162+0000 7effdd228640 1 -- 172.21.15.111:0/1699040908 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0031980 con 0x7effd80cea30 2024-09-13T17:59:22.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.163+0000 7effe048e640 1 -- 172.21.15.111:0/1699040908 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effd80cea30 msgr2=0x7effd806c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.163+0000 7effe048e640 1 --2- 172.21.15.111:0/1699040908 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effd80cea30 0x7effd806c640 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7effd80cbd70 tx=0x7effd002a6a0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.163+0000 7effe048e640 1 -- 172.21.15.111:0/1699040908 shutdown_connections 2024-09-13T17:59:22.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.163+0000 7effe048e640 1 --2- 172.21.15.111:0/1699040908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effd80b1720 0x7effd80ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.163+0000 7effe048e640 1 --2- 172.21.15.111:0/1699040908 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effd80cea30 0x7effd806c640 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.163+0000 7effe048e640 1 --2- 172.21.15.111:0/1699040908 >> v2:172.21.15.188:3300/0 conn(0x7effd815a3b0 0x7effd80b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.163+0000 7effe048e640 1 -- 172.21.15.111:0/1699040908 >> 172.21.15.111:0/1699040908 conn(0x7effd806a0b0 msgr2=0x7effd806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:22.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.163+0000 7effe048e640 1 -- 172.21.15.111:0/1699040908 shutdown_connections 2024-09-13T17:59:22.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.163+0000 7effe048e640 1 -- 172.21.15.111:0/1699040908 wait complete. 2024-09-13T17:59:22.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.164+0000 7effe048e640 1 Processor -- start 2024-09-13T17:59:22.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.164+0000 7effe048e640 1 -- start start 2024-09-13T17:59:22.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effe048e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effd80b1720 0x7effd80c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effe048e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effd80cea30 0x7effd815c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effe048e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effd815a3b0 0x7effd81f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effe048e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd815c9a0 con 0x7effd80b1720 2024-09-13T17:59:22.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effe048e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd815cc80 con 0x7effd815a3b0 2024-09-13T17:59:22.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effe048e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd815cf60 con 0x7effd80cea30 2024-09-13T17:59:22.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effde22a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effd80b1720 0x7effd80c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effde22a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effd80b1720 0x7effd80c4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55618/0 (socket says 172.21.15.111:55618) 2024-09-13T17:59:22.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effde22a640 1 -- 172.21.15.111:0/3006956574 learned_addr learned my addr 172.21.15.111:0/3006956574 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:22.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effdea2b640 1 --2- 172.21.15.111:0/3006956574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effd815a3b0 0x7effd81f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effdda29640 1 --2- 172.21.15.111:0/3006956574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effd80cea30 0x7effd815c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effde22a640 1 -- 172.21.15.111:0/3006956574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effd80cea30 msgr2=0x7effd815c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effde22a640 1 --2- 172.21.15.111:0/3006956574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effd80cea30 0x7effd815c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effde22a640 1 -- 172.21.15.111:0/3006956574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effd815a3b0 msgr2=0x7effd81f1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effde22a640 1 --2- 172.21.15.111:0/3006956574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effd815a3b0 0x7effd81f1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effde22a640 1 -- 172.21.15.111:0/3006956574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd0009b30 con 0x7effd80b1720 2024-09-13T17:59:22.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effdda29640 1 --2- 172.21.15.111:0/3006956574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effd80cea30 0x7effd815c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:22.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effde22a640 1 --2- 172.21.15.111:0/3006956574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effd80b1720 0x7effd80c4b90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7effc800d800 tx=0x7effc800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:22.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effceffd640 1 -- 172.21.15.111:0/3006956574 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc800b6c0 con 0x7effd80b1720 2024-09-13T17:59:22.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effceffd640 1 -- 172.21.15.111:0/3006956574 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc800b820 con 0x7effd80b1720 2024-09-13T17:59:22.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effe048e640 1 -- 172.21.15.111:0/3006956574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd815d240 con 0x7effd80b1720 2024-09-13T17:59:22.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effceffd640 1 -- 172.21.15.111:0/3006956574 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc800bbb0 con 0x7effd80b1720 2024-09-13T17:59:22.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.165+0000 7effe048e640 1 -- 172.21.15.111:0/3006956574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd8071b20 con 0x7effd80b1720 2024-09-13T17:59:22.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.166+0000 7effe048e640 1 -- 172.21.15.111:0/3006956574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7effd80ab4c0 con 0x7effd80b1720 2024-09-13T17:59:22.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.167+0000 7effceffd640 1 -- 172.21.15.111:0/3006956574 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7effc800bd10 con 0x7effd80b1720 2024-09-13T17:59:22.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.167+0000 7effceffd640 1 --2- 172.21.15.111:0/3006956574 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7effbc06ac00 0x7effbc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.167+0000 7effdda29640 1 --2- 172.21.15.111:0/3006956574 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7effbc06ac00 0x7effbc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.167+0000 7effceffd640 1 -- 172.21.15.111:0/3006956574 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7effc8089670 con 0x7effd80b1720 2024-09-13T17:59:22.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.167+0000 7effceffd640 1 --2- 172.21.15.111:0/3006956574 >> [v2:172.21.15.188:6808/3407929299,v1:172.21.15.188:6809/3407929299] conn(0x7effbc070fa0 0x7effbc0733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.167+0000 7effceffd640 1 -- 172.21.15.111:0/3006956574 --> [v2:172.21.15.188:6808/3407929299,v1:172.21.15.188:6809/3407929299] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7effbc073ab0 con 0x7effbc070fa0 2024-09-13T17:59:22.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.167+0000 7effceffd640 1 -- 172.21.15.111:0/3006956574 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7effc808bd30 con 0x7effd80b1720 2024-09-13T17:59:22.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.168+0000 7effdea2b640 1 --2- 172.21.15.111:0/3006956574 >> [v2:172.21.15.188:6808/3407929299,v1:172.21.15.188:6809/3407929299] conn(0x7effbc070fa0 0x7effbc0733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.168+0000 7effdda29640 1 --2- 172.21.15.111:0/3006956574 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7effbc06ac00 0x7effbc06d0c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7effd4006fd0 tx=0x7effd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:22.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.168+0000 7effdea2b640 1 --2- 172.21.15.111:0/3006956574 >> [v2:172.21.15.188:6808/3407929299,v1:172.21.15.188:6809/3407929299] conn(0x7effbc070fa0 0x7effbc0733e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:22.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.169+0000 7effceffd640 1 -- 172.21.15.111:0/3006956574 <== osd.5 v2:172.21.15.188:6808/3407929299 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7effbc073ab0 con 0x7effbc070fa0 2024-09-13T17:59:22.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.181+0000 7effe048e640 1 -- 172.21.15.111:0/3006956574 --> [v2:172.21.15.188:6808/3407929299,v1:172.21.15.188:6809/3407929299] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7effd80ab1e0 con 0x7effbc070fa0 2024-09-13T17:59:22.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.181+0000 7effceffd640 1 -- 172.21.15.111:0/3006956574 <== osd.5 v2:172.21.15.188:6808/3407929299 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7effd0009510 con 0x7effbc070fa0 2024-09-13T17:59:22.183 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T17:59:22.183 INFO:teuthology.orchestra.run.smithi111.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:59:22.183 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T17:59:22.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.182+0000 7effe048e640 1 -- 172.21.15.111:0/3006956574 >> [v2:172.21.15.188:6808/3407929299,v1:172.21.15.188:6809/3407929299] conn(0x7effbc070fa0 msgr2=0x7effbc0733e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.182+0000 7effe048e640 1 --2- 172.21.15.111:0/3006956574 >> [v2:172.21.15.188:6808/3407929299,v1:172.21.15.188:6809/3407929299] conn(0x7effbc070fa0 0x7effbc0733e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.182+0000 7effe048e640 1 -- 172.21.15.111:0/3006956574 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7effbc06ac00 msgr2=0x7effbc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.182+0000 7effe048e640 1 --2- 172.21.15.111:0/3006956574 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7effbc06ac00 0x7effbc06d0c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7effd4006fd0 tx=0x7effd4008040 comp rx=0 tx=0).stop 2024-09-13T17:59:22.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.182+0000 7effe048e640 1 -- 172.21.15.111:0/3006956574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effd80b1720 msgr2=0x7effd80c4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.182+0000 7effe048e640 1 --2- 172.21.15.111:0/3006956574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effd80b1720 0x7effd80c4b90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7effc800d800 tx=0x7effc800dcc0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.182+0000 7effdea2b640 1 -- 172.21.15.111:0/3006956574 reap_dead start 2024-09-13T17:59:22.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.182+0000 7effe048e640 1 -- 172.21.15.111:0/3006956574 shutdown_connections 2024-09-13T17:59:22.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.182+0000 7effe048e640 1 -- 172.21.15.111:0/3006956574 >> 172.21.15.111:0/3006956574 conn(0x7effd806a0b0 msgr2=0x7effd80c9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:22.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.182+0000 7effe048e640 1 -- 172.21.15.111:0/3006956574 shutdown_connections 2024-09-13T17:59:22.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.182+0000 7effe048e640 1 -- 172.21.15.111:0/3006956574 wait complete. 2024-09-13T17:59:22.193 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-13T17:59:22.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.289+0000 7f240ca73640 1 Processor -- start 2024-09-13T17:59:22.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.289+0000 7f240ca73640 1 -- start start 2024-09-13T17:59:22.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.290+0000 7f240ca73640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f240815fc50 0x7f24080d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.290+0000 7f240ca73640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24080cdb30 0x7f24080b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.290+0000 7f240ca73640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24080d1900 con 0x7f24080b3060 2024-09-13T17:59:22.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.290+0000 7f240ca73640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24080d2380 con 0x7f240815fc50 2024-09-13T17:59:22.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.290+0000 7f240ca73640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24080b5f70 con 0x7f24080cdb30 2024-09-13T17:59:22.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.290+0000 7f2405d9b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f24080b3060 0x7f24080b5320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48906/0 (socket says 172.21.15.111:48906) 2024-09-13T17:59:22.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.290+0000 7f2405d9b640 1 -- 172.21.15.111:0/212107898 learned_addr learned my addr 172.21.15.111:0/212107898 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:22.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240559a640 1 -- 172.21.15.111:0/212107898 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879934896 0 0) 0x7f23f0001110 con 0x7f24080b3060 2024-09-13T17:59:22.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240559a640 1 -- 172.21.15.111:0/212107898 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23f4003520 con 0x7f24080b3060 2024-09-13T17:59:22.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f2406d9d640 1 --2- 172.21.15.111:0/212107898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24080cdb30 0x7f24080b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240659c640 1 --2- 172.21.15.111:0/212107898 >> v2:172.21.15.188:3300/0 conn(0x7f240815fc50 0x7f24080d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240559a640 1 -- 172.21.15.111:0/212107898 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2536926790 0 0) 0x7f23f4003520 con 0x7f24080b3060 2024-09-13T17:59:22.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240559a640 1 -- 172.21.15.111:0/212107898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24080cdb30 msgr2=0x7f24080b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:22.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240559a640 1 --2- 172.21.15.111:0/212107898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24080cdb30 0x7f24080b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240559a640 1 -- 172.21.15.111:0/212107898 >> v2:172.21.15.188:3300/0 conn(0x7f240815fc50 msgr2=0x7f24080d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:22.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240559a640 1 --2- 172.21.15.111:0/212107898 >> v2:172.21.15.188:3300/0 conn(0x7f240815fc50 0x7f24080d13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240559a640 1 -- 172.21.15.111:0/212107898 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24080d2be0 con 0x7f24080b3060 2024-09-13T17:59:22.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240559a640 1 -- 172.21.15.111:0/212107898 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7f23f0001820 con 0x7f24080b3060 2024-09-13T17:59:22.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f2406d9d640 1 --2- 172.21.15.111:0/212107898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24080cdb30 0x7f24080b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:22.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240559a640 1 -- 172.21.15.111:0/212107898 >> v1:172.21.15.111:6789/0 conn(0x7f24080b3060 legacy=0x7f24080b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240559a640 1 --2- 172.21.15.111:0/212107898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23f4003970 0x7f23f4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240559a640 1 --2- 172.21.15.111:0/212107898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f40047c0 0x7f23f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240559a640 1 --2- 172.21.15.111:0/212107898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24080cdb30 0x7f23f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240559a640 1 -- 172.21.15.111:0/212107898 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24080b5f70 con 0x7f23f40047c0 2024-09-13T17:59:22.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240559a640 1 -- 172.21.15.111:0/212107898 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f400ba30 con 0x7f23f4003970 2024-09-13T17:59:22.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240559a640 1 -- 172.21.15.111:0/212107898 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f400bbf0 con 0x7f24080cdb30 2024-09-13T17:59:22.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240659c640 1 --2- 172.21.15.111:0/212107898 >> v2:172.21.15.188:3300/0 conn(0x7f240815fc50 0x7f24080d13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:22.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f2405d9b640 1 --2- 172.21.15.111:0/212107898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f40047c0 0x7f23f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f2406d9d640 1 --2- 172.21.15.111:0/212107898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24080cdb30 0x7f23f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f240659c640 1 --2- 172.21.15.111:0/212107898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23f4003970 0x7f23f4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f2405d9b640 1 -- 172.21.15.111:0/212107898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24080cdb30 msgr2=0x7f23f4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:22.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f2405d9b640 1 --2- 172.21.15.111:0/212107898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24080cdb30 0x7f23f4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f2405d9b640 1 -- 172.21.15.111:0/212107898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23f4003970 msgr2=0x7f23f4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:22.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f2405d9b640 1 --2- 172.21.15.111:0/212107898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23f4003970 0x7f23f4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.291+0000 7f2405d9b640 1 -- 172.21.15.111:0/212107898 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f4003520 con 0x7f23f40047c0 2024-09-13T17:59:22.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.292+0000 7f2405d9b640 1 --2- 172.21.15.111:0/212107898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f40047c0 0x7f23f4004c40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f24080b8580 tx=0x7f23f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c58f8b598ff2078 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:22.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.292+0000 7f240559a640 1 -- 172.21.15.111:0/212107898 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f0005ad0 con 0x7f23f40047c0 2024-09-13T17:59:22.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.292+0000 7f240559a640 1 -- 172.21.15.111:0/212107898 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f002d070 con 0x7f23f40047c0 2024-09-13T17:59:22.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.292+0000 7f240559a640 1 -- 172.21.15.111:0/212107898 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f0005e90 con 0x7f23f40047c0 2024-09-13T17:59:22.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.293+0000 7f240ca73640 1 -- 172.21.15.111:0/212107898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f40047c0 msgr2=0x7f23f4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.293+0000 7f240ca73640 1 --2- 172.21.15.111:0/212107898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f40047c0 0x7f23f4004c40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f24080b8580 tx=0x7f23f40057f0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.293+0000 7f2406d9d640 1 -- 172.21.15.111:0/212107898 reap_dead start 2024-09-13T17:59:22.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.293+0000 7f240ca73640 1 -- 172.21.15.111:0/212107898 shutdown_connections 2024-09-13T17:59:22.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.293+0000 7f240ca73640 1 -- 172.21.15.111:0/212107898 >> 172.21.15.111:0/212107898 conn(0x7f2408089f60 msgr2=0x7f24080d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:22.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.293+0000 7f240ca73640 1 -- 172.21.15.111:0/212107898 shutdown_connections 2024-09-13T17:59:22.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.293+0000 7f240ca73640 1 -- 172.21.15.111:0/212107898 wait complete. 2024-09-13T17:59:22.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.294+0000 7f240ca73640 1 Processor -- start 2024-09-13T17:59:22.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.294+0000 7f240ca73640 1 -- start start 2024-09-13T17:59:22.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f240ca73640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f40047c0 0x7f240815c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f240ca73640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24080c8df0 0x7f240809f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f240ca73640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24080c7790 0x7f24080b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f240ca73640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24080c89d0 con 0x7f23f40047c0 2024-09-13T17:59:22.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f240ca73640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2408159950 con 0x7f24080c8df0 2024-09-13T17:59:22.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f240ca73640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24080c7d50 con 0x7f24080c7790 2024-09-13T17:59:22.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f240659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f40047c0 0x7f240815c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f240659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f40047c0 0x7f240815c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53460/0 (socket says 172.21.15.111:53460) 2024-09-13T17:59:22.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f240659c640 1 -- 172.21.15.111:0/1686917257 learned_addr learned my addr 172.21.15.111:0/1686917257 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:22.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f2406d9d640 1 --2- 172.21.15.111:0/1686917257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24080c7790 0x7f24080b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f2405d9b640 1 --2- 172.21.15.111:0/1686917257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24080c8df0 0x7f240809f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f240659c640 1 -- 172.21.15.111:0/1686917257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24080c7790 msgr2=0x7f24080b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f240659c640 1 --2- 172.21.15.111:0/1686917257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24080c7790 0x7f24080b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f240659c640 1 -- 172.21.15.111:0/1686917257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24080c8df0 msgr2=0x7f240809f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f240659c640 1 --2- 172.21.15.111:0/1686917257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24080c8df0 0x7f240809f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f240659c640 1 -- 172.21.15.111:0/1686917257 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24080c8030 con 0x7f23f40047c0 2024-09-13T17:59:22.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f2406d9d640 1 --2- 172.21.15.111:0/1686917257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24080c7790 0x7f24080b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:22.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f240659c640 1 --2- 172.21.15.111:0/1686917257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f40047c0 0x7f240815c600 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2400004700 tx=0x7f2400007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:22.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f23feffd640 1 -- 172.21.15.111:0/1686917257 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400014610 con 0x7f23f40047c0 2024-09-13T17:59:22.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.295+0000 7f23feffd640 1 -- 172.21.15.111:0/1686917257 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240000f050 con 0x7f23f40047c0 2024-09-13T17:59:22.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.296+0000 7f240ca73640 1 -- 172.21.15.111:0/1686917257 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24080c8310 con 0x7f23f40047c0 2024-09-13T17:59:22.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.296+0000 7f23feffd640 1 -- 172.21.15.111:0/1686917257 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24000149b0 con 0x7f23f40047c0 2024-09-13T17:59:22.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.296+0000 7f240ca73640 1 -- 172.21.15.111:0/1686917257 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240809fe80 con 0x7f23f40047c0 2024-09-13T17:59:22.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.297+0000 7f240ca73640 1 -- 172.21.15.111:0/1686917257 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24080c93d0 con 0x7f23f40047c0 2024-09-13T17:59:22.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.297+0000 7f23feffd640 1 -- 172.21.15.111:0/1686917257 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2400014b10 con 0x7f23f40047c0 2024-09-13T17:59:22.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.297+0000 7f23feffd640 1 --2- 172.21.15.111:0/1686917257 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f23e806ac30 0x7f23e806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.297+0000 7f23feffd640 1 -- 172.21.15.111:0/1686917257 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f2400089ba0 con 0x7f23f40047c0 2024-09-13T17:59:22.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.298+0000 7f2405d9b640 1 --2- 172.21.15.111:0/1686917257 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f23e806ac30 0x7f23e806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.298+0000 7f2405d9b640 1 --2- 172.21.15.111:0/1686917257 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f23e806ac30 0x7f23e806d0f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f23f0032000 tx=0x7f23f0009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:22.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.302+0000 7f23feffd640 1 -- 172.21.15.111:0/1686917257 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f2400055030 con 0x7f23f40047c0 2024-09-13T17:59:22.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.461+0000 7f240ca73640 1 -- 172.21.15.111:0/1686917257 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f24080d3180 con 0x7f23e806ac30 2024-09-13T17:59:22.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.462+0000 7f23feffd640 1 -- 172.21.15.111:0/1686917257 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f23f0001880 con 0x7f23e806ac30 2024-09-13T17:59:22.464 INFO:teuthology.orchestra.run.smithi111.stderr:instructed osd(s) 5 to deep-scrub 2024-09-13T17:59:22.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.464+0000 7f240ca73640 1 -- 172.21.15.111:0/1686917257 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f23e806ac30 msgr2=0x7f23e806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.464+0000 7f240ca73640 1 --2- 172.21.15.111:0/1686917257 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f23e806ac30 0x7f23e806d0f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f23f0032000 tx=0x7f23f0009030 comp rx=0 tx=0).stop 2024-09-13T17:59:22.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.464+0000 7f240ca73640 1 -- 172.21.15.111:0/1686917257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f40047c0 msgr2=0x7f240815c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.464+0000 7f240ca73640 1 --2- 172.21.15.111:0/1686917257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f40047c0 0x7f240815c600 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2400004700 tx=0x7f2400007700 comp rx=0 tx=0).stop 2024-09-13T17:59:22.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.464+0000 7f240ca73640 1 -- 172.21.15.111:0/1686917257 shutdown_connections 2024-09-13T17:59:22.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.464+0000 7f240ca73640 1 --2- 172.21.15.111:0/1686917257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24080c7790 0x7f24080b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.464+0000 7f240ca73640 1 --2- 172.21.15.111:0/1686917257 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f23e806ac30 0x7f23e806d0f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.464+0000 7f240ca73640 1 --2- 172.21.15.111:0/1686917257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24080c8df0 0x7f240809f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.464+0000 7f240ca73640 1 --2- 172.21.15.111:0/1686917257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f40047c0 0x7f240815c600 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.464+0000 7f240ca73640 1 -- 172.21.15.111:0/1686917257 >> 172.21.15.111:0/1686917257 conn(0x7f2408089f60 msgr2=0x7f240806c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:22.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.464+0000 7f240ca73640 1 -- 172.21.15.111:0/1686917257 shutdown_connections 2024-09-13T17:59:22.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.464+0000 7f240ca73640 1 -- 172.21.15.111:0/1686917257 wait complete. 2024-09-13T17:59:22.476 INFO:tasks.ceph:Scrubbing osd.6 2024-09-13T17:59:22.476 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:59:22.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.575+0000 7f720c2b1640 1 Processor -- start 2024-09-13T17:59:22.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.575+0000 7f720c2b1640 1 -- start start 2024-09-13T17:59:22.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.576+0000 7f720c2b1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f72040cec00 0x7f72040ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.576+0000 7f720c2b1640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72040ccbe0 con 0x7f72040b2b40 2024-09-13T17:59:22.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.576+0000 7f720c2b1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72040b1810 con 0x7f7204159310 2024-09-13T17:59:22.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.576+0000 7f720c2b1640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72040ad940 con 0x7f72040cec00 2024-09-13T17:59:22.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.576+0000 7f720a84e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f72040cec00 0x7f72040ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.576+0000 7f720984c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f72040b2b40 0x7f72040ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48914/0 (socket says 172.21.15.111:48914) 2024-09-13T17:59:22.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.576+0000 7f720984c640 1 -- 172.21.15.111:0/2836885054 learned_addr learned my addr 172.21.15.111:0/2836885054 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:22.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.576+0000 7f720a84e640 1 -- 172.21.15.111:0/2836885054 >> v1:172.21.15.188:6789/0 conn(0x7f7204159310 legacy=0x7f72040ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:22.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.576+0000 7f720a84e640 1 -- 172.21.15.111:0/2836885054 >> v1:172.21.15.111:6789/0 conn(0x7f72040b2b40 legacy=0x7f72040ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.576+0000 7f720a84e640 1 -- 172.21.15.111:0/2836885054 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720406d9e0 con 0x7f72040cec00 2024-09-13T17:59:22.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.576+0000 7f720904b640 1 -- 172.21.15.111:0/2836885054 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1360601415 0 0) 0x7f72040ccbe0 con 0x7f72040b2b40 2024-09-13T17:59:22.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.577+0000 7f720a84e640 1 --2- 172.21.15.111:0/2836885054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72040cec00 0x7f72040ad400 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009d90 tx=0x7f71fc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1b8dba837fd41b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:22.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.577+0000 7f720904b640 1 -- 172.21.15.111:0/2836885054 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc031600 con 0x7f72040cec00 2024-09-13T17:59:22.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.577+0000 7f720904b640 1 -- 172.21.15.111:0/2836885054 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71fc02d050 con 0x7f72040cec00 2024-09-13T17:59:22.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.577+0000 7f720904b640 1 -- 172.21.15.111:0/2836885054 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc031980 con 0x7f72040cec00 2024-09-13T17:59:22.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.577+0000 7f720c2b1640 1 -- 172.21.15.111:0/2836885054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72040cec00 msgr2=0x7f72040ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.577+0000 7f720c2b1640 1 --2- 172.21.15.111:0/2836885054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72040cec00 0x7f72040ad400 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009d90 tx=0x7f71fc02a610 comp rx=0 tx=0).stop 2024-09-13T17:59:22.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.578+0000 7f720c2b1640 1 -- 172.21.15.111:0/2836885054 shutdown_connections 2024-09-13T17:59:22.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.578+0000 7f720c2b1640 1 --2- 172.21.15.111:0/2836885054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72040cec00 0x7f72040ad400 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.578+0000 7f720c2b1640 1 -- 172.21.15.111:0/2836885054 >> 172.21.15.111:0/2836885054 conn(0x7f720406a0b0 msgr2=0x7f720406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:22.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.578+0000 7f720c2b1640 1 -- 172.21.15.111:0/2836885054 shutdown_connections 2024-09-13T17:59:22.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.578+0000 7f720c2b1640 1 -- 172.21.15.111:0/2836885054 wait complete. 2024-09-13T17:59:22.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.579+0000 7f720c2b1640 1 Processor -- start 2024-09-13T17:59:22.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.579+0000 7f720c2b1640 1 -- start start 2024-09-13T17:59:22.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.579+0000 7f720c2b1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72040b2b40 0x7f7204098250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.579+0000 7f720c2b1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72040cec00 0x7f72040c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f720c2b1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7204159310 0x7f7204156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f720c2b1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72040ca130 con 0x7f7204159310 2024-09-13T17:59:22.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f720c2b1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72040988a0 con 0x7f72040b2b40 2024-09-13T17:59:22.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f720c2b1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7204098b80 con 0x7f72040cec00 2024-09-13T17:59:22.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f720a84e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7204159310 0x7f7204156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f720a84e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7204159310 0x7f7204156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53490/0 (socket says 172.21.15.111:53490) 2024-09-13T17:59:22.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f720a84e640 1 -- 172.21.15.111:0/1983983711 learned_addr learned my addr 172.21.15.111:0/1983983711 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:22.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f720984c640 1 --2- 172.21.15.111:0/1983983711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72040cec00 0x7f72040c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f720a04d640 1 --2- 172.21.15.111:0/1983983711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72040b2b40 0x7f7204098250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f720a84e640 1 -- 172.21.15.111:0/1983983711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72040cec00 msgr2=0x7f72040c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f720a84e640 1 --2- 172.21.15.111:0/1983983711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72040cec00 0x7f72040c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f720a84e640 1 -- 172.21.15.111:0/1983983711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72040b2b40 msgr2=0x7f7204098250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f720a84e640 1 --2- 172.21.15.111:0/1983983711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72040b2b40 0x7f7204098250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f720a84e640 1 -- 172.21.15.111:0/1983983711 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc009a70 con 0x7f7204159310 2024-09-13T17:59:22.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f720a84e640 1 --2- 172.21.15.111:0/1983983711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7204159310 0x7f7204156470 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009eb0 tx=0x7f71fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:22.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f71faffd640 1 -- 172.21.15.111:0/1983983711 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc03c830 con 0x7f7204159310 2024-09-13T17:59:22.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f71faffd640 1 -- 172.21.15.111:0/1983983711 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71fc03c990 con 0x7f7204159310 2024-09-13T17:59:22.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f71faffd640 1 -- 172.21.15.111:0/1983983711 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc03cd00 con 0x7f7204159310 2024-09-13T17:59:22.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f720c2b1640 1 -- 172.21.15.111:0/1983983711 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72040ade70 con 0x7f7204159310 2024-09-13T17:59:22.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.580+0000 7f720c2b1640 1 -- 172.21.15.111:0/1983983711 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72041569b0 con 0x7f7204159310 2024-09-13T17:59:22.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.581+0000 7f720c2b1640 1 -- 172.21.15.111:0/1983983711 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f72040ab850 con 0x7f7204159310 2024-09-13T17:59:22.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.582+0000 7f71faffd640 1 -- 172.21.15.111:0/1983983711 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f71fc045460 con 0x7f7204159310 2024-09-13T17:59:22.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.582+0000 7f71faffd640 1 --2- 172.21.15.111:0/1983983711 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f71e806ac30 0x7f71e806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.582+0000 7f71faffd640 1 -- 172.21.15.111:0/1983983711 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f71fc0aacb0 con 0x7f7204159310 2024-09-13T17:59:22.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.582+0000 7f720a04d640 1 --2- 172.21.15.111:0/1983983711 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f71e806ac30 0x7f71e806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.582+0000 7f71faffd640 1 --2- 172.21.15.111:0/1983983711 >> [v2:172.21.15.188:6800/1593506388,v1:172.21.15.188:6801/1593506388] conn(0x7f71e8070fd0 0x7f71e8073410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.582+0000 7f71faffd640 1 -- 172.21.15.111:0/1983983711 --> [v2:172.21.15.188:6800/1593506388,v1:172.21.15.188:6801/1593506388] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71e8073ae0 con 0x7f71e8070fd0 2024-09-13T17:59:22.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.582+0000 7f71faffd640 1 -- 172.21.15.111:0/1983983711 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71fc0ad370 con 0x7f7204159310 2024-09-13T17:59:22.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.583+0000 7f720984c640 1 --2- 172.21.15.111:0/1983983711 >> [v2:172.21.15.188:6800/1593506388,v1:172.21.15.188:6801/1593506388] conn(0x7f71e8070fd0 0x7f71e8073410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.583+0000 7f720a04d640 1 --2- 172.21.15.111:0/1983983711 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f71e806ac30 0x7f71e806d0f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7200006fd0 tx=0x7f7200009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:22.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.583+0000 7f720984c640 1 --2- 172.21.15.111:0/1983983711 >> [v2:172.21.15.188:6800/1593506388,v1:172.21.15.188:6801/1593506388] conn(0x7f71e8070fd0 0x7f71e8073410 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:22.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.584+0000 7f71faffd640 1 -- 172.21.15.111:0/1983983711 <== osd.6 v2:172.21.15.188:6800/1593506388 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f71e8073ae0 con 0x7f71e8070fd0 2024-09-13T17:59:22.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.595+0000 7f720c2b1640 1 -- 172.21.15.111:0/1983983711 --> [v2:172.21.15.188:6800/1593506388,v1:172.21.15.188:6801/1593506388] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f72040b1a80 con 0x7f71e8070fd0 2024-09-13T17:59:22.598 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.596+0000 7f71faffd640 1 -- 172.21.15.111:0/1983983711 <== osd.6 v2:172.21.15.188:6800/1593506388 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f71f40072e0 con 0x7f71e8070fd0 2024-09-13T17:59:22.598 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T17:59:22.598 INFO:teuthology.orchestra.run.smithi111.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:59:22.598 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T17:59:22.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.596+0000 7f720c2b1640 1 -- 172.21.15.111:0/1983983711 >> [v2:172.21.15.188:6800/1593506388,v1:172.21.15.188:6801/1593506388] conn(0x7f71e8070fd0 msgr2=0x7f71e8073410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.596+0000 7f720c2b1640 1 --2- 172.21.15.111:0/1983983711 >> [v2:172.21.15.188:6800/1593506388,v1:172.21.15.188:6801/1593506388] conn(0x7f71e8070fd0 0x7f71e8073410 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.596+0000 7f720c2b1640 1 -- 172.21.15.111:0/1983983711 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f71e806ac30 msgr2=0x7f71e806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.596+0000 7f720c2b1640 1 --2- 172.21.15.111:0/1983983711 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f71e806ac30 0x7f71e806d0f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7200006fd0 tx=0x7f7200009060 comp rx=0 tx=0).stop 2024-09-13T17:59:22.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.596+0000 7f720c2b1640 1 -- 172.21.15.111:0/1983983711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7204159310 msgr2=0x7f7204156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.596+0000 7f720c2b1640 1 --2- 172.21.15.111:0/1983983711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7204159310 0x7f7204156470 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009eb0 tx=0x7f71fc004a40 comp rx=0 tx=0).stop 2024-09-13T17:59:22.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.596+0000 7f720a84e640 1 -- 172.21.15.111:0/1983983711 reap_dead start 2024-09-13T17:59:22.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.597+0000 7f720c2b1640 1 -- 172.21.15.111:0/1983983711 shutdown_connections 2024-09-13T17:59:22.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.597+0000 7f720c2b1640 1 -- 172.21.15.111:0/1983983711 >> 172.21.15.111:0/1983983711 conn(0x7f720406a0b0 msgr2=0x7f72040ab140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:22.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.597+0000 7f720c2b1640 1 -- 172.21.15.111:0/1983983711 shutdown_connections 2024-09-13T17:59:22.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.597+0000 7f720c2b1640 1 -- 172.21.15.111:0/1983983711 wait complete. 2024-09-13T17:59:22.608 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-13T17:59:22.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.705+0000 7f5d4c9ce640 1 Processor -- start 2024-09-13T17:59:22.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.705+0000 7f5d4c9ce640 1 -- start start 2024-09-13T17:59:22.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.705+0000 7f5d4c9ce640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5d480b2880 0x7f5d48163690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.705+0000 7f5d4c9ce640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d480b2d10 con 0x7f5d48158b00 2024-09-13T17:59:22.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.705+0000 7f5d4c9ce640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d480b2ff0 con 0x7f5d4806c8a0 2024-09-13T17:59:22.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.705+0000 7f5d4c9ce640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d48070310 con 0x7f5d480b2880 2024-09-13T17:59:22.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.706+0000 7f5d46d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5d480b2880 0x7f5d48163690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.706+0000 7f5d4659c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5d48158b00 0x7f5d4806c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48918/0 (socket says 172.21.15.111:48918) 2024-09-13T17:59:22.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.706+0000 7f5d4659c640 1 -- 172.21.15.111:0/2268604058 learned_addr learned my addr 172.21.15.111:0/2268604058 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:22.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.706+0000 7f5d46d9d640 1 -- 172.21.15.111:0/2268604058 >> v1:172.21.15.188:6789/0 conn(0x7f5d4806c8a0 legacy=0x7f5d480b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:22.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.706+0000 7f5d46d9d640 1 -- 172.21.15.111:0/2268604058 >> v1:172.21.15.111:6789/0 conn(0x7f5d48158b00 legacy=0x7f5d4806c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.706+0000 7f5d4559a640 1 -- 172.21.15.111:0/2268604058 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005677364 0 0) 0x7f5d480b2d10 con 0x7f5d48158b00 2024-09-13T17:59:22.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.706+0000 7f5d46d9d640 1 -- 172.21.15.111:0/2268604058 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d480705f0 con 0x7f5d480b2880 2024-09-13T17:59:22.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.707+0000 7f5d46d9d640 1 --2- 172.21.15.111:0/2268604058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d480b2880 0x7f5d48163690 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000aea0 tx=0x7f5d4002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dae39c8dbf047da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:22.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.707+0000 7f5d4559a640 1 -- 172.21.15.111:0/2268604058 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40008640 con 0x7f5d480b2880 2024-09-13T17:59:22.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.707+0000 7f5d4559a640 1 -- 172.21.15.111:0/2268604058 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4002e050 con 0x7f5d480b2880 2024-09-13T17:59:22.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.707+0000 7f5d4559a640 1 -- 172.21.15.111:0/2268604058 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d400089c0 con 0x7f5d480b2880 2024-09-13T17:59:22.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.707+0000 7f5d4c9ce640 1 -- 172.21.15.111:0/2268604058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d480b2880 msgr2=0x7f5d48163690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.707+0000 7f5d4c9ce640 1 --2- 172.21.15.111:0/2268604058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d480b2880 0x7f5d48163690 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000aea0 tx=0x7f5d4002b7f0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.707+0000 7f5d4c9ce640 1 -- 172.21.15.111:0/2268604058 shutdown_connections 2024-09-13T17:59:22.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.707+0000 7f5d4c9ce640 1 --2- 172.21.15.111:0/2268604058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d480b2880 0x7f5d48163690 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.707+0000 7f5d4c9ce640 1 -- 172.21.15.111:0/2268604058 >> 172.21.15.111:0/2268604058 conn(0x7f5d4806a0b0 msgr2=0x7f5d4806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:22.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.707+0000 7f5d4c9ce640 1 -- 172.21.15.111:0/2268604058 shutdown_connections 2024-09-13T17:59:22.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.707+0000 7f5d4c9ce640 1 -- 172.21.15.111:0/2268604058 wait complete. 2024-09-13T17:59:22.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.709+0000 7f5d4c9ce640 1 Processor -- start 2024-09-13T17:59:22.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.709+0000 7f5d4c9ce640 1 -- start start 2024-09-13T17:59:22.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.709+0000 7f5d4c9ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4806c8a0 0x7f5d4809a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.709+0000 7f5d4c9ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d48158b00 0x7f5d48098c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.709+0000 7f5d4c9ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d480ae750 0x7f5d4815b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.709+0000 7f5d4c9ce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4809aca0 con 0x7f5d480ae750 2024-09-13T17:59:22.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.709+0000 7f5d4c9ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480996c0 con 0x7f5d48158b00 2024-09-13T17:59:22.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.709+0000 7f5d4c9ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480999a0 con 0x7f5d4806c8a0 2024-09-13T17:59:22.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.710+0000 7f5d46d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d480ae750 0x7f5d4815b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.710+0000 7f5d46d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d480ae750 0x7f5d4815b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53508/0 (socket says 172.21.15.111:53508) 2024-09-13T17:59:22.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.710+0000 7f5d46d9d640 1 -- 172.21.15.111:0/1812411817 learned_addr learned my addr 172.21.15.111:0/1812411817 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:22.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.710+0000 7f5d45d9b640 1 --2- 172.21.15.111:0/1812411817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d48158b00 0x7f5d48098c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.710+0000 7f5d4659c640 1 --2- 172.21.15.111:0/1812411817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4806c8a0 0x7f5d4809a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.710+0000 7f5d46d9d640 1 -- 172.21.15.111:0/1812411817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4806c8a0 msgr2=0x7f5d4809a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.710+0000 7f5d46d9d640 1 --2- 172.21.15.111:0/1812411817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4806c8a0 0x7f5d4809a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.710+0000 7f5d46d9d640 1 -- 172.21.15.111:0/1812411817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d48158b00 msgr2=0x7f5d48098c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.710+0000 7f5d46d9d640 1 --2- 172.21.15.111:0/1812411817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d48158b00 0x7f5d48098c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.710+0000 7f5d46d9d640 1 -- 172.21.15.111:0/1812411817 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4000ab80 con 0x7f5d480ae750 2024-09-13T17:59:22.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.710+0000 7f5d4659c640 1 --2- 172.21.15.111:0/1812411817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4806c8a0 0x7f5d4809a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:22.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.710+0000 7f5d46d9d640 1 --2- 172.21.15.111:0/1812411817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d480ae750 0x7f5d4815b820 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000aea0 tx=0x7f5d40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:22.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.710+0000 7f5d3effd640 1 -- 172.21.15.111:0/1812411817 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4003e860 con 0x7f5d480ae750 2024-09-13T17:59:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.710+0000 7f5d3effd640 1 -- 172.21.15.111:0/1812411817 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4003e9e0 con 0x7f5d480ae750 2024-09-13T17:59:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.710+0000 7f5d4c9ce640 1 -- 172.21.15.111:0/1812411817 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d48099c80 con 0x7f5d480ae750 2024-09-13T17:59:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.710+0000 7f5d3effd640 1 -- 172.21.15.111:0/1812411817 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4003ed90 con 0x7f5d480ae750 2024-09-13T17:59:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.710+0000 7f5d4c9ce640 1 -- 172.21.15.111:0/1812411817 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4809a240 con 0x7f5d480ae750 2024-09-13T17:59:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.711+0000 7f5d4c9ce640 1 -- 172.21.15.111:0/1812411817 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4806c190 con 0x7f5d480ae750 2024-09-13T17:59:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.712+0000 7f5d3effd640 1 -- 172.21.15.111:0/1812411817 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5d4003c400 con 0x7f5d480ae750 2024-09-13T17:59:22.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.712+0000 7f5d3effd640 1 --2- 172.21.15.111:0/1812411817 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5d2806ac00 0x7f5d2806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.712+0000 7f5d3effd640 1 -- 172.21.15.111:0/1812411817 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f5d400aa270 con 0x7f5d480ae750 2024-09-13T17:59:22.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.712+0000 7f5d4659c640 1 --2- 172.21.15.111:0/1812411817 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5d2806ac00 0x7f5d2806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.713+0000 7f5d4659c640 1 --2- 172.21.15.111:0/1812411817 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5d2806ac00 0x7f5d2806d0c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5d38000f60 tx=0x7f5d38007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:22.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.716+0000 7f5d3effd640 1 -- 172.21.15.111:0/1812411817 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f5d40075760 con 0x7f5d480ae750 2024-09-13T17:59:22.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.877+0000 7f5d4c9ce640 1 -- 172.21.15.111:0/1812411817 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d480cb8c0 con 0x7f5d2806ac00 2024-09-13T17:59:22.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.877+0000 7f5d3effd640 1 -- 172.21.15.111:0/1812411817 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5d480cb8c0 con 0x7f5d2806ac00 2024-09-13T17:59:22.879 INFO:teuthology.orchestra.run.smithi111.stderr:instructed osd(s) 6 to deep-scrub 2024-09-13T17:59:22.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.879+0000 7f5d4c9ce640 1 -- 172.21.15.111:0/1812411817 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5d2806ac00 msgr2=0x7f5d2806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.879+0000 7f5d4c9ce640 1 --2- 172.21.15.111:0/1812411817 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5d2806ac00 0x7f5d2806d0c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5d38000f60 tx=0x7f5d38007c10 comp rx=0 tx=0).stop 2024-09-13T17:59:22.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.879+0000 7f5d4c9ce640 1 -- 172.21.15.111:0/1812411817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d480ae750 msgr2=0x7f5d4815b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.879+0000 7f5d4c9ce640 1 --2- 172.21.15.111:0/1812411817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d480ae750 0x7f5d4815b820 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000aea0 tx=0x7f5d40004a40 comp rx=0 tx=0).stop 2024-09-13T17:59:22.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.879+0000 7f5d4c9ce640 1 -- 172.21.15.111:0/1812411817 shutdown_connections 2024-09-13T17:59:22.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.879+0000 7f5d4c9ce640 1 --2- 172.21.15.111:0/1812411817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d480ae750 0x7f5d4815b820 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.879+0000 7f5d4c9ce640 1 --2- 172.21.15.111:0/1812411817 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f5d2806ac00 0x7f5d2806d0c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.879+0000 7f5d4c9ce640 1 --2- 172.21.15.111:0/1812411817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d48158b00 0x7f5d48098c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.879+0000 7f5d4c9ce640 1 --2- 172.21.15.111:0/1812411817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4806c8a0 0x7f5d4809a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.879+0000 7f5d4c9ce640 1 -- 172.21.15.111:0/1812411817 >> 172.21.15.111:0/1812411817 conn(0x7f5d4806a0b0 msgr2=0x7f5d4806a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:22.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.879+0000 7f5d4c9ce640 1 -- 172.21.15.111:0/1812411817 shutdown_connections 2024-09-13T17:59:22.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.879+0000 7f5d4c9ce640 1 -- 172.21.15.111:0/1812411817 wait complete. 2024-09-13T17:59:22.891 INFO:tasks.ceph:Scrubbing osd.7 2024-09-13T17:59:22.891 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:59:22.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.991+0000 7fa988220640 1 Processor -- start 2024-09-13T17:59:22.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.992+0000 7fa988220640 1 -- start start 2024-09-13T17:59:22.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.992+0000 7fa988220640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa9800d2760 0x7fa9800cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.992+0000 7fa988220640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9800cde20 0x7fa9800d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.992+0000 7fa988220640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9800cd450 con 0x7fa98015c500 2024-09-13T17:59:22.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.992+0000 7fa988220640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9800d0cf0 con 0x7fa9800d2760 2024-09-13T17:59:22.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.992+0000 7fa988220640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9800d0fd0 con 0x7fa9800cde20 2024-09-13T17:59:22.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.992+0000 7fa985fbc640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa98015c500 0x7fa9800d2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48920/0 (socket says 172.21.15.111:48920) 2024-09-13T17:59:22.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.992+0000 7fa985fbc640 1 -- 172.21.15.111:0/790313259 learned_addr learned my addr 172.21.15.111:0/790313259 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:22.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa984fba640 1 -- 172.21.15.111:0/790313259 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1841474522 0 0) 0x7fa9800cd450 con 0x7fa98015c500 2024-09-13T17:59:22.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa9857bb640 1 --2- 172.21.15.111:0/790313259 >> v2:172.21.15.188:3300/0 conn(0x7fa9800d2760 0x7fa9800cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa9867bd640 1 --2- 172.21.15.111:0/790313259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9800cde20 0x7fa9800d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa984fba640 1 -- 172.21.15.111:0/790313259 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa968003520 con 0x7fa98015c500 2024-09-13T17:59:22.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa984fba640 1 -- 172.21.15.111:0/790313259 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3766165262 0 0) 0x7fa968003520 con 0x7fa98015c500 2024-09-13T17:59:22.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa984fba640 1 -- 172.21.15.111:0/790313259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9800cde20 msgr2=0x7fa9800d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:22.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa984fba640 1 --2- 172.21.15.111:0/790313259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9800cde20 0x7fa9800d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa984fba640 1 -- 172.21.15.111:0/790313259 >> v2:172.21.15.188:3300/0 conn(0x7fa9800d2760 msgr2=0x7fa9800cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:22.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa984fba640 1 --2- 172.21.15.111:0/790313259 >> v2:172.21.15.188:3300/0 conn(0x7fa9800d2760 0x7fa9800cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa984fba640 1 -- 172.21.15.111:0/790313259 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9800cd0e0 con 0x7fa98015c500 2024-09-13T17:59:22.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa984fba640 1 -- 172.21.15.111:0/790313259 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2571307892 0 0) 0x7fa9780019f0 con 0x7fa98015c500 2024-09-13T17:59:22.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa984fba640 1 -- 172.21.15.111:0/790313259 >> v1:172.21.15.111:6789/0 conn(0x7fa98015c500 legacy=0x7fa9800d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:22.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa984fba640 1 --2- 172.21.15.111:0/790313259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa968003970 0x7fa968003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa984fba640 1 --2- 172.21.15.111:0/790313259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9800cde20 0x7fa9680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa984fba640 1 --2- 172.21.15.111:0/790313259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa968004d00 0x7fa968005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:22.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa984fba640 1 -- 172.21.15.111:0/790313259 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9800d0cf0 con 0x7fa968004d00 2024-09-13T17:59:22.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa984fba640 1 -- 172.21.15.111:0/790313259 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9800d0fd0 con 0x7fa968003970 2024-09-13T17:59:22.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa984fba640 1 -- 172.21.15.111:0/790313259 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96800b980 con 0x7fa9800cde20 2024-09-13T17:59:22.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa9867bd640 1 --2- 172.21.15.111:0/790313259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa968004d00 0x7fa968005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa985fbc640 1 --2- 172.21.15.111:0/790313259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa968003970 0x7fa968003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa9857bb640 1 --2- 172.21.15.111:0/790313259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9800cde20 0x7fa9680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:22.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa9867bd640 1 -- 172.21.15.111:0/790313259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9800cde20 msgr2=0x7fa9680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:22.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa9867bd640 1 --2- 172.21.15.111:0/790313259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9800cde20 0x7fa9680047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa9867bd640 1 -- 172.21.15.111:0/790313259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa968003970 msgr2=0x7fa968003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:22.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa9867bd640 1 --2- 172.21.15.111:0/790313259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa968003970 0x7fa968003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:22.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.993+0000 7fa9867bd640 1 -- 172.21.15.111:0/790313259 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa968003520 con 0x7fa968004d00 2024-09-13T17:59:22.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.994+0000 7fa9867bd640 1 --2- 172.21.15.111:0/790313259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa968004d00 0x7fa968005180 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa970009ac0 tx=0x7fa9680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d323a73a48117e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:22.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.994+0000 7fa984fba640 1 -- 172.21.15.111:0/790313259 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa970031400 con 0x7fa968004d00 2024-09-13T17:59:22.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.994+0000 7fa984fba640 1 -- 172.21.15.111:0/790313259 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97002d050 con 0x7fa968004d00 2024-09-13T17:59:22.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.994+0000 7fa984fba640 1 -- 172.21.15.111:0/790313259 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa970031780 con 0x7fa968004d00 2024-09-13T17:59:22.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.995+0000 7fa988220640 1 -- 172.21.15.111:0/790313259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa968004d00 msgr2=0x7fa968005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.995+0000 7fa988220640 1 --2- 172.21.15.111:0/790313259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa968004d00 0x7fa968005180 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa970009ac0 tx=0x7fa9680057f0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.995+0000 7fa9867bd640 1 -- 172.21.15.111:0/790313259 reap_dead start 2024-09-13T17:59:23.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.995+0000 7fa988220640 1 -- 172.21.15.111:0/790313259 shutdown_connections 2024-09-13T17:59:23.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.995+0000 7fa988220640 1 -- 172.21.15.111:0/790313259 >> 172.21.15.111:0/790313259 conn(0x7fa980089f60 msgr2=0x7fa98009b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:23.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.995+0000 7fa988220640 1 -- 172.21.15.111:0/790313259 shutdown_connections 2024-09-13T17:59:23.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.995+0000 7fa988220640 1 -- 172.21.15.111:0/790313259 wait complete. 2024-09-13T17:59:23.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.996+0000 7fa988220640 1 Processor -- start 2024-09-13T17:59:23.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.996+0000 7fa988220640 1 -- start start 2024-09-13T17:59:23.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.997+0000 7fa988220640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa968004d00 0x7fa9800cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:23.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.997+0000 7fa988220640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa98015f3e0 0x7fa9800b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:23.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.997+0000 7fa988220640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98014d5f0 0x7fa98014da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:23.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.997+0000 7fa988220640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9800caa20 con 0x7fa98014d5f0 2024-09-13T17:59:23.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.997+0000 7fa988220640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98014dfa0 con 0x7fa968004d00 2024-09-13T17:59:23.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.997+0000 7fa988220640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9800cac70 con 0x7fa98015f3e0 2024-09-13T17:59:23.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.997+0000 7fa9867bd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98014d5f0 0x7fa98014da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:23.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.997+0000 7fa9867bd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98014d5f0 0x7fa98014da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53528/0 (socket says 172.21.15.111:53528) 2024-09-13T17:59:23.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.997+0000 7fa9867bd640 1 -- 172.21.15.111:0/837611407 learned_addr learned my addr 172.21.15.111:0/837611407 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:23.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.997+0000 7fa9867bd640 1 -- 172.21.15.111:0/837611407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa98015f3e0 msgr2=0x7fa9800b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.997+0000 7fa9867bd640 1 --2- 172.21.15.111:0/837611407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa98015f3e0 0x7fa9800b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.997+0000 7fa9867bd640 1 -- 172.21.15.111:0/837611407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa968004d00 msgr2=0x7fa9800cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.997+0000 7fa9867bd640 1 --2- 172.21.15.111:0/837611407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa968004d00 0x7fa9800cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.997+0000 7fa9867bd640 1 -- 172.21.15.111:0/837611407 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9800caf50 con 0x7fa98014d5f0 2024-09-13T17:59:23.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.997+0000 7fa9867bd640 1 --2- 172.21.15.111:0/837611407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98014d5f0 0x7fa98014da60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa970034040 tx=0x7fa970004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:23.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.997+0000 7fa97e7fc640 1 -- 172.21.15.111:0/837611407 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97003a600 con 0x7fa98014d5f0 2024-09-13T17:59:23.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.998+0000 7fa97e7fc640 1 -- 172.21.15.111:0/837611407 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97003a7a0 con 0x7fa98014d5f0 2024-09-13T17:59:23.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.998+0000 7fa988220640 1 -- 172.21.15.111:0/837611407 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9800cb230 con 0x7fa98014d5f0 2024-09-13T17:59:23.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.998+0000 7fa97e7fc640 1 -- 172.21.15.111:0/837611407 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97003ab50 con 0x7fa98014d5f0 2024-09-13T17:59:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.998+0000 7fa988220640 1 -- 172.21.15.111:0/837611407 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9800cb7f0 con 0x7fa98014d5f0 2024-09-13T17:59:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.998+0000 7fa988220640 1 -- 172.21.15.111:0/837611407 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa9800d12b0 con 0x7fa98014d5f0 2024-09-13T17:59:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.999+0000 7fa97e7fc640 1 -- 172.21.15.111:0/837611407 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa970038040 con 0x7fa98014d5f0 2024-09-13T17:59:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:22.999+0000 7fa97e7fc640 1 --2- 172.21.15.111:0/837611407 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa96c06ac30 0x7fa96c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.000+0000 7fa97e7fc640 1 -- 172.21.15.111:0/837611407 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fa970000e40 con 0x7fa98014d5f0 2024-09-13T17:59:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.000+0000 7fa985fbc640 1 --2- 172.21.15.111:0/837611407 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa96c06ac30 0x7fa96c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.000+0000 7fa97e7fc640 1 --2- 172.21.15.111:0/837611407 >> [v2:172.21.15.188:6824/2518120865,v1:172.21.15.188:6825/2518120865] conn(0x7fa96c070fd0 0x7fa96c073410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.000+0000 7fa97e7fc640 1 -- 172.21.15.111:0/837611407 --> [v2:172.21.15.188:6824/2518120865,v1:172.21.15.188:6825/2518120865] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa96c073ae0 con 0x7fa96c070fd0 2024-09-13T17:59:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.000+0000 7fa97e7fc640 1 -- 172.21.15.111:0/837611407 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9700ab5f0 con 0x7fa98014d5f0 2024-09-13T17:59:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.000+0000 7fa985fbc640 1 --2- 172.21.15.111:0/837611407 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa96c06ac30 0x7fa96c06d0f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa9800b84f0 tx=0x7fa978000f40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.000+0000 7fa9857bb640 1 --2- 172.21.15.111:0/837611407 >> [v2:172.21.15.188:6824/2518120865,v1:172.21.15.188:6825/2518120865] conn(0x7fa96c070fd0 0x7fa96c073410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.000+0000 7fa9857bb640 1 --2- 172.21.15.111:0/837611407 >> [v2:172.21.15.188:6824/2518120865,v1:172.21.15.188:6825/2518120865] conn(0x7fa96c070fd0 0x7fa96c073410 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.001+0000 7fa97e7fc640 1 -- 172.21.15.111:0/837611407 <== osd.7 v2:172.21.15.188:6824/2518120865 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa96c073ae0 con 0x7fa96c070fd0 2024-09-13T17:59:23.014 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.013+0000 7fa988220640 1 -- 172.21.15.111:0/837611407 --> [v2:172.21.15.188:6824/2518120865,v1:172.21.15.188:6825/2518120865] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa9800ce270 con 0x7fa96c070fd0 2024-09-13T17:59:23.015 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.013+0000 7fa97e7fc640 1 -- 172.21.15.111:0/837611407 <== osd.7 v2:172.21.15.188:6824/2518120865 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa974009b90 con 0x7fa96c070fd0 2024-09-13T17:59:23.015 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T17:59:23.015 INFO:teuthology.orchestra.run.smithi111.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:59:23.016 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T17:59:23.016 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.014+0000 7fa988220640 1 -- 172.21.15.111:0/837611407 >> [v2:172.21.15.188:6824/2518120865,v1:172.21.15.188:6825/2518120865] conn(0x7fa96c070fd0 msgr2=0x7fa96c073410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.014+0000 7fa988220640 1 --2- 172.21.15.111:0/837611407 >> [v2:172.21.15.188:6824/2518120865,v1:172.21.15.188:6825/2518120865] conn(0x7fa96c070fd0 0x7fa96c073410 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.014+0000 7fa988220640 1 -- 172.21.15.111:0/837611407 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa96c06ac30 msgr2=0x7fa96c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.014+0000 7fa988220640 1 --2- 172.21.15.111:0/837611407 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fa96c06ac30 0x7fa96c06d0f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa9800b84f0 tx=0x7fa978000f40 comp rx=0 tx=0).stop 2024-09-13T17:59:23.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.014+0000 7fa988220640 1 -- 172.21.15.111:0/837611407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98014d5f0 msgr2=0x7fa98014da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.014+0000 7fa988220640 1 --2- 172.21.15.111:0/837611407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98014d5f0 0x7fa98014da60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa970034040 tx=0x7fa970004730 comp rx=0 tx=0).stop 2024-09-13T17:59:23.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.014+0000 7fa9867bd640 1 -- 172.21.15.111:0/837611407 reap_dead start 2024-09-13T17:59:23.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.014+0000 7fa988220640 1 -- 172.21.15.111:0/837611407 shutdown_connections 2024-09-13T17:59:23.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.014+0000 7fa988220640 1 -- 172.21.15.111:0/837611407 >> 172.21.15.111:0/837611407 conn(0x7fa980089f60 msgr2=0x7fa9800aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:23.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.014+0000 7fa988220640 1 -- 172.21.15.111:0/837611407 shutdown_connections 2024-09-13T17:59:23.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.014+0000 7fa988220640 1 -- 172.21.15.111:0/837611407 wait complete. 2024-09-13T17:59:23.026 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-13T17:59:23.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.125+0000 7fd295765640 1 Processor -- start 2024-09-13T17:59:23.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.125+0000 7fd295765640 1 -- start start 2024-09-13T17:59:23.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.126+0000 7fd295765640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd2900cd500 0x7fd2900b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:23.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.126+0000 7fd295765640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd2900b1370 0x7fd2900d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:23.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.126+0000 7fd295765640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2900d1f40 con 0x7fd2900d1650 2024-09-13T17:59:23.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.126+0000 7fd295765640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2900d2190 con 0x7fd2900b1370 2024-09-13T17:59:23.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.126+0000 7fd295765640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2900ccb20 con 0x7fd2900cd500 2024-09-13T17:59:23.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.126+0000 7fd287fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd2900b1370 0x7fd2900d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:23.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.126+0000 7fd287fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd2900b1370 0x7fd2900d2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53534/0 (socket says 172.21.15.111:53534) 2024-09-13T17:59:23.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.126+0000 7fd287fff640 1 -- 172.21.15.111:0/1054084766 learned_addr learned my addr 172.21.15.111:0/1054084766 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:23.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.126+0000 7fd28effd640 1 --2- 172.21.15.111:0/1054084766 >> v2:172.21.15.188:3300/0 conn(0x7fd2900cd500 0x7fd2900b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:23.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.126+0000 7fd287fff640 1 -- 172.21.15.111:0/1054084766 >> v2:172.21.15.188:3300/0 conn(0x7fd2900cd500 msgr2=0x7fd2900b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:23.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.126+0000 7fd287fff640 1 --2- 172.21.15.111:0/1054084766 >> v2:172.21.15.188:3300/0 conn(0x7fd2900cd500 0x7fd2900b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.126+0000 7fd287fff640 1 -- 172.21.15.111:0/1054084766 >> v1:172.21.15.188:6789/0 conn(0x7fd2900d1650 legacy=0x7fd2900b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.126+0000 7fd287fff640 1 -- 172.21.15.111:0/1054084766 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2900cce00 con 0x7fd2900b1370 2024-09-13T17:59:23.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.127+0000 7fd287fff640 1 --2- 172.21.15.111:0/1054084766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2900b1370 0x7fd2900d2f50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd278009870 tx=0x7fd27802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5af77d52f12391 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:23.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.127+0000 7fd28e7fc640 1 -- 172.21.15.111:0/1054084766 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278032440 con 0x7fd2900b1370 2024-09-13T17:59:23.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.127+0000 7fd28e7fc640 1 -- 172.21.15.111:0/1054084766 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2780325a0 con 0x7fd2900b1370 2024-09-13T17:59:23.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.127+0000 7fd28e7fc640 1 -- 172.21.15.111:0/1054084766 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278032930 con 0x7fd2900b1370 2024-09-13T17:59:23.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.127+0000 7fd295765640 1 -- 172.21.15.111:0/1054084766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2900b1370 msgr2=0x7fd2900d2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.127+0000 7fd295765640 1 --2- 172.21.15.111:0/1054084766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2900b1370 0x7fd2900d2f50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd278009870 tx=0x7fd27802a2a0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.128+0000 7fd295765640 1 -- 172.21.15.111:0/1054084766 shutdown_connections 2024-09-13T17:59:23.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.128+0000 7fd295765640 1 --2- 172.21.15.111:0/1054084766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2900b1370 0x7fd2900d2f50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.128+0000 7fd295765640 1 --2- 172.21.15.111:0/1054084766 >> v2:172.21.15.188:3300/0 conn(0x7fd2900cd500 0x7fd2900b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.128+0000 7fd295765640 1 -- 172.21.15.111:0/1054084766 >> 172.21.15.111:0/1054084766 conn(0x7fd290089f60 msgr2=0x7fd2900d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:23.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.128+0000 7fd295765640 1 -- 172.21.15.111:0/1054084766 shutdown_connections 2024-09-13T17:59:23.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.128+0000 7fd295765640 1 -- 172.21.15.111:0/1054084766 wait complete. 2024-09-13T17:59:23.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.129+0000 7fd295765640 1 Processor -- start 2024-09-13T17:59:23.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.129+0000 7fd295765640 1 -- start start 2024-09-13T17:59:23.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.129+0000 7fd295765640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2900cd500 0x7fd29015bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:23.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.129+0000 7fd295765640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2900d1650 0x7fd29014cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:23.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.129+0000 7fd295765640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2901f6790 0x7fd2901f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:23.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.129+0000 7fd295765640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2900b5300 con 0x7fd2900d1650 2024-09-13T17:59:23.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.129+0000 7fd295765640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2900b5550 con 0x7fd2900cd500 2024-09-13T17:59:23.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.129+0000 7fd295765640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29015afe0 con 0x7fd2901f6790 2024-09-13T17:59:23.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.130+0000 7fd287fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2900d1650 0x7fd29014cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:23.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.130+0000 7fd287fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2900d1650 0x7fd29014cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53538/0 (socket says 172.21.15.111:53538) 2024-09-13T17:59:23.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.130+0000 7fd287fff640 1 -- 172.21.15.111:0/1534459466 learned_addr learned my addr 172.21.15.111:0/1534459466 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:23.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.130+0000 7fd28effd640 1 --2- 172.21.15.111:0/1534459466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2900cd500 0x7fd29015bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:23.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.130+0000 7fd28f7fe640 1 --2- 172.21.15.111:0/1534459466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2901f6790 0x7fd2901f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:23.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.130+0000 7fd287fff640 1 -- 172.21.15.111:0/1534459466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2901f6790 msgr2=0x7fd2901f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.130+0000 7fd287fff640 1 --2- 172.21.15.111:0/1534459466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2901f6790 0x7fd2901f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.130+0000 7fd287fff640 1 -- 172.21.15.111:0/1534459466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2900cd500 msgr2=0x7fd29015bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.130+0000 7fd287fff640 1 --2- 172.21.15.111:0/1534459466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2900cd500 0x7fd29015bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.130+0000 7fd287fff640 1 -- 172.21.15.111:0/1534459466 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2780094d0 con 0x7fd2900d1650 2024-09-13T17:59:23.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.130+0000 7fd28f7fe640 1 --2- 172.21.15.111:0/1534459466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2901f6790 0x7fd2901f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:23.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.130+0000 7fd287fff640 1 --2- 172.21.15.111:0/1534459466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2900d1650 0x7fd29014cab0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd278000c00 tx=0x7fd2780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:23.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.130+0000 7fd2877fe640 1 -- 172.21.15.111:0/1534459466 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278032440 con 0x7fd2900d1650 2024-09-13T17:59:23.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.130+0000 7fd2877fe640 1 -- 172.21.15.111:0/1534459466 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd278030b80 con 0x7fd2900d1650 2024-09-13T17:59:23.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.130+0000 7fd295765640 1 -- 172.21.15.111:0/1534459466 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29015b2c0 con 0x7fd2900d1650 2024-09-13T17:59:23.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.130+0000 7fd2877fe640 1 -- 172.21.15.111:0/1534459466 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27803d440 con 0x7fd2900d1650 2024-09-13T17:59:23.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.130+0000 7fd295765640 1 -- 172.21.15.111:0/1534459466 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29014cff0 con 0x7fd2900d1650 2024-09-13T17:59:23.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.131+0000 7fd295765640 1 -- 172.21.15.111:0/1534459466 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2900b2510 con 0x7fd2900d1650 2024-09-13T17:59:23.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.132+0000 7fd2877fe640 1 -- 172.21.15.111:0/1534459466 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd278039040 con 0x7fd2900d1650 2024-09-13T17:59:23.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.132+0000 7fd2877fe640 1 --2- 172.21.15.111:0/1534459466 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd27006ac30 0x7fd27006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:23.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.132+0000 7fd2877fe640 1 -- 172.21.15.111:0/1534459466 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fd2780a9c40 con 0x7fd2900d1650 2024-09-13T17:59:23.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.132+0000 7fd28effd640 1 --2- 172.21.15.111:0/1534459466 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd27006ac30 0x7fd27006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:23.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.133+0000 7fd28effd640 1 --2- 172.21.15.111:0/1534459466 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd27006ac30 0x7fd27006d0f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd280006fd0 tx=0x7fd280008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:23.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.136+0000 7fd2877fe640 1 -- 172.21.15.111:0/1534459466 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fd278074f80 con 0x7fd2900d1650 2024-09-13T17:59:23.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.298+0000 7fd295765640 1 -- 172.21.15.111:0/1534459466 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2900ce130 con 0x7fd27006ac30 2024-09-13T17:59:23.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.299+0000 7fd2877fe640 1 -- 172.21.15.111:0/1534459466 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd2900ce130 con 0x7fd27006ac30 2024-09-13T17:59:23.301 INFO:teuthology.orchestra.run.smithi111.stderr:instructed osd(s) 7 to deep-scrub 2024-09-13T17:59:23.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.300+0000 7fd295765640 1 -- 172.21.15.111:0/1534459466 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd27006ac30 msgr2=0x7fd27006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.300+0000 7fd295765640 1 --2- 172.21.15.111:0/1534459466 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd27006ac30 0x7fd27006d0f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd280006fd0 tx=0x7fd280008040 comp rx=0 tx=0).stop 2024-09-13T17:59:23.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.300+0000 7fd295765640 1 -- 172.21.15.111:0/1534459466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2900d1650 msgr2=0x7fd29014cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.300+0000 7fd295765640 1 --2- 172.21.15.111:0/1534459466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2900d1650 0x7fd29014cab0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd278000c00 tx=0x7fd2780046c0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.301+0000 7fd295765640 1 -- 172.21.15.111:0/1534459466 shutdown_connections 2024-09-13T17:59:23.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.301+0000 7fd295765640 1 --2- 172.21.15.111:0/1534459466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2901f6790 0x7fd2901f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.301+0000 7fd295765640 1 --2- 172.21.15.111:0/1534459466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2900d1650 0x7fd29014cab0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.301+0000 7fd295765640 1 --2- 172.21.15.111:0/1534459466 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fd27006ac30 0x7fd27006d0f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.301+0000 7fd295765640 1 --2- 172.21.15.111:0/1534459466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2900cd500 0x7fd29015bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.301+0000 7fd295765640 1 -- 172.21.15.111:0/1534459466 >> 172.21.15.111:0/1534459466 conn(0x7fd290089f60 msgr2=0x7fd29006c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:23.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.301+0000 7fd295765640 1 -- 172.21.15.111:0/1534459466 shutdown_connections 2024-09-13T17:59:23.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.301+0000 7fd295765640 1 -- 172.21.15.111:0/1534459466 wait complete. 2024-09-13T17:59:23.313 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:59:23.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.408+0000 7fbd4315a640 1 Processor -- start 2024-09-13T17:59:23.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.408+0000 7fbd4315a640 1 -- start start 2024-09-13T17:59:23.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.409+0000 7fbd4315a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbd3c15c6e0 0x7fbd3c0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:23.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.409+0000 7fbd4315a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd3c0d07b0 0x7fbd3c0b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:23.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.409+0000 7fbd4315a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3c0d1840 con 0x7fbd3c0ccb20 2024-09-13T17:59:23.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.409+0000 7fbd4315a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd3c06bd20 con 0x7fbd3c15c6e0 2024-09-13T17:59:23.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.409+0000 7fbd4315a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c0d0c30 con 0x7fbd3c0d07b0 2024-09-13T17:59:23.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.409+0000 7fbd40ef6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbd3c15c6e0 0x7fbd3c0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:23.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.409+0000 7fbd416f7640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbd3c0ccb20 0x7fbd3c160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48924/0 (socket says 172.21.15.111:48924) 2024-09-13T17:59:23.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.409+0000 7fbd416f7640 1 -- 172.21.15.111:0/1504071455 learned_addr learned my addr 172.21.15.111:0/1504071455 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:23.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.409+0000 7fbd40ef6640 1 -- 172.21.15.111:0/1504071455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd3c0d07b0 msgr2=0x7fbd3c0b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:23.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.409+0000 7fbd3bfff640 1 --2- 172.21.15.111:0/1504071455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd3c0d07b0 0x7fbd3c0b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:23.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.409+0000 7fbd3b7fe640 1 -- 172.21.15.111:0/1504071455 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615691067 0 0) 0x7fbd3c0d1840 con 0x7fbd3c0ccb20 2024-09-13T17:59:23.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.409+0000 7fbd40ef6640 1 --2- 172.21.15.111:0/1504071455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd3c0d07b0 0x7fbd3c0b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.409+0000 7fbd40ef6640 1 -- 172.21.15.111:0/1504071455 >> v1:172.21.15.111:6789/0 conn(0x7fbd3c0ccb20 legacy=0x7fbd3c160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.409+0000 7fbd40ef6640 1 -- 172.21.15.111:0/1504071455 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c0d0f10 con 0x7fbd3c15c6e0 2024-09-13T17:59:23.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.410+0000 7fbd40ef6640 1 --2- 172.21.15.111:0/1504071455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd3c15c6e0 0x7fbd3c0b3f50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbd300097a0 tx=0x7fbd3002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a707069ae06bb0ad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:23.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.410+0000 7fbd3b7fe640 1 -- 172.21.15.111:0/1504071455 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3002ae50 con 0x7fbd3c15c6e0 2024-09-13T17:59:23.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.410+0000 7fbd3b7fe640 1 -- 172.21.15.111:0/1504071455 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd30032440 con 0x7fbd3c15c6e0 2024-09-13T17:59:23.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.410+0000 7fbd3b7fe640 1 -- 172.21.15.111:0/1504071455 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd300327d0 con 0x7fbd3c15c6e0 2024-09-13T17:59:23.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.411+0000 7fbd4315a640 1 -- 172.21.15.111:0/1504071455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd3c15c6e0 msgr2=0x7fbd3c0b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.411+0000 7fbd4315a640 1 --2- 172.21.15.111:0/1504071455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd3c15c6e0 0x7fbd3c0b3f50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbd300097a0 tx=0x7fbd3002a100 comp rx=0 tx=0).stop 2024-09-13T17:59:23.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.411+0000 7fbd4315a640 1 -- 172.21.15.111:0/1504071455 shutdown_connections 2024-09-13T17:59:23.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.411+0000 7fbd4315a640 1 --2- 172.21.15.111:0/1504071455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd3c0d07b0 0x7fbd3c0b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.411+0000 7fbd4315a640 1 --2- 172.21.15.111:0/1504071455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd3c15c6e0 0x7fbd3c0b3f50 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.411+0000 7fbd4315a640 1 -- 172.21.15.111:0/1504071455 >> 172.21.15.111:0/1504071455 conn(0x7fbd3c089f60 msgr2=0x7fbd3c0d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:23.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.411+0000 7fbd4315a640 1 -- 172.21.15.111:0/1504071455 shutdown_connections 2024-09-13T17:59:23.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.411+0000 7fbd4315a640 1 -- 172.21.15.111:0/1504071455 wait complete. 2024-09-13T17:59:23.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.412+0000 7fbd4315a640 1 Processor -- start 2024-09-13T17:59:23.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.412+0000 7fbd4315a640 1 -- start start 2024-09-13T17:59:23.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd4315a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd3c0ccb20 0x7fbd3c15a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:23.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd4315a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd3c0d07b0 0x7fbd3c15ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:23.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd4315a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd3c15c6e0 0x7fbd3c150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:23.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd4315a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c0b53a0 con 0x7fbd3c15c6e0 2024-09-13T17:59:23.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd4315a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c0ca130 con 0x7fbd3c0d07b0 2024-09-13T17:59:23.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd4315a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c0ca410 con 0x7fbd3c0ccb20 2024-09-13T17:59:23.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd416f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd3c15c6e0 0x7fbd3c150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:23.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd416f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd3c15c6e0 0x7fbd3c150540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53558/0 (socket says 172.21.15.111:53558) 2024-09-13T17:59:23.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd416f7640 1 -- 172.21.15.111:0/2250496037 learned_addr learned my addr 172.21.15.111:0/2250496037 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:23.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd40ef6640 1 --2- 172.21.15.111:0/2250496037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd3c0ccb20 0x7fbd3c15a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:23.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd3bfff640 1 --2- 172.21.15.111:0/2250496037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd3c0d07b0 0x7fbd3c15ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:23.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd416f7640 1 -- 172.21.15.111:0/2250496037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd3c0ccb20 msgr2=0x7fbd3c15a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd416f7640 1 --2- 172.21.15.111:0/2250496037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd3c0ccb20 0x7fbd3c15a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd416f7640 1 -- 172.21.15.111:0/2250496037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd3c0d07b0 msgr2=0x7fbd3c15ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd416f7640 1 --2- 172.21.15.111:0/2250496037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd3c0d07b0 0x7fbd3c15ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd416f7640 1 -- 172.21.15.111:0/2250496037 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd30009400 con 0x7fbd3c15c6e0 2024-09-13T17:59:23.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd416f7640 1 --2- 172.21.15.111:0/2250496037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd3c15c6e0 0x7fbd3c150540 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400c700 tx=0x7fbd3400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:23.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd397fa640 1 -- 172.21.15.111:0/2250496037 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3400ce10 con 0x7fbd3c15c6e0 2024-09-13T17:59:23.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd397fa640 1 -- 172.21.15.111:0/2250496037 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3400a640 con 0x7fbd3c15c6e0 2024-09-13T17:59:23.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd4315a640 1 -- 172.21.15.111:0/2250496037 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3c0b5640 con 0x7fbd3c15c6e0 2024-09-13T17:59:23.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.413+0000 7fbd397fa640 1 -- 172.21.15.111:0/2250496037 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3400a9f0 con 0x7fbd3c15c6e0 2024-09-13T17:59:23.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.414+0000 7fbd4315a640 1 -- 172.21.15.111:0/2250496037 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3c0ca6f0 con 0x7fbd3c15c6e0 2024-09-13T17:59:23.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.414+0000 7fbd4315a640 1 -- 172.21.15.111:0/2250496037 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3c0b4c80 con 0x7fbd3c15c6e0 2024-09-13T17:59:23.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.415+0000 7fbd397fa640 1 -- 172.21.15.111:0/2250496037 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbd3400ab50 con 0x7fbd3c15c6e0 2024-09-13T17:59:23.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.415+0000 7fbd397fa640 1 --2- 172.21.15.111:0/2250496037 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbd2006ac00 0x7fbd2006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:23.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.416+0000 7fbd397fa640 1 -- 172.21.15.111:0/2250496037 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fbd340889b0 con 0x7fbd3c15c6e0 2024-09-13T17:59:23.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.416+0000 7fbd40ef6640 1 --2- 172.21.15.111:0/2250496037 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbd2006ac00 0x7fbd2006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:23.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.416+0000 7fbd40ef6640 1 --2- 172.21.15.111:0/2250496037 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbd2006ac00 0x7fbd2006d0c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbd30034000 tx=0x7fbd3002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:23.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.420+0000 7fbd397fa640 1 -- 172.21.15.111:0/2250496037 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fbd34053cf0 con 0x7fbd3c15c6e0 2024-09-13T17:59:23.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.578+0000 7fbd4315a640 1 -- 172.21.15.111:0/2250496037 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbd3c0d14d0 con 0x7fbd2006ac00 2024-09-13T17:59:23.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.580+0000 7fbd397fa640 1 -- 172.21.15.111:0/2250496037 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51302 (secure 0 0 0) 0x7fbd30032a80 con 0x7fbd2006ac00 2024-09-13T17:59:23.583 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:23.583 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-09-13T17:59:23.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.583+0000 7fbd4315a640 1 -- 172.21.15.111:0/2250496037 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbd2006ac00 msgr2=0x7fbd2006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.583+0000 7fbd4315a640 1 --2- 172.21.15.111:0/2250496037 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbd2006ac00 0x7fbd2006d0c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbd30034000 tx=0x7fbd3002e000 comp rx=0 tx=0).stop 2024-09-13T17:59:23.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.583+0000 7fbd4315a640 1 -- 172.21.15.111:0/2250496037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd3c15c6e0 msgr2=0x7fbd3c150540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:23.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.583+0000 7fbd4315a640 1 --2- 172.21.15.111:0/2250496037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd3c15c6e0 0x7fbd3c150540 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400c700 tx=0x7fbd3400cbc0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.583+0000 7fbd4315a640 1 -- 172.21.15.111:0/2250496037 shutdown_connections 2024-09-13T17:59:23.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.583+0000 7fbd4315a640 1 --2- 172.21.15.111:0/2250496037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd3c15c6e0 0x7fbd3c150540 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.583+0000 7fbd4315a640 1 --2- 172.21.15.111:0/2250496037 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7fbd2006ac00 0x7fbd2006d0c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.583+0000 7fbd4315a640 1 --2- 172.21.15.111:0/2250496037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd3c0d07b0 0x7fbd3c15ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.583+0000 7fbd4315a640 1 --2- 172.21.15.111:0/2250496037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd3c0ccb20 0x7fbd3c15a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:23.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.583+0000 7fbd4315a640 1 -- 172.21.15.111:0/2250496037 >> 172.21.15.111:0/2250496037 conn(0x7fbd3c089f60 msgr2=0x7fbd3c0b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:23.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.583+0000 7fbd4315a640 1 -- 172.21.15.111:0/2250496037 shutdown_connections 2024-09-13T17:59:23.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:23.583+0000 7fbd4315a640 1 -- 172.21.15.111:0/2250496037 wait complete. 2024-09-13T17:59:23.595 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-09-13T17:59:22.289898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":221504,"kb_used_data":5316,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753216,"statfs":{"total":773094113280,"available":772867293184,"internally_reserved":0,"allocated":5443584,"data_stored":3037181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14667,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-86,"num_read_kb":-88,"num_write":-74,"num_write_kb":-39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001367"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T17:59:16.053069+0000","last_change":"2024-09-13T17:57:15.560506+0000","last_active":"2024-09-13T17:59:16.053069+0000","last_peered":"2024-09-13T17:59:16.053069+0000","last_clean":"2024-09-13T17:59:16.053069+0000","last_became_active":"2024-09-13T17:57:15.559612+0000","last_became_peered":"2024-09-13T17:57:15.559612+0000","last_unstale":"2024-09-13T17:59:16.053069+0000","last_undegraded":"2024-09-13T17:59:16.053069+0000","last_fullsized":"2024-09-13T17:59:16.053069+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:57:14.317378+0000","last_clean_scrub_stamp":"2024-09-13T17:57:14.317378+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:16:27.663312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343716,"statfs":{"total":96636764160,"available":96607965184,"internally_reserved":0,"allocated":1126400,"data_stored":822551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72499999999999998,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64000000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71199999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67500000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.48099999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66600000000000004,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70199999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41799999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55800000000000005,"15min":0.56100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.748,"5min":0.879,"15min":0.879},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72299999999999998,"5min":1.034,"15min":1.034},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.629},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.629,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.60399999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69599999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55800000000000005,"15min":0.56399999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71799999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.59199999999999997}]},{"osd":6,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.61399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":232183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.41499999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70999999999999996,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.375,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.53600000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.38700000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90200000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.502},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53400000000000003,"15min":0.53300000000000003},"min":{"1min":0.41499999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69599999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.51400000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.40100000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.73299999999999998,"5min":1.194,"15min":1.194},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73599999999999999,"5min":1.002,"15min":1.002},"last":0.443}]},{"osd":3,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77500000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56000000000000005,"15min":0.56100000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0800000000000001,"5min":1.159,"15min":1.159},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.33100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.376},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.35799999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.35799999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.52500000000000002,"15min":0.501},"min":{"1min":0.33800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":4.2210000000000001,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.31900000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.39600000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.433},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55100000000000005,"15min":0.55600000000000005},"min":{"1min":0.35699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.52600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":232183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.376,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65700000000000003,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51500000000000001,"15min":0.51900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60599999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.40500000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51400000000000001,"15min":0.50900000000000001},"min":{"1min":0.38900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":0.745},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.39500000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.56000000000000005}]},{"osd":2,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52500000000000002,"15min":0.52400000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.64600000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.42699999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.66300000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.66000000000000003}]},{"osd":3,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.44500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.63100000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.48799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.70999999999999996,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67800000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35899999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.51800000000000002}]},{"osd":6,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.66500000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54800000000000004,"15min":0.55100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68200000000000005,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.48399999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.40799999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85799999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.70999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":232183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35199999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.37,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.51400000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71599999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.39300000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.64300000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.40500000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61299999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53800000000000003,"15min":0.53500000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.53000000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55200000000000005,"15min":0.55500000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.67700000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.58699999999999997}]},{"osd":6,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65800000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76900000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.64800000000000002,"15min":0.65700000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86299999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.626}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":232183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.32900000000000001,"15min":0.32700000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.246},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.32100000000000001,"15min":0.317},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.33900000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.33500000000000002,"15min":0.33400000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.50700000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.35799999999999998,"15min":0.35499999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.28299999999999997}]},{"osd":2,"last update":"Fri Sep 13 17:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.38,"15min":0.38},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.376},{"interface":"front","average":{"1min":0.432,"5min":0.40699999999999997,"15min":0.40300000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.33000000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54700000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79600000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.623},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.50800000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.41499999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80000000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.499}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":232183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.32200000000000001,"15min":0.32600000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.42999999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.34499999999999997,"15min":0.34699999999999998},"min":{"1min":0.252,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.49399999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.28699999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.33400000000000002,"15min":0.33600000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.46400000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.248,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.59199999999999997}]},{"osd":3,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.372,"15min":0.374},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.54000000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.375},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39900000000000002,"15min":0.39800000000000002},"min":{"1min":0.29099999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.79200000000000004}]},{"osd":4,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.42999999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75900000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79500000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.39000000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81299999999999994,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.69999999999999996}]},{"osd":6,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59999999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78100000000000003,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.42999999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81699999999999995,"5min":0.998,"15min":0.998},"last":0.72199999999999998}]},{"osd":7,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82999999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.76400000000000001,"5min":0.998,"15min":0.998},"last":0.67100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":232183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.28499999999999998,"5min":0.34200000000000003,"15min":0.35199999999999998},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.35999999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.316,"5min":0.35699999999999998,"15min":0.36399999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.47199999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.33700000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.34100000000000003,"15min":0.34599999999999997},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.38800000000000001,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.318},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.29099999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.443,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.40600000000000003}]},{"osd":3,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.40699999999999997,"15min":0.41299999999999998},"min":{"1min":0.27700000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.52700000000000002,"5min":0.88,"15min":0.88},"last":0.375},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.40999999999999998,"15min":0.42099999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.44900000000000001,"5min":1.004,"15min":1.004},"last":0.29299999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70899999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.504},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76400000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.51700000000000002}]},{"osd":5,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.753,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70999999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.43099999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78500000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.68899999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.47699999999999998}]},{"osd":7,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81499999999999995,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.42399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69899999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.55000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":821532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.30399999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.318},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.34499999999999997}]},{"osd":3,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.35799999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.52700000000000002}]},{"osd":5,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51100000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.56200000000000006}]},{"osd":7,"last update":"Fri Sep 13 17:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.44800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:59:23.600 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T17:57:14.317378+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=57, tm_sec=14, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=59, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:59:23.600 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T17:59:26.749 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:59:26.752 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:59:43.601 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:59:43.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.710+0000 7f52dffb5640 1 Processor -- start 2024-09-13T17:59:43.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.710+0000 7f52dffb5640 1 -- start start 2024-09-13T17:59:43.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.711+0000 7f52dffb5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f52d8071d30 0x7f52d8070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:43.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.711+0000 7f52dffb5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52d80c8920 0x7f52d80c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:43.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.711+0000 7f52dffb5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d80abdf0 con 0x7f52d8072110 2024-09-13T17:59:43.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.711+0000 7f52dffb5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d80c92f0 con 0x7f52d80c8920 2024-09-13T17:59:43.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.711+0000 7f52dffb5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d80b2280 con 0x7f52d8071d30 2024-09-13T17:59:43.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.711+0000 7f52de552640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52d80c8920 0x7f52d80c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:43.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.711+0000 7f52dd550640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f52d8072110 0x7f52d80b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56020/0 (socket says 172.21.15.111:56020) 2024-09-13T17:59:43.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.711+0000 7f52dd550640 1 -- 172.21.15.111:0/877619831 learned_addr learned my addr 172.21.15.111:0/877619831 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:43.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.712+0000 7f52ddd51640 1 --2- 172.21.15.111:0/877619831 >> v2:172.21.15.188:3300/0 conn(0x7f52d8071d30 0x7f52d8070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:43.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.712+0000 7f52de552640 1 -- 172.21.15.111:0/877619831 >> v2:172.21.15.188:3300/0 conn(0x7f52d8071d30 msgr2=0x7f52d8070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:43.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.712+0000 7f52de552640 1 --2- 172.21.15.111:0/877619831 >> v2:172.21.15.188:3300/0 conn(0x7f52d8071d30 0x7f52d8070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:43.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.712+0000 7f52de552640 1 -- 172.21.15.111:0/877619831 >> v1:172.21.15.111:6789/0 conn(0x7f52d8072110 legacy=0x7f52d80b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:43.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.712+0000 7f52de552640 1 -- 172.21.15.111:0/877619831 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d80ab1e0 con 0x7f52d80c8920 2024-09-13T17:59:43.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.712+0000 7f52ddd51640 1 --2- 172.21.15.111:0/877619831 >> v2:172.21.15.188:3300/0 conn(0x7f52d8071d30 0x7f52d8070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:59:43.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.712+0000 7f52de552640 1 --2- 172.21.15.111:0/877619831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52d80c8920 0x7f52d80c8db0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f52d80cee60 tx=0x7f52d002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ed2192f6ba9bcc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:43.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.712+0000 7f52dcd4f640 1 -- 172.21.15.111:0/877619831 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d002aea0 con 0x7f52d80c8920 2024-09-13T17:59:43.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.713+0000 7f52dcd4f640 1 -- 172.21.15.111:0/877619831 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d002d050 con 0x7f52d80c8920 2024-09-13T17:59:43.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.713+0000 7f52dcd4f640 1 -- 172.21.15.111:0/877619831 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d0007960 con 0x7f52d80c8920 2024-09-13T17:59:43.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.713+0000 7f52dffb5640 1 -- 172.21.15.111:0/877619831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52d80c8920 msgr2=0x7f52d80c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:43.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.713+0000 7f52dffb5640 1 --2- 172.21.15.111:0/877619831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52d80c8920 0x7f52d80c8db0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f52d80cee60 tx=0x7f52d002a680 comp rx=0 tx=0).stop 2024-09-13T17:59:43.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.713+0000 7f52dffb5640 1 -- 172.21.15.111:0/877619831 shutdown_connections 2024-09-13T17:59:43.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.713+0000 7f52dffb5640 1 --2- 172.21.15.111:0/877619831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52d80c8920 0x7f52d80c8db0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:43.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.713+0000 7f52dffb5640 1 --2- 172.21.15.111:0/877619831 >> v2:172.21.15.188:3300/0 conn(0x7f52d8071d30 0x7f52d8070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:43.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.713+0000 7f52dffb5640 1 -- 172.21.15.111:0/877619831 >> 172.21.15.111:0/877619831 conn(0x7f52d8069ed0 msgr2=0x7f52d806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:43.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.713+0000 7f52dffb5640 1 -- 172.21.15.111:0/877619831 shutdown_connections 2024-09-13T17:59:43.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.713+0000 7f52dffb5640 1 -- 172.21.15.111:0/877619831 wait complete. 2024-09-13T17:59:43.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.714+0000 7f52dffb5640 1 Processor -- start 2024-09-13T17:59:43.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.714+0000 7f52dffb5640 1 -- start start 2024-09-13T17:59:43.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.715+0000 7f52dffb5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52d8071d30 0x7f52d814e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:43.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.715+0000 7f52dffb5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52d8072110 0x7f52d80cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:43.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.715+0000 7f52dffb5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52d80c8920 0x7f52d814d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:43.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.715+0000 7f52dffb5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d814d170 con 0x7f52d8071d30 2024-09-13T17:59:43.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.715+0000 7f52dffb5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d814ef10 con 0x7f52d8072110 2024-09-13T17:59:43.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.715+0000 7f52dffb5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d814c4f0 con 0x7f52d80c8920 2024-09-13T17:59:43.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.715+0000 7f52ddd51640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52d8071d30 0x7f52d814e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:43.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.715+0000 7f52ddd51640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52d8071d30 0x7f52d814e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34412/0 (socket says 172.21.15.111:34412) 2024-09-13T17:59:43.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.715+0000 7f52ddd51640 1 -- 172.21.15.111:0/2983883439 learned_addr learned my addr 172.21.15.111:0/2983883439 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:43.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.716+0000 7f52ddd51640 1 -- 172.21.15.111:0/2983883439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52d80c8920 msgr2=0x7f52d814d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:43.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.716+0000 7f52de552640 1 --2- 172.21.15.111:0/2983883439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52d80c8920 0x7f52d814d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:43.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.716+0000 7f52dd550640 1 --2- 172.21.15.111:0/2983883439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52d8072110 0x7f52d80cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:43.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.716+0000 7f52ddd51640 1 --2- 172.21.15.111:0/2983883439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52d80c8920 0x7f52d814d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:43.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.716+0000 7f52ddd51640 1 -- 172.21.15.111:0/2983883439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52d8072110 msgr2=0x7f52d80cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:43.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.716+0000 7f52ddd51640 1 --2- 172.21.15.111:0/2983883439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52d8072110 0x7f52d80cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:43.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.716+0000 7f52ddd51640 1 -- 172.21.15.111:0/2983883439 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d0009a70 con 0x7f52d8071d30 2024-09-13T17:59:43.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.716+0000 7f52dd550640 1 --2- 172.21.15.111:0/2983883439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52d8072110 0x7f52d80cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:59:43.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.716+0000 7f52ddd51640 1 --2- 172.21.15.111:0/2983883439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52d8071d30 0x7f52d814e9d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f52d400c7f0 tx=0x7f52d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:43.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.716+0000 7f52ce7fc640 1 -- 172.21.15.111:0/2983883439 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d4007a30 con 0x7f52d8071d30 2024-09-13T17:59:43.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.716+0000 7f52ce7fc640 1 -- 172.21.15.111:0/2983883439 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d4007b90 con 0x7f52d8071d30 2024-09-13T17:59:43.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.716+0000 7f52ce7fc640 1 -- 172.21.15.111:0/2983883439 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400ce40 con 0x7f52d8071d30 2024-09-13T17:59:43.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.716+0000 7f52dffb5640 1 -- 172.21.15.111:0/2983883439 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d814c7d0 con 0x7f52d8071d30 2024-09-13T17:59:43.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.716+0000 7f52dffb5640 1 -- 172.21.15.111:0/2983883439 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d814cd90 con 0x7f52d8071d30 2024-09-13T17:59:43.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.717+0000 7f52dffb5640 1 -- 172.21.15.111:0/2983883439 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d8070190 con 0x7f52d8071d30 2024-09-13T17:59:43.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.717+0000 7f52ce7fc640 1 -- 172.21.15.111:0/2983883439 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f52d40113f0 con 0x7f52d8071d30 2024-09-13T17:59:43.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.718+0000 7f52ce7fc640 1 --2- 172.21.15.111:0/2983883439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f52bc06ac00 0x7f52bc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:43.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.718+0000 7f52ce7fc640 1 -- 172.21.15.111:0/2983883439 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f52d40893d0 con 0x7f52d8071d30 2024-09-13T17:59:43.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.718+0000 7f52dd550640 1 --2- 172.21.15.111:0/2983883439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f52bc06ac00 0x7f52bc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:43.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.718+0000 7f52dd550640 1 --2- 172.21.15.111:0/2983883439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f52bc06ac00 0x7f52bc06d0c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f52d815c000 tx=0x7f52c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:43.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.722+0000 7f52ce7fc640 1 -- 172.21.15.111:0/2983883439 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f52d4054710 con 0x7f52d8071d30 2024-09-13T17:59:43.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.881+0000 7f52dffb5640 1 -- 172.21.15.111:0/2983883439 --> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f52d80aba80 con 0x7f52bc06ac00 2024-09-13T17:59:43.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.883+0000 7f52ce7fc640 1 -- 172.21.15.111:0/2983883439 <== mgr.4101 v2:172.21.15.111:6832/2183469065 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51473 (secure 0 0 0) 0x7f52c8001ce0 con 0x7f52bc06ac00 2024-09-13T17:59:43.885 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:43.886 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-09-13T17:59:43.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.885+0000 7f52dffb5640 1 -- 172.21.15.111:0/2983883439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f52bc06ac00 msgr2=0x7f52bc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:43.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.885+0000 7f52dffb5640 1 --2- 172.21.15.111:0/2983883439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f52bc06ac00 0x7f52bc06d0c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f52d815c000 tx=0x7f52c8009040 comp rx=0 tx=0).stop 2024-09-13T17:59:43.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.885+0000 7f52dffb5640 1 -- 172.21.15.111:0/2983883439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52d8071d30 msgr2=0x7f52d814e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:43.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.885+0000 7f52dffb5640 1 --2- 172.21.15.111:0/2983883439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52d8071d30 0x7f52d814e9d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f52d400c7f0 tx=0x7f52d400cbf0 comp rx=0 tx=0).stop 2024-09-13T17:59:43.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.885+0000 7f52dffb5640 1 -- 172.21.15.111:0/2983883439 shutdown_connections 2024-09-13T17:59:43.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.885+0000 7f52dffb5640 1 --2- 172.21.15.111:0/2983883439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52d80c8920 0x7f52d814d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:43.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.885+0000 7f52dffb5640 1 --2- 172.21.15.111:0/2983883439 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f52bc06ac00 0x7f52bc06d0c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:43.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.885+0000 7f52dffb5640 1 --2- 172.21.15.111:0/2983883439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52d8072110 0x7f52d80cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:43.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.885+0000 7f52dffb5640 1 --2- 172.21.15.111:0/2983883439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52d8071d30 0x7f52d814e9d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:43.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.885+0000 7f52dffb5640 1 -- 172.21.15.111:0/2983883439 >> 172.21.15.111:0/2983883439 conn(0x7f52d8069ed0 msgr2=0x7f52d80ad2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:43.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.885+0000 7f52dffb5640 1 -- 172.21.15.111:0/2983883439 shutdown_connections 2024-09-13T17:59:43.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.885+0000 7f52dffb5640 1 -- 172.21.15.111:0/2983883439 wait complete. 2024-09-13T17:59:43.898 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-09-13T17:59:42.293486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":221492,"kb_used_data":5316,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753228,"statfs":{"total":773094113280,"available":772867305472,"internally_reserved":0,"allocated":5443584,"data_stored":3037181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002228"},"pg_stats":[{"pgid":"1.0","version":"45'119","reported_seq":204,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-13T17:59:23.582161+0000","last_change":"2024-09-13T17:59:23.582257+0000","last_active":"2024-09-13T17:59:23.582161+0000","last_peered":"2024-09-13T17:59:23.582161+0000","last_clean":"2024-09-13T17:59:23.582161+0000","last_became_active":"2024-09-13T17:57:15.559612+0000","last_became_peered":"2024-09-13T17:57:15.559612+0000","last_unstale":"2024-09-13T17:59:23.582161+0000","last_undegraded":"2024-09-13T17:59:23.582161+0000","last_fullsized":"2024-09-13T17:59:23.582161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'119","last_scrub_stamp":"2024-09-13T17:59:23.582136+0000","last_deep_scrub":"45'119","last_deep_scrub_stamp":"2024-09-13T17:59:23.582136+0000","last_clean_scrub_stamp":"2024-09-13T17:59:23.582136+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:37.277467+0000","scrub_duration":0.0048083470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1126400,"data_stored":822551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72499999999999998,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64000000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.55000000000000004}]},{"osd":1,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71199999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67500000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66600000000000004,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.505,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70199999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55800000000000005,"15min":0.56100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.748,"5min":0.879,"15min":0.879},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72299999999999998,"5min":1.034,"15min":1.034},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.629,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69599999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55800000000000005,"15min":0.56399999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71799999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:59:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.73999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":232183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.41499999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70999999999999996,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.375,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.38700000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90200000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.622},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53400000000000003,"15min":0.53300000000000003},"min":{"1min":0.41499999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69599999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.40100000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.73299999999999998,"5min":1.194,"15min":1.194},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73599999999999999,"5min":1.002,"15min":1.002},"last":0.54700000000000004}]},{"osd":3,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77500000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56000000000000005,"15min":0.56100000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0800000000000001,"5min":1.159,"15min":1.159},"last":0.58199999999999996}]},{"osd":4,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.33100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.35799999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.63600000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.52500000000000002,"15min":0.501},"min":{"1min":0.33800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":4.2210000000000001,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.495}]},{"osd":7,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.39600000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55100000000000005,"15min":0.55600000000000005},"min":{"1min":0.35699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.51400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":232183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.376,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65700000000000003,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51500000000000001,"15min":0.51900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60599999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51400000000000001,"15min":0.50900000000000001},"min":{"1min":0.38900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":0.745},"last":0.621},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.39500000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52500000000000002,"15min":0.52400000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.64600000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.42699999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.66300000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.64500000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.44500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.63100000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.48799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.70999999999999996,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.58599999999999997}]},{"osd":4,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67800000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35899999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.437}]},{"osd":6,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.66500000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54800000000000004,"15min":0.55100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68200000000000005,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.46700000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.40799999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85799999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.503}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":232183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35199999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.37,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71599999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.64300000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.40500000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.39700000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53800000000000003,"15min":0.53500000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.54100000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55200000000000005,"15min":0.55500000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.67700000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.59899999999999998}]},{"osd":6,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76900000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.64800000000000002,"15min":0.65700000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86299999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.61699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":232183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.32900000000000001,"15min":0.32700000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.32100000000000001,"15min":0.317},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.34200000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.33500000000000002,"15min":0.33400000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.50700000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.35799999999999998,"15min":0.35499999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.32900000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.38,"15min":0.38},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.40699999999999997,"15min":0.40300000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.39800000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.48199999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79600000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55800000000000005}]},{"osd":6,"last update":"Fri Sep 13 17:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54000000000000004}]},{"osd":7,"last update":"Fri Sep 13 17:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.41499999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80000000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":232183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.32200000000000001,"15min":0.32600000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.42999999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.34499999999999997,"15min":0.34699999999999998},"min":{"1min":0.252,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.49399999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.29799999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.33400000000000002,"15min":0.33600000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.46400000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.248,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.41099999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.372,"15min":0.374},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.54000000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.38},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39900000000000002,"15min":0.39800000000000002},"min":{"1min":0.29099999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.33500000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.42999999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75900000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79500000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46600000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81299999999999994,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.55000000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59999999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78100000000000003,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.42999999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81699999999999995,"5min":0.998,"15min":0.998},"last":0.58599999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82999999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.76400000000000001,"5min":0.998,"15min":0.998},"last":0.498}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":232183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.28499999999999998,"5min":0.34200000000000003,"15min":0.35199999999999998},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.35999999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.316,"5min":0.35699999999999998,"15min":0.36399999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.47199999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.38500000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.34100000000000003,"15min":0.34599999999999997},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.38800000000000001,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.317},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.29099999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.443,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.40300000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.40699999999999997,"15min":0.41299999999999998},"min":{"1min":0.27700000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.52700000000000002,"5min":0.88,"15min":0.88},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.40999999999999998,"15min":0.42099999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.44900000000000001,"5min":1.004,"15min":1.004},"last":0.34699999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70899999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76400000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.65500000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.753,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70999999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78500000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.68899999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Sep 13 17:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81499999999999995,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.42399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69899999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.624}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1122304,"data_stored":821532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:59:19 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39800000000000002,"15min":0.39200000000000002},"min":{"1min":0.22700000000000001,"5min":0.215,"15min":0.215},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.38800000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.083,"5min":1.266,"15min":1.266},"last":0.29199999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:59:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.40899999999999997,"15min":0.41099999999999998},"min":{"1min":0.253,"5min":0.246,"15min":0.246},"max":{"1min":0.86099999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41999999999999998,"15min":0.41299999999999998},"min":{"1min":0.28199999999999997,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.36699999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:59:19 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.38900000000000001,"15min":0.38200000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.45100000000000001},"min":{"1min":0.28799999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.35099999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:59:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.625,"15min":0.61099999999999999},"min":{"1min":0.45100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.72699999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:59:19 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.626},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Sep 13 17:59:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.58499999999999996,"15min":0.56999999999999995},"min":{"1min":0.42499999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.3980000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":0.753},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.623,"15min":0.61899999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.59399999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:59:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61899999999999999,"15min":0.61099999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.59999999999999998,"15min":0.58699999999999997},"min":{"1min":0.442,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.4319999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":0.60599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:59:43.899 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T17:59:43.996 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.994+0000 7f22c4ec9640 1 Processor -- start 2024-09-13T17:59:43.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.995+0000 7f22c4ec9640 1 -- start start 2024-09-13T17:59:43.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.995+0000 7f22c4ec9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f22c015c500 0x7f22c00d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:43.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.995+0000 7f22c4ec9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c00d2590 0x7f22c00cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:43.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.995+0000 7f22c4ec9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22c00cd450 con 0x7f22c00cde20 2024-09-13T17:59:43.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.995+0000 7f22c4ec9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c00d29f0 con 0x7f22c015c500 2024-09-13T17:59:43.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.995+0000 7f22c4ec9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c00d0cf0 con 0x7f22c00d2590 2024-09-13T17:59:43.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.996+0000 7f22bed9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f22c00cde20 0x7f22c00d0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56028/0 (socket says 172.21.15.111:56028) 2024-09-13T17:59:43.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.996+0000 7f22bed9d640 1 -- 172.21.15.111:0/288056746 learned_addr learned my addr 172.21.15.111:0/288056746 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:43.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.996+0000 7f22be59c640 1 --2- 172.21.15.111:0/288056746 >> v2:172.21.15.188:3300/0 conn(0x7f22c015c500 0x7f22c00d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:44.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.996+0000 7f22bdd9b640 1 --2- 172.21.15.111:0/288056746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c00d2590 0x7f22c00cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:44.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.996+0000 7f22bd59a640 1 -- 172.21.15.111:0/288056746 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3499996252 0 0) 0x7f22c00cd450 con 0x7f22c00cde20 2024-09-13T17:59:44.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.996+0000 7f22bd59a640 1 -- 172.21.15.111:0/288056746 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22a8003520 con 0x7f22c00cde20 2024-09-13T17:59:44.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.996+0000 7f22be59c640 1 -- 172.21.15.111:0/288056746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c00d2590 msgr2=0x7f22c00cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:44.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.996+0000 7f22be59c640 1 --2- 172.21.15.111:0/288056746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c00d2590 0x7f22c00cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.996+0000 7f22be59c640 1 -- 172.21.15.111:0/288056746 >> v1:172.21.15.111:6789/0 conn(0x7f22c00cde20 legacy=0x7f22c00d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:44.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.996+0000 7f22be59c640 1 -- 172.21.15.111:0/288056746 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c00d0fd0 con 0x7f22c015c500 2024-09-13T17:59:44.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.996+0000 7f22bdd9b640 1 --2- 172.21.15.111:0/288056746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c00d2590 0x7f22c00cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:59:44.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.997+0000 7f22be59c640 1 --2- 172.21.15.111:0/288056746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c015c500 0x7f22c00d2050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f22b00097a0 tx=0x7f22b002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d9c734e9e51715 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:44.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.997+0000 7f22bd59a640 1 -- 172.21.15.111:0/288056746 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0032440 con 0x7f22c015c500 2024-09-13T17:59:44.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.997+0000 7f22bd59a640 1 -- 172.21.15.111:0/288056746 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b00325a0 con 0x7f22c015c500 2024-09-13T17:59:44.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.997+0000 7f22bd59a640 1 -- 172.21.15.111:0/288056746 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0032910 con 0x7f22c015c500 2024-09-13T17:59:44.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.997+0000 7f22c4ec9640 1 -- 172.21.15.111:0/288056746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c015c500 msgr2=0x7f22c00d2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:44.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.997+0000 7f22c4ec9640 1 --2- 172.21.15.111:0/288056746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c015c500 0x7f22c00d2050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f22b00097a0 tx=0x7f22b002a180 comp rx=0 tx=0).stop 2024-09-13T17:59:44.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.998+0000 7f22c4ec9640 1 -- 172.21.15.111:0/288056746 shutdown_connections 2024-09-13T17:59:44.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.998+0000 7f22c4ec9640 1 --2- 172.21.15.111:0/288056746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c00d2590 0x7f22c00cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.998+0000 7f22c4ec9640 1 --2- 172.21.15.111:0/288056746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c015c500 0x7f22c00d2050 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.998+0000 7f22c4ec9640 1 -- 172.21.15.111:0/288056746 >> 172.21.15.111:0/288056746 conn(0x7f22c0089f60 msgr2=0x7f22c009b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:44.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.998+0000 7f22c4ec9640 1 -- 172.21.15.111:0/288056746 shutdown_connections 2024-09-13T17:59:44.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.998+0000 7f22c4ec9640 1 -- 172.21.15.111:0/288056746 wait complete. 2024-09-13T17:59:44.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.999+0000 7f22c4ec9640 1 Processor -- start 2024-09-13T17:59:44.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:43.999+0000 7f22c4ec9640 1 -- start start 2024-09-13T17:59:44.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22c4ec9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c00cde20 0x7f22c00b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:44.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22c4ec9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22c00d2590 0x7f22c01f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:44.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22c4ec9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c015c500 0x7f22c006b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:44.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22c4ec9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c01591f0 con 0x7f22c00d2590 2024-09-13T17:59:44.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22c4ec9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c00b8310 con 0x7f22c00cde20 2024-09-13T17:59:44.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22c4ec9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c01f1ce0 con 0x7f22c015c500 2024-09-13T17:59:44.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22bdd9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22c00d2590 0x7f22c01f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:44.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22bdd9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22c00d2590 0x7f22c01f17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34424/0 (socket says 172.21.15.111:34424) 2024-09-13T17:59:44.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22bdd9b640 1 -- 172.21.15.111:0/2124835848 learned_addr learned my addr 172.21.15.111:0/2124835848 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:44.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22be59c640 1 --2- 172.21.15.111:0/2124835848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c00cde20 0x7f22c00b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:44.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22bed9d640 1 --2- 172.21.15.111:0/2124835848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c015c500 0x7f22c006b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:44.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22bdd9b640 1 -- 172.21.15.111:0/2124835848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c015c500 msgr2=0x7f22c006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:44.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22bdd9b640 1 --2- 172.21.15.111:0/2124835848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c015c500 0x7f22c006b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22bdd9b640 1 -- 172.21.15.111:0/2124835848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c00cde20 msgr2=0x7f22c00b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:44.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22bdd9b640 1 --2- 172.21.15.111:0/2124835848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c00cde20 0x7f22c00b7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22bdd9b640 1 -- 172.21.15.111:0/2124835848 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b4009400 con 0x7f22c00d2590 2024-09-13T17:59:44.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22bed9d640 1 --2- 172.21.15.111:0/2124835848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c015c500 0x7f22c006b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:59:44.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22be59c640 1 --2- 172.21.15.111:0/2124835848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c00cde20 0x7f22c00b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:44.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22bdd9b640 1 --2- 172.21.15.111:0/2124835848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22c00d2590 0x7f22c01f17a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f22b400ebb0 tx=0x7f22b400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:44.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22aeffd640 1 -- 172.21.15.111:0/2124835848 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b400eeb0 con 0x7f22c00d2590 2024-09-13T17:59:44.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22aeffd640 1 -- 172.21.15.111:0/2124835848 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b400cc30 con 0x7f22c00d2590 2024-09-13T17:59:44.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22c4ec9640 1 -- 172.21.15.111:0/2124835848 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b0009400 con 0x7f22c00d2590 2024-09-13T17:59:44.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22aeffd640 1 -- 172.21.15.111:0/2124835848 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b4013470 con 0x7f22c00d2590 2024-09-13T17:59:44.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.000+0000 7f22c4ec9640 1 -- 172.21.15.111:0/2124835848 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c00af6f0 con 0x7f22c00d2590 2024-09-13T17:59:44.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.001+0000 7f22c4ec9640 1 -- 172.21.15.111:0/2124835848 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c00cd8e0 con 0x7f22c00d2590 2024-09-13T17:59:44.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.002+0000 7f22aeffd640 1 -- 172.21.15.111:0/2124835848 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f22b40135d0 con 0x7f22c00d2590 2024-09-13T17:59:44.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.003+0000 7f22aeffd640 1 --2- 172.21.15.111:0/2124835848 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f22a006ac00 0x7f22a006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:44.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.003+0000 7f22be59c640 1 --2- 172.21.15.111:0/2124835848 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f22a006ac00 0x7f22a006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:44.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.003+0000 7f22aeffd640 1 -- 172.21.15.111:0/2124835848 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f22b4089480 con 0x7f22c00d2590 2024-09-13T17:59:44.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.003+0000 7f22be59c640 1 --2- 172.21.15.111:0/2124835848 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f22a006ac00 0x7f22a006d0c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f22b0034000 tx=0x7f22b002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:44.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.007+0000 7f22aeffd640 1 -- 172.21.15.111:0/2124835848 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f22b4054850 con 0x7f22c00d2590 2024-09-13T17:59:44.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.166+0000 7f22c4ec9640 1 -- 172.21.15.111:0/2124835848 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f22c006b160 con 0x7f22c00d2590 2024-09-13T17:59:44.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.170+0000 7f22aeffd640 1 -- 172.21.15.111:0/2124835848 <== mon.0 v2:172.21.15.111:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f22b4054850 con 0x7f22c00d2590 2024-09-13T17:59:44.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.170+0000 7f22aeffd640 1 -- 172.21.15.111:0/2124835848 <== mon.0 v2:172.21.15.111:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f22b4057e70 con 0x7f22c00d2590 2024-09-13T17:59:44.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.172+0000 7f22c4ec9640 1 -- 172.21.15.111:0/2124835848 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f22a006ac00 msgr2=0x7f22a006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:44.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.172+0000 7f22c4ec9640 1 --2- 172.21.15.111:0/2124835848 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f22a006ac00 0x7f22a006d0c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f22b0034000 tx=0x7f22b002e000 comp rx=0 tx=0).stop 2024-09-13T17:59:44.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.172+0000 7f22c4ec9640 1 -- 172.21.15.111:0/2124835848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22c00d2590 msgr2=0x7f22c01f17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:44.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.172+0000 7f22c4ec9640 1 --2- 172.21.15.111:0/2124835848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22c00d2590 0x7f22c01f17a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f22b400ebb0 tx=0x7f22b400c520 comp rx=0 tx=0).stop 2024-09-13T17:59:44.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.173+0000 7f22c4ec9640 1 -- 172.21.15.111:0/2124835848 shutdown_connections 2024-09-13T17:59:44.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.173+0000 7f22c4ec9640 1 --2- 172.21.15.111:0/2124835848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c015c500 0x7f22c006b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.173+0000 7f22c4ec9640 1 --2- 172.21.15.111:0/2124835848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22c00d2590 0x7f22c01f17a0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.173+0000 7f22c4ec9640 1 --2- 172.21.15.111:0/2124835848 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f22a006ac00 0x7f22a006d0c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.173+0000 7f22c4ec9640 1 --2- 172.21.15.111:0/2124835848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c00cde20 0x7f22c00b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.173+0000 7f22c4ec9640 1 -- 172.21.15.111:0/2124835848 >> 172.21.15.111:0/2124835848 conn(0x7f22c0089f60 msgr2=0x7f22c00b24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:44.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.173+0000 7f22c4ec9640 1 -- 172.21.15.111:0/2124835848 shutdown_connections 2024-09-13T17:59:44.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.173+0000 7f22c4ec9640 1 -- 172.21.15.111:0/2124835848 wait complete. 2024-09-13T17:59:44.185 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T17:59:44.186 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:59:44.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.280+0000 7f143f75a640 1 Processor -- start 2024-09-13T17:59:44.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.280+0000 7f143f75a640 1 -- start start 2024-09-13T17:59:44.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.281+0000 7f143f75a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f143815c6e0 0x7f14380b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:44.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.281+0000 7f143f75a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f14380d07b0 0x7f14380b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:44.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.281+0000 7f143f75a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14380d1840 con 0x7f14380ccb20 2024-09-13T17:59:44.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.281+0000 7f143f75a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f143806bd20 con 0x7f14380d07b0 2024-09-13T17:59:44.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.281+0000 7f143f75a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14380d0c30 con 0x7f143815c6e0 2024-09-13T17:59:44.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.282+0000 7f143ccf5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f14380d07b0 0x7f14380b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:44.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.282+0000 7f143dcf7640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f14380ccb20 0x7f1438160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56036/0 (socket says 172.21.15.111:56036) 2024-09-13T17:59:44.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.282+0000 7f143dcf7640 1 -- 172.21.15.111:0/3746946539 learned_addr learned my addr 172.21.15.111:0/3746946539 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-09-13T17:59:44.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.282+0000 7f143ccf5640 1 -- 172.21.15.111:0/3746946539 >> v2:172.21.15.188:3300/0 conn(0x7f143815c6e0 msgr2=0x7f14380b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:44.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.282+0000 7f143ccf5640 1 --2- 172.21.15.111:0/3746946539 >> v2:172.21.15.188:3300/0 conn(0x7f143815c6e0 0x7f14380b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.282+0000 7f143ccf5640 1 -- 172.21.15.111:0/3746946539 >> v1:172.21.15.111:6789/0 conn(0x7f14380ccb20 legacy=0x7f1438160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:44.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.282+0000 7f143ccf5640 1 -- 172.21.15.111:0/3746946539 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14380d0f10 con 0x7f14380d07b0 2024-09-13T17:59:44.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.282+0000 7f142ffff640 1 -- 172.21.15.111:0/3746946539 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2279907301 0 0) 0x7f14380d1840 con 0x7f14380ccb20 2024-09-13T17:59:44.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.282+0000 7f143ccf5640 1 --2- 172.21.15.111:0/3746946539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14380d07b0 0x7f14380b4490 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f14380b4c10 tx=0x7f142802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4824e1b3ba62155 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:44.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.283+0000 7f142ffff640 1 -- 172.21.15.111:0/3746946539 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142802ae60 con 0x7f14380d07b0 2024-09-13T17:59:44.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.283+0000 7f142ffff640 1 -- 172.21.15.111:0/3746946539 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1428032440 con 0x7f14380d07b0 2024-09-13T17:59:44.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.283+0000 7f142ffff640 1 -- 172.21.15.111:0/3746946539 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14280327f0 con 0x7f14380d07b0 2024-09-13T17:59:44.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.283+0000 7f143f75a640 1 -- 172.21.15.111:0/3746946539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14380d07b0 msgr2=0x7f14380b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:44.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.283+0000 7f143f75a640 1 --2- 172.21.15.111:0/3746946539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14380d07b0 0x7f14380b4490 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f14380b4c10 tx=0x7f142802a110 comp rx=0 tx=0).stop 2024-09-13T17:59:44.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.283+0000 7f143f75a640 1 -- 172.21.15.111:0/3746946539 shutdown_connections 2024-09-13T17:59:44.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.283+0000 7f143f75a640 1 --2- 172.21.15.111:0/3746946539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14380d07b0 0x7f14380b4490 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.283+0000 7f143f75a640 1 --2- 172.21.15.111:0/3746946539 >> v2:172.21.15.188:3300/0 conn(0x7f143815c6e0 0x7f14380b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.283+0000 7f143f75a640 1 -- 172.21.15.111:0/3746946539 >> 172.21.15.111:0/3746946539 conn(0x7f1438089f60 msgr2=0x7f14380d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:44.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.283+0000 7f143f75a640 1 -- 172.21.15.111:0/3746946539 shutdown_connections 2024-09-13T17:59:44.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.283+0000 7f143f75a640 1 -- 172.21.15.111:0/3746946539 wait complete. 2024-09-13T17:59:44.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.284+0000 7f143f75a640 1 Processor -- start 2024-09-13T17:59:44.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.284+0000 7f143f75a640 1 -- start start 2024-09-13T17:59:44.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.285+0000 7f143f75a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14380ccb20 0x7f143815ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:44.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.285+0000 7f143f75a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14380d07b0 0x7f14380cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:44.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.285+0000 7f143f75a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f143815c6e0 0x7f14380cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:44.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.285+0000 7f143f75a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143815b150 con 0x7f143815c6e0 2024-09-13T17:59:44.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.285+0000 7f143f75a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14381e6d40 con 0x7f14380d07b0 2024-09-13T17:59:44.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.285+0000 7f143f75a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14381e7020 con 0x7f14380ccb20 2024-09-13T17:59:44.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.285+0000 7f143dcf7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f143815c6e0 0x7f14380cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:44.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.285+0000 7f143dcf7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f143815c6e0 0x7f14380cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34450/0 (socket says 172.21.15.111:34450) 2024-09-13T17:59:44.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.285+0000 7f143dcf7640 1 -- 172.21.15.111:0/1696174892 learned_addr learned my addr 172.21.15.111:0/1696174892 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T17:59:44.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.285+0000 7f143ccf5640 1 --2- 172.21.15.111:0/1696174892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14380d07b0 0x7f14380cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:44.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.285+0000 7f143d4f6640 1 --2- 172.21.15.111:0/1696174892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14380ccb20 0x7f143815ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:44.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.286+0000 7f143dcf7640 1 -- 172.21.15.111:0/1696174892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14380ccb20 msgr2=0x7f143815ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:44.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.286+0000 7f143dcf7640 1 --2- 172.21.15.111:0/1696174892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14380ccb20 0x7f143815ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.286+0000 7f143dcf7640 1 -- 172.21.15.111:0/1696174892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14380d07b0 msgr2=0x7f14380cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:44.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.286+0000 7f143dcf7640 1 --2- 172.21.15.111:0/1696174892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14380d07b0 0x7f14380cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.286+0000 7f143dcf7640 1 -- 172.21.15.111:0/1696174892 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1428009400 con 0x7f143815c6e0 2024-09-13T17:59:44.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.286+0000 7f143ccf5640 1 --2- 172.21.15.111:0/1696174892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14380d07b0 0x7f14380cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:44.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.286+0000 7f143dcf7640 1 --2- 172.21.15.111:0/1696174892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f143815c6e0 0x7f14380cb350 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1430001eb0 tx=0x7f143000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:44.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.286+0000 7f142dffb640 1 -- 172.21.15.111:0/1696174892 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14300074e0 con 0x7f143815c6e0 2024-09-13T17:59:44.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.286+0000 7f142dffb640 1 -- 172.21.15.111:0/1696174892 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1430007640 con 0x7f143815c6e0 2024-09-13T17:59:44.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.286+0000 7f143f75a640 1 -- 172.21.15.111:0/1696174892 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14381e7300 con 0x7f143815c6e0 2024-09-13T17:59:44.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.286+0000 7f142dffb640 1 -- 172.21.15.111:0/1696174892 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430004a20 con 0x7f143815c6e0 2024-09-13T17:59:44.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.286+0000 7f143f75a640 1 -- 172.21.15.111:0/1696174892 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14381e78c0 con 0x7f143815c6e0 2024-09-13T17:59:44.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.287+0000 7f143f75a640 1 -- 172.21.15.111:0/1696174892 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14380b4c80 con 0x7f143815c6e0 2024-09-13T17:59:44.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.287+0000 7f142dffb640 1 -- 172.21.15.111:0/1696174892 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1430004b80 con 0x7f143815c6e0 2024-09-13T17:59:44.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.288+0000 7f142dffb640 1 --2- 172.21.15.111:0/1696174892 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f141c06ac00 0x7f141c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:44.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.288+0000 7f142dffb640 1 -- 172.21.15.111:0/1696174892 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f1430089020 con 0x7f143815c6e0 2024-09-13T17:59:44.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.288+0000 7f143d4f6640 1 --2- 172.21.15.111:0/1696174892 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f141c06ac00 0x7f141c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:44.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.288+0000 7f143d4f6640 1 --2- 172.21.15.111:0/1696174892 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f141c06ac00 0x7f141c06d0c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1434009f20 tx=0x7f14340094f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:44.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.292+0000 7f142dffb640 1 -- 172.21.15.111:0/1696174892 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f1430057df0 con 0x7f143815c6e0 2024-09-13T17:59:44.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.487+0000 7f143f75a640 1 -- 172.21.15.111:0/1696174892 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f143808c610 con 0x7f143815c6e0 2024-09-13T17:59:44.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.488+0000 7f142dffb640 1 -- 172.21.15.111:0/1696174892 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f1430057980 con 0x7f143815c6e0 2024-09-13T17:59:44.490 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T17:59:44.490 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":23,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4217138481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4217138481},{"type":"v1","addr":"172.21.15.188:6834","nonce":4217138481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3099948287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3099948287},{"type":"v1","addr":"172.21.15.188:6835","nonce":3099948287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/904506495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":904506495},{"type":"v1","addr":"172.21.15.111:6837","nonce":904506495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5871,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2173844504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2173844504},{"type":"v1","addr":"172.21.15.111:6835","nonce":2173844504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[]} 2024-09-13T17:59:44.490 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 23 2024-09-13T17:59:44.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.490+0000 7f143f75a640 1 -- 172.21.15.111:0/1696174892 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f141c06ac00 msgr2=0x7f141c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:44.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.490+0000 7f143f75a640 1 --2- 172.21.15.111:0/1696174892 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f141c06ac00 0x7f141c06d0c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1434009f20 tx=0x7f14340094f0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.490+0000 7f143f75a640 1 -- 172.21.15.111:0/1696174892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f143815c6e0 msgr2=0x7f14380cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:44.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.490+0000 7f143f75a640 1 --2- 172.21.15.111:0/1696174892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f143815c6e0 0x7f14380cb350 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1430001eb0 tx=0x7f143000c870 comp rx=0 tx=0).stop 2024-09-13T17:59:44.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.490+0000 7f143f75a640 1 -- 172.21.15.111:0/1696174892 shutdown_connections 2024-09-13T17:59:44.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.490+0000 7f143f75a640 1 --2- 172.21.15.111:0/1696174892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f143815c6e0 0x7f14380cb350 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.490+0000 7f143f75a640 1 --2- 172.21.15.111:0/1696174892 >> [v2:172.21.15.111:6832/2183469065,v1:172.21.15.111:6833/2183469065] conn(0x7f141c06ac00 0x7f141c06d0c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.490+0000 7f143f75a640 1 --2- 172.21.15.111:0/1696174892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14380d07b0 0x7f14380cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.490+0000 7f143f75a640 1 --2- 172.21.15.111:0/1696174892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14380ccb20 0x7f143815ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:44.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.490+0000 7f143f75a640 1 -- 172.21.15.111:0/1696174892 >> 172.21.15.111:0/1696174892 conn(0x7f1438089f60 msgr2=0x7f143806c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:44.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.490+0000 7f143f75a640 1 -- 172.21.15.111:0/1696174892 shutdown_connections 2024-09-13T17:59:44.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T17:59:44.490+0000 7f143f75a640 1 -- 172.21.15.111:0/1696174892 wait complete. 2024-09-13T17:59:44.503 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-13T17:59:44.503 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T17:59:44.503 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:59:44.503 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:44.512 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:59:44.512 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T17:59:44.512 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:44.562 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T17:59:44.562 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T17:59:44.562 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:44.574 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T17:59:44.574 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T17:59:44.575 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:44.626 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T17:59:44.627 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T17:59:44.627 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T17:59:44.627 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:44.654 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T17:59:44.654 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T17:59:44.654 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:44.718 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T17:59:44.718 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T17:59:44.718 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:44.790 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T17:59:44.790 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T17:59:44.791 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:44.863 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T17:59:44.864 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-13T17:59:44.864 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:44.899 INFO:tasks.ceph.osd.4:Stopped 2024-09-13T17:59:44.900 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T17:59:44.900 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:44.969 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T17:59:44.970 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-13T17:59:44.970 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:45.040 INFO:tasks.ceph.osd.6:Stopped 2024-09-13T17:59:45.040 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-13T17:59:45.041 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:45.111 INFO:tasks.ceph.osd.7:Stopped 2024-09-13T17:59:45.112 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T17:59:45.112 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T17:59:45.112 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:45.139 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T17:59:45.139 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T17:59:45.139 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:45.180 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T17:59:45.181 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T17:59:45.181 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T17:59:45.181 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:45.192 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T17:59:45.193 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T17:59:45.193 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:45.233 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T17:59:45.233 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T17:59:45.233 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:45.286 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T17:59:45.286 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T17:59:45.286 DEBUG:teuthology.orchestra.run.smithi111:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'Replacing daemon mds.a' | head -n 1 2024-09-13T17:59:45.318 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T17:59:45.319 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T17:59:45.512 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T17:59:45.512 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T17:59:45.644 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T17:59:45.644 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T17:59:45.716 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T17:59:45.716 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T17:59:45.773 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi188.front.sepia.ceph.com 2024-09-13T17:59:45.774 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-13T17:59:45.926 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi188.front.sepia.ceph.com 2024-09-13T17:59:45.926 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-13T17:59:46.004 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi188.front.sepia.ceph.com 2024-09-13T17:59:46.005 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-13T17:59:46.053 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi188.front.sepia.ceph.com 2024-09-13T17:59:46.053 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-13T17:59:46.121 INFO:tasks.ceph:Archiving mon data... 2024-09-13T17:59:46.122 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903942/data/mon.a.tgz 2024-09-13T17:59:46.122 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2024-09-13T17:59:46.138 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.QhtJeB5LMW 2024-09-13T17:59:46.138 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.QhtJeB5LMW 2024-09-13T17:59:46.274 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.QhtJeB5LMW 2024-09-13T17:59:46.359 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.QhtJeB5LMW is 305KB 2024-09-13T17:59:46.428 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.QhtJeB5LMW 2024-09-13T17:59:46.443 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903942/data/mon.b.tgz 2024-09-13T17:59:46.443 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-09-13T17:59:46.462 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.IwSbNFUkPQ 2024-09-13T17:59:46.462 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.IwSbNFUkPQ 2024-09-13T17:59:46.621 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.IwSbNFUkPQ 2024-09-13T17:59:46.707 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.IwSbNFUkPQ is 409KB 2024-09-13T17:59:46.776 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.IwSbNFUkPQ 2024-09-13T17:59:46.792 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903942/data/mon.c.tgz 2024-09-13T17:59:46.792 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-09-13T17:59:46.847 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.90qkaCFsUi 2024-09-13T17:59:46.847 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.90qkaCFsUi 2024-09-13T17:59:46.989 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.90qkaCFsUi 2024-09-13T17:59:47.072 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.90qkaCFsUi is 409KB 2024-09-13T17:59:47.169 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.90qkaCFsUi 2024-09-13T17:59:47.184 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T17:59:47.184 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T17:59:47.187 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T17:59:47.248 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T17:59:47.248 DEBUG:teuthology.orchestra.run.smithi111:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T17:59:47.254 DEBUG:teuthology.orchestra.run.smithi188:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T17:59:47.281 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T17:59:47.315 INFO:teuthology.orchestra.run.smithi188.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T17:59:47.320 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T17:59:47.321 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903942/remote/smithi111/crash 2024-09-13T17:59:47.322 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T17:59:47.351 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903942/remote/smithi188/crash 2024-09-13T17:59:47.352 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T17:59:47.383 INFO:tasks.ceph:Shutting down logrotate 2024-09-13T17:59:47.383 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:59:47.396 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:59:47.689 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-13T17:59:47.714 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-13T17:59:47.740 INFO:tasks.ceph:Compressing logs... 2024-09-13T17:59:47.740 DEBUG:teuthology.orchestra.run.smithi111:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:59:47.757 DEBUG:teuthology.orchestra.run.smithi188:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:59:51.169 INFO:tasks.ceph:Archiving logs... 2024-09-13T17:59:51.170 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903942/remote/smithi111/log 2024-09-13T17:59:51.171 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T17:59:51.788 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903942/remote/smithi188/log 2024-09-13T17:59:51.789 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T17:59:52.190 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T17:59:52.199 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T17:59:52.199 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T17:59:52.202 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T17:59:52.307 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T17:59:52.316 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T17:59:52.316 DEBUG:teuthology.orchestra.run.smithi111:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:59:52.319 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:59:52.333 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-09-13T17:59:52.335 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-09-13T17:59:52.338 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T17:59:52.338 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-09-13T17:59:52.338 INFO:teuthology.orchestra.run.smithi111.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 -625us[ -614us] +/- 52ms 2024-09-13T17:59:52.338 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 30 +1798us[+1798us] +/- 60ms 2024-09-13T17:59:52.338 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 -1171us[-1171us] +/- 69ms 2024-09-13T17:59:52.338 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:59:52.340 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T17:59:52.340 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-09-13T17:59:52.340 INFO:teuthology.orchestra.run.smithi188.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 -618us[ -617us] +/- 52ms 2024-09-13T17:59:52.340 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 31 +1798us[+1798us] +/- 60ms 2024-09-13T17:59:52.340 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 -1161us[-1160us] +/- 69ms 2024-09-13T17:59:52.340 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:59:52.340 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T17:59:52.349 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T17:59:52.349 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T17:59:52.378 DEBUG:teuthology.orchestra.run.smithi111:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T17:59:52.455 DEBUG:teuthology.orchestra.run.smithi188:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T17:59:52.521 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T17:59:52.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:59:52.552 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T17:59:52.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:59:52.581 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T17:59:52.591 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T17:59:52.620 INFO:teuthology.task.internal:Duration was 519.809791 seconds 2024-09-13T17:59:52.621 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T17:59:52.653 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T17:59:52.653 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T17:59:52.656 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T17:59:52.694 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T17:59:52.696 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T17:59:53.056 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T17:59:53.056 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T17:59:53.057 DEBUG:teuthology.orchestra.run.smithi111:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T17:59:53.119 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2024-09-13T17:59:53.119 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T17:59:53.142 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T17:59:53.142 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:59:53.162 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:59:53.208 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T17:59:53.208 DEBUG:teuthology.orchestra.run.smithi111:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T17:59:53.230 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T17:59:53.351 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T17:59:53.367 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T17:59:53.368 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T17:59:53.393 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T17:59:53.418 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T17:59:53.429 DEBUG:teuthology.orchestra.run.smithi111:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:59:53.436 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:59:53.457 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = core 2024-09-13T17:59:53.481 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2024-09-13T17:59:53.495 DEBUG:teuthology.orchestra.run.smithi111:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:59:53.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:59:53.543 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:59:53.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:59:53.558 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T17:59:53.568 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T17:59:53.568 DEBUG:teuthology.misc:Transferring archived files from smithi111:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903942/remote/smithi111 2024-09-13T17:59:53.569 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T17:59:53.617 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903942/remote/smithi188 2024-09-13T17:59:53.617 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T17:59:53.651 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T17:59:53.652 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T17:59:53.655 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T17:59:53.703 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T17:59:53.714 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T17:59:53.714 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T17:59:53.738 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T17:59:53.738 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T17:59:53.741 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T17:59:53.753 INFO:teuthology.orchestra.run.smithi111.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 17:59 /home/ubuntu/cephtest 2024-09-13T17:59:53.757 INFO:teuthology.orchestra.run.smithi188.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 17:59 /home/ubuntu/cephtest 2024-09-13T17:59:53.758 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T17:59:53.770 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T17:59:53.843 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/cap-flush} duration: 519.8097906112671 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-13T17:59:53.843 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T17:59:53.964 INFO:teuthology.run:pass